Date   

CIP update 08 WK 03

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

this is the report from the activities that Codethink is driving on and or involved. This week is not that we worked more but that we completed more tasks.

++ Meetings

* Members meeting on Monday January 16th
** ELC booth/sponsorship logistics as main topic
** Report about testing and kernel maintenance efforts.

* Next Members meeting on Monday January 23rd
** Ben Hutchings attending to provide update about the kernel maintenance effort

++ Kernel maintenance

* 4.4 CIP kernel updated to the latest LTS.

* Backported patches being merged.

* Kernel maintenance policies under review.

The goal is to arrive to ELC with the policies finished and the backported patches merged. Some additional tasks might be done meanwhile.

++ Testing: Board @desk - Single Developer

* KernelCI Virtual Machine Setup & Configuration Howto published[1]
** You can follow other tasks related with the beta release in the ticket #10

* Improvements in the Kernel CI & LAVA v2 configuration.
** LAVA v2 Web Server now sees the devices that are available in the application #4
** LAVA v2 tests now complete #5
** QEMU health check are executing and does not get a KVM kernel module related error any more. #8 #9
** KernelCI VM now boots in Debian Jessie #12

* Beta release initially set for February 4th #10
** The plan at this point is to release right before ELC.

++ Other topics

* Robert Marshall has is no longer part of the team. Christos Karamitsos has substituted him. Chris is also based in Manchester. He will be focused in the issues we have in the kernelci front and back-en related with DDBB.
** Thanks Robert for your effort in pushing the development further. Welcome Christos. Please introduce yourself to the audience (when you recover from that flu).

* Ben H. and Don Brown has now edit access to the public wiki. If you have a Linux Foundation ID, please review if you have them too. If not, please let us know through this list. #11

* Agustin (and Yoshi) will attend to FOSDEM. What about you?

++ Coming activities

* Investigate DDBB related issues #2 and #3

* Consolidate the KernelCI & LAVA v2 servers into one VM #12

* Add documentation #10

* Preparation of the slides for the Open Source Leadership Summit (Agustin and Nori).

#2 https://gitlab.com/cip-project/testing/issues/2
#3 https://gitlab.com/cip-project/testing/issues/3
#4 https://gitlab.com/cip-project/testing/issues/4
#5 https://gitlab.com/cip-project/testing/issues/5
#8 https://gitlab.com/cip-project/testing/issues/8
#9 https://gitlab.com/cip-project/testing/issues/9
#10 https://gitlab.com/cip-project/testing/issues/10
#11 https://gitlab.com/cip-project/testing/issues/11
#12 https://gitlab.com/cip-project/testing/issues/12

[1] https://wiki.linuxfoundation.org/civilinfrastructureplatform/testinglavav2vmsetup

Best Regards

--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Re: [PATCH] Use local copy of kernel-ci if present

Daniel Wagner <daniel.wagner@...>
 

On 01/19/2017 04:14 PM, Wolfgang Mauerer wrote:
Am 19/01/2017 um 15:57 schrieb Daniel Wagner:
This is done by cloning all the remote repos to the project main
folder. This folder is shared with the VM. The intergration-scripts
are testing if those repos are in the shared folder than git clone
uses this as source.
I've had to read the last sentence for a couple of times before
I could make sense of it. Perhaps something along the lines of

'The integration scripts test if the repos are present in the
shared folder. They are used as source for "git clone" in this case.'
Good point. I lost myself in the depth of cloudy thoughts.


Re: [PATCH] Use local copy of kernel-ci if present

Wolfgang Mauerer <wm@...>
 

Am 19/01/2017 um 15:57 schrieb Daniel Wagner:
In order to develop or debug the setup we should allow to a developer
to use local copy of the sources.

This is done by cloning all the remote repos to the project main
folder. This folder is shared with the VM. The intergration-scripts
are testing if those repos are in the shared folder than git clone
uses this as source.
I've had to read the last sentence for a couple of times before
I could make sense of it. Perhaps something along the lines of

'The integration scripts test if the repos are present in the
shared folder. They are used as source for "git clone" in this case.'

To simplify the developer setup a simple setup-dev-env.sh script
is added. Note the current path should be the top folder of the
project.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
---
Hi,

This helped me to quite a bit to get things tested in a more simpler
way. What do you think?
the patch as such seems most helpful to me -- struggling with
modifications that do not render any effect because sources are pulled
from a differnet source have bitten me quite a few times when working
with kernelci.

Thanks, Wolfgang

cheers,
daniel

integration-scripts/install_backend.sh | 8 +++++++-
integration-scripts/install_build_script.sh | 9 ++++++++-
integration-scripts/install_frontend.sh | 8 +++++++-
scripts/setup-dev-env.sh | 11 +++++++++++
4 files changed, 33 insertions(+), 3 deletions(-)
create mode 100644 scripts/setup-dev-env.sh

diff --git a/integration-scripts/install_backend.sh b/integration-scripts/install_backend.sh
index 72a974033a3c..157e7184e349 100755
--- a/integration-scripts/install_backend.sh
+++ b/integration-scripts/install_backend.sh
@@ -4,7 +4,13 @@
# Install kernelci backend

cd $HOME && mkdir git-repos && cd git-repos
-git clone https://github.com/kernelci/kernelci-backend-config.git kernelci-backend
+
+GIT_SRC="https://github.com/kernelci/kernelci-backend-config.git"
+if [ -d /vagrant/kernelci-backend-config ]; then
+ GIT_SRC=/vagrant/kernelci-backend-config
+fi
+git clone $GIT_SRC kernelci-backend
+
cp /vagrant/config/secrets-backend.yml kernelci-backend/secrets.yml

# Fixme: Don't let ansible try to create the file in the first place.
diff --git a/integration-scripts/install_build_script.sh b/integration-scripts/install_build_script.sh
index 322619317af6..879aaed01792 100755
--- a/integration-scripts/install_build_script.sh
+++ b/integration-scripts/install_build_script.sh
@@ -3,7 +3,14 @@
# Copyright (C) 2016, Siemens AG, Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
# SPDX-License-Identifier: Apache-2.0

-cd $HOME && git clone https://github.com/kernelci/kernelci-build.git
+cd $HOME
+
+GIT_SRC="https://github.com/kernelci/kernelci-build.git"
+if [ -d /vagrant/kernelci-build ]; then
+ GIT_SRC=/vagrant/kernelci-build
+fi
+git clone $GIT_SRC
+
cd kernelci-build

MASTER_KEY=`cat $HOME/backend-admin-token.txt`
diff --git a/integration-scripts/install_frontend.sh b/integration-scripts/install_frontend.sh
index 48ab91ab83ce..251ef89d28f3 100755
--- a/integration-scripts/install_frontend.sh
+++ b/integration-scripts/install_frontend.sh
@@ -4,7 +4,13 @@
# Install kernelci frontend

cd $HOME/git-repos
-git clone https://github.com/kernelci/kernelci-frontend-config.git kernelci-frontend
+
+GIT_SRC="https://github.com/kernelci/kernelci-frontend-config.git"
+if [ -d /vagrant/kernelci-frontend-config ]; then
+ GIT_SRC=/vagrant/kernelci-frontend-config
+fi
+git clone $GIT_SRC kernelci-frontend
+
sed -i kernelci-frontend/roles/install-app/tasks/main.yml \
-e 's/kernelci\/kernelci-frontend.git/siemens\/kernelci-frontend.git/'

diff --git a/scripts/setup-dev-env.sh b/scripts/setup-dev-env.sh
new file mode 100644
index 000000000000..59c13b233065
--- /dev/null
+++ b/scripts/setup-dev-env.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+if [ ! -f "Vagrantfile" ]; then
+ echo "script is supposed to be run from the top folder where"
+ echo "the Vagrantfile is."
+ exit 1
+fi
+
+git clone https://github.com/kernelci/kernelci-backend-config.git
+git clone https://github.com/kernelci/kernelci-build.git
+git clone https://github.com/kernelci/kernelci-frontend-config.git


[PATCH] Use local copy of kernel-ci if present

Daniel Wagner <daniel.wagner@...>
 

In order to develop or debug the setup we should allow to a developer
to use local copy of the sources.

This is done by cloning all the remote repos to the project main
folder. This folder is shared with the VM. The intergration-scripts
are testing if those repos are in the shared folder than git clone
uses this as source.

To simplify the developer setup a simple setup-dev-env.sh script
is added. Note the current path should be the top folder of the
project.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
---
Hi,

This helped me to quite a bit to get things tested in a more simpler
way. What do you think?

cheers,
daniel

integration-scripts/install_backend.sh | 8 +++++++-
integration-scripts/install_build_script.sh | 9 ++++++++-
integration-scripts/install_frontend.sh | 8 +++++++-
scripts/setup-dev-env.sh | 11 +++++++++++
4 files changed, 33 insertions(+), 3 deletions(-)
create mode 100644 scripts/setup-dev-env.sh

diff --git a/integration-scripts/install_backend.sh b/integration-scripts/install_backend.sh
index 72a974033a3c..157e7184e349 100755
--- a/integration-scripts/install_backend.sh
+++ b/integration-scripts/install_backend.sh
@@ -4,7 +4,13 @@
# Install kernelci backend

cd $HOME && mkdir git-repos && cd git-repos
-git clone https://github.com/kernelci/kernelci-backend-config.git kernelci-backend
+
+GIT_SRC="https://github.com/kernelci/kernelci-backend-config.git"
+if [ -d /vagrant/kernelci-backend-config ]; then
+ GIT_SRC=/vagrant/kernelci-backend-config
+fi
+git clone $GIT_SRC kernelci-backend
+
cp /vagrant/config/secrets-backend.yml kernelci-backend/secrets.yml

# Fixme: Don't let ansible try to create the file in the first place.
diff --git a/integration-scripts/install_build_script.sh b/integration-scripts/install_build_script.sh
index 322619317af6..879aaed01792 100755
--- a/integration-scripts/install_build_script.sh
+++ b/integration-scripts/install_build_script.sh
@@ -3,7 +3,14 @@
# Copyright (C) 2016, Siemens AG, Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
# SPDX-License-Identifier: Apache-2.0

-cd $HOME && git clone https://github.com/kernelci/kernelci-build.git
+cd $HOME
+
+GIT_SRC="https://github.com/kernelci/kernelci-build.git"
+if [ -d /vagrant/kernelci-build ]; then
+ GIT_SRC=/vagrant/kernelci-build
+fi
+git clone $GIT_SRC
+
cd kernelci-build

MASTER_KEY=`cat $HOME/backend-admin-token.txt`
diff --git a/integration-scripts/install_frontend.sh b/integration-scripts/install_frontend.sh
index 48ab91ab83ce..251ef89d28f3 100755
--- a/integration-scripts/install_frontend.sh
+++ b/integration-scripts/install_frontend.sh
@@ -4,7 +4,13 @@
# Install kernelci frontend

cd $HOME/git-repos
-git clone https://github.com/kernelci/kernelci-frontend-config.git kernelci-frontend
+
+GIT_SRC="https://github.com/kernelci/kernelci-frontend-config.git"
+if [ -d /vagrant/kernelci-frontend-config ]; then
+ GIT_SRC=/vagrant/kernelci-frontend-config
+fi
+git clone $GIT_SRC kernelci-frontend
+
sed -i kernelci-frontend/roles/install-app/tasks/main.yml \
-e 's/kernelci\/kernelci-frontend.git/siemens\/kernelci-frontend.git/'

diff --git a/scripts/setup-dev-env.sh b/scripts/setup-dev-env.sh
new file mode 100644
index 000000000000..59c13b233065
--- /dev/null
+++ b/scripts/setup-dev-env.sh
@@ -0,0 +1,11 @@
+#!/bin/sh
+
+if [ ! -f "Vagrantfile" ]; then
+ echo "script is supposed to be run from the top folder where"
+ echo "the Vagrantfile is."
+ exit 1
+fi
+
+git clone https://github.com/kernelci/kernelci-backend-config.git
+git clone https://github.com/kernelci/kernelci-build.git
+git clone https://github.com/kernelci/kernelci-frontend-config.git
--
2.9.3


Re: kernelci setup fails

Don Brown <don.f.brown@...>
 

Hi Everyone,

I'm reporting on 2 issues today:
1. Re: kernelci setup fails
2. Re: Kernel-CI Frontend broken links to build.log et al.

-----
#1
I have fixed the issue of the Kernel-CI Virtual Machine failing to boot on the 'python3-setproctitle' and various other libraries.

You can read about it here: https://gitlab.com/cip-project/testing/issues/12

The TL;DR is that the kernelci_backend was broken between 12-Dec-2016 and the current version. I found that by using the 12-Dec-2016 commit, the backend booted up just fine.

So, we are back to where we were at the end of December where we can create builds and view them on the Web Frontend, but the links to the build.log, kernel.config, kernel image and dtbs are still broken

-----
#2
I have tracked down the general area where the links to the build.log, kernel.config, kernel image and dtbs are broken. I just need someone who knows the code to help fix it.


Sorry, there is no TL;DR for this one.


Sincerely,

Don Brown. PMP
don.f.brown@...
Mobile: (317) 560-0513
Here's to Life, Linux and the Pursuit of Happiness

On Tue, Jan 17, 2017 at 12:14 PM, <cip-dev-request@...> wrote:
Send cip-dev mailing list submissions to
        cip-dev@...

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.cip-project.org/mailman/listinfo/cip-dev
or, via email, send a message with subject or body 'help' to
        cip-dev-request@...project.org

You can reach the person managing the list at
        cip-dev-owner@...project.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of cip-dev digest..."


Today's Topics:

   1. Re: kernelci setup fails (Agustin Benito Bethencourt)
   2. Re: kernelci setup fails (Daniel Wagner)
   3. Re: kernelci setup fails (Don Brown)
   4. Re: kernelci setup fails (Daniel Wagner)
   5. Re: kernelci setup fails (Daniel Wagner)
   6. Re: kernelci setup fails (Agustin Benito Bethencourt)


----------------------------------------------------------------------

Message: 1
Date: Tue, 17 Jan 2017 15:05:00 +0100
From: Agustin Benito Bethencourt <agustin.benito@....uk>
To: cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID: <587E248C.2090002@codethink.co.uk>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi Daniel

On 17/01/17 09:49, Daniel Wagner wrote:
> Hi,
>
> running 'vagrant up' from a clean setup consistently fails with this:
>
>
> ==> default: TASK [setup] *******************************************************************
> ==> default: ok: [kernel-ci-backend]
> ==> default:
> ==> default: TASK [common : Tweak kernel configuration for performance] *********************
> ==> default: changed: [kernel-ci-backend]
> ==> default:
> ==> default: TASK [install-deps : Install OS dependencies] **********************************
> ==> default: failed: [kernel-ci-backend] (item=[u'build-essential', u'git', u'lsb-release', u'python-apt', u'python-pip', u'python-pycurl', u'python-virtualenv', u'python2.7-dev', u'sysfsutils', u'python3', u'python3-yaml', u'python3-setproctitle', u'python3-zmq']) => {"failed": true, "item": ["build-essential", "git", "lsb-release", "python-apt", "python-pip", "python-pycurl", "python-virtualenv", "python2.7-dev", "sysfsutils", "python3", "python3-yaml", "python3-setproctitle", "python3-zmq"], "msg": "No package matching 'python3-setproctitle' is available"}
>
>
> A quick search for setproctitle tells me it is installed via
>
>       kernelci-backend-config/roles/install-deps/tasks/main.yml:    - python3-setproctitle
>
> But ubuntu doesn't seem to know this package:
>
> vagrant@vagrant-ubuntu-trusty-64:~$ apt-cache search setproctitle
> libbsd-dev - utility functions from BSD systems - development files
> python-setproctitle - A setproctitle implementation for Python
> python-setproctitle-dbg - A setproctitle implementation for Python
>
>
> I haven't spend time to figure out what this package is doing or
> how it used. So my quick question how to fix this?

maybe this helps?

https://gitlab.com/cip-project/testing/issues/12

If this is your case, Don is on it. He talked to one of the kernelci
developers, Milo Casagrande about it.

>
> cheers,
> daniel
> _______________________________________________
> cip-dev mailing list
> cip-dev@...
> https://lists.cip-project.org/mailman/listinfo/cip-dev
>

--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@...


------------------------------

Message: 2
Date: Tue, 17 Jan 2017 15:43:46 +0100
From: Daniel Wagner <daniel.wagner@...>
To: Agustin Benito Bethencourt <agustin.benito@....uk>,
        cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID: <e50a3001-d769-5a28-7dde-0850f5247324@...>
Content-Type: text/plain; charset=windows-1252

Hi Agustin,

On 01/17/2017 03:05 PM, Agustin Benito Bethencourt wrote:
> maybe this helps?
>
> https://gitlab.com/cip-project/testing/issues/12
>
> If this is your case, Don is on it. He talked to one of the kernelci
> developers, Milo Casagrande about it.

Indeed the second bug report in the ticket is the problem I see too.

http://packages.ubuntu.com/search?keywords=python3-setproctitle

    xenial (16.04LTS) (python): A setproctitle implementation for Python
(Python 3) [universe]
    1.1.8-1build2: amd64 arm64 armhf i386 powerpc ppc64el s390x
    yakkety (16.10) (python): A setproctitle implementation for Python
(Python 3)
    1.1.8-1ubuntu2: amd64 arm64 armhf i386 powerpc ppc64el s390x
    zesty (python): Setproctitle implementation for Python 3
    1.1.10-1: amd64 arm64 armhf i386 powerpc ppc64el s390x

So this package is just not part of the trusty version. Is there a
specific reason to stick to trusty?

Thanks,
Daniel


------------------------------

Message: 3
Date: Tue, 17 Jan 2017 11:04:04 -0500
From: Don Brown <don.f.brown@...>
To: cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID:
        <CANm2L4=-ZamtbxhvuY7wVRBja3P1YiL0cGkgVb_FPrFciFS3nA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Daniel,

I have the same problem and worked on it yesterday from several angles. My
notes are logged in Issue #12 in the GitLab repo. The link is:

https://gitlab.com/cip-project/testing/issues/12


Sincerely,

Don Brown. PMP
don.f.brown@...
Mobile: (317) 560-0513
Here's to Life, Linux and the Pursuit of Happiness

On Tue, Jan 17, 2017 at 7:00 AM, <cip-dev-request@...project.org>
wrote:

> Send cip-dev mailing list submissions to
>         cip-dev@...
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.cip-project.org/mailman/listinfo/cip-dev
> or, via email, send a message with subject or body 'help' to
>         cip-dev-request@...project.org
>
> You can reach the person managing the list at
>         cip-dev-owner@...project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of cip-dev digest..."
>
>
> Today's Topics:
>
>    1. kernelci setup fails (Daniel Wagner)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 17 Jan 2017 09:49:37 +0100
> From: Daniel Wagner <wagi@...>
> To: cip-dev@...
> Subject: [cip-dev] kernelci setup fails
> Message-ID: <f6c5c137-3fc9-c175-4889-5cf16480f167@...>
> Content-Type: text/plain; charset=utf-8
>
> Hi,
>
> running 'vagrant up' from a clean setup consistently fails with this:
>
>
> ==> default: TASK [setup] ******************************
> *************************************
> ==> default: ok: [kernel-ci-backend]
> ==> default:
> ==> default: TASK [common : Tweak kernel configuration for performance]
> *********************
> ==> default: changed: [kernel-ci-backend]
> ==> default:
> ==> default: TASK [install-deps : Install OS dependencies]
> **********************************
> ==> default: failed: [kernel-ci-backend] (item=[u'build-essential',
> u'git', u'lsb-release', u'python-apt', u'python-pip', u'python-pycurl',
> u'python-virtualenv', u'python2.7-dev', u'sysfsutils', u'python3',
> u'python3-yaml', u'python3-setproctitle', u'python3-zmq']) => {"failed":
> true, "item": ["build-essential", "git", "lsb-release", "python-apt",
> "python-pip", "python-pycurl", "python-virtualenv", "python2.7-dev",
> "sysfsutils", "python3", "python3-yaml", "python3-setproctitle",
> "python3-zmq"], "msg": "No package matching 'python3-setproctitle' is
> available"}
>
>
> A quick search for setproctitle tells me it is installed via
>
>         kernelci-backend-config/roles/install-deps/tasks/main.yml:    -
> python3-setproctitle
>
> But ubuntu doesn't seem to know this package:
>
> vagrant@vagrant-ubuntu-trusty-64:~$ apt-cache search setproctitle
> libbsd-dev - utility functions from BSD systems - development files
> python-setproctitle - A setproctitle implementation for Python
> python-setproctitle-dbg - A setproctitle implementation for Python
>
>
> I haven't spend time to figure out what this package is doing or
> how it used. So my quick question how to fix this?
>
> cheers,
> daniel
>
>
> ------------------------------
>
> _______________________________________________
> cip-dev mailing list
> cip-dev@...
> https://lists.cip-project.org/mailman/listinfo/cip-dev
>
>
> End of cip-dev Digest, Vol 8, Issue 6
> *************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20170117/01ad92a1/attachment-0001.html>

------------------------------

Message: 4
Date: Tue, 17 Jan 2017 17:32:18 +0100
From: Daniel Wagner <daniel.wagner@...>
To: Don Brown <don.f.brown@...>, cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID: <bf0051c6-fa6f-3ed0-a578-b1cc5e9142bc@...>
Content-Type: text/plain; charset=windows-1252

Hi Don,

On 01/17/2017 05:04 PM, Don Brown wrote:
> I have the same problem and worked on it yesterday from several angles.
> My notes are logged in Issue #12 in the GitLab repo. The link is:
>
> https://gitlab.com/cip-project/testing/issues/12

I'll create a gitlab account soon, so that I can collaborate via the
ticket system.

Thanks.
Daniel


------------------------------

Message: 5
Date: Tue, 17 Jan 2017 17:36:54 +0100
From: Daniel Wagner <daniel.wagner@...>
To: Agustin Benito Bethencourt <agustin.benito@....uk>
Cc: cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID: <9cf9f75f-30bd-9be5-cbe2-4ca1baa6d193@...>
Content-Type: text/plain; charset=windows-1252

On 01/17/2017 03:43 PM, Daniel Wagner wrote:
> http://packages.ubuntu.com/search?keywords=python3-setproctitle
>
>     xenial (16.04LTS) (python): A setproctitle implementation for Python
> (Python 3) [universe]
>     1.1.8-1build2: amd64 arm64 armhf i386 powerpc ppc64el s390x
>     yakkety (16.10) (python): A setproctitle implementation for Python
> (Python 3)
>     1.1.8-1ubuntu2: amd64 arm64 armhf i386 powerpc ppc64el s390x
>     zesty (python): Setproctitle implementation for Python 3
>     1.1.10-1: amd64 arm64 armhf i386 powerpc ppc64el s390x
>
> So this package is just not part of the trusty version. Is there a
> specific reason to stick to trusty?

I gave it a try, no luck though:

==> default: TASK [install-deps : Install MongoDB]
******************************************
==> default: failed: [kernel-ci-backend] (item=[u'mongodb-org',
u'libsnappy1']) => {"failed": true, "item": ["mongodb-org",
"libsnappy1"], "msg": "No package matching 'libsnappy1' is available"}


So I am stuck at the same level as Don is. Anyway, I was wondering how
to be able to have local copies of the kernelci code and use that one as
source instead of the hard coded git urls. It seems rather hard to work
on the setup if you can't change it :)

What about having an option which mounts the local kernelci repos into
the virtual machine? Obviously, the kernelci ansible script would also
need to be modified to allow overwrites.

Thanks,
Daniel


------------------------------

Message: 6
Date: Tue, 17 Jan 2017 18:14:11 +0100
From: Agustin Benito Bethencourt <agustin.benito@....uk>
To: cip-dev@...
Subject: Re: [cip-dev] kernelci setup fails
Message-ID: <587E50E3.3080202@codethink.co.uk>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi,

On 17/01/17 17:32, Daniel Wagner wrote:
> Hi Don,
>
> On 01/17/2017 05:04 PM, Don Brown wrote:
>> I have the same problem and worked on it yesterday from several angles.
>> My notes are logged in Issue #12 in the GitLab repo. The link is:
>>
>> https://gitlab.com/cip-project/testing/issues/12
>
> I'll create a gitlab account soon, so that I can collaborate via the
> ticket system.

until then...

We have discussed what to do. There are several options:
1 Make Python 3.5.2 work on trusty.
2 Move on to Ubuntu 16.04
3 Move to an in-between version.
4 Try to move it into Debian so we can go for a single VM for the whole
kernelci tolling.

Don is describing in the gitlab ticket his early tries of each option to
determine which way to go. Although #4 is the desired one, it seems the
most difficult one as well, so we decided to focus on the initial three.

We want to have the system working first and then we will figure out how
we can simplify the deployment.

Best Regards

--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@...


------------------------------

_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


End of cip-dev Digest, Vol 8, Issue 7
*************************************


Re: kernelci setup fails

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 17/01/17 17:32, Daniel Wagner wrote:
Hi Don,

On 01/17/2017 05:04 PM, Don Brown wrote:
I have the same problem and worked on it yesterday from several angles.
My notes are logged in Issue #12 in the GitLab repo. The link is:

https://gitlab.com/cip-project/testing/issues/12
I'll create a gitlab account soon, so that I can collaborate via the
ticket system.
until then...

We have discussed what to do. There are several options:
1 Make Python 3.5.2 work on trusty.
2 Move on to Ubuntu 16.04
3 Move to an in-between version.
4 Try to move it into Debian so we can go for a single VM for the whole kernelci tolling.

Don is describing in the gitlab ticket his early tries of each option to determine which way to go. Although #4 is the desired one, it seems the most difficult one as well, so we decided to focus on the initial three.

We want to have the system working first and then we will figure out how we can simplify the deployment.

Best Regards

--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Re: kernelci setup fails

Daniel Wagner <daniel.wagner@...>
 

On 01/17/2017 03:43 PM, Daniel Wagner wrote:
http://packages.ubuntu.com/search?keywords=python3-setproctitle

xenial (16.04LTS) (python): A setproctitle implementation for Python
(Python 3) [universe]
1.1.8-1build2: amd64 arm64 armhf i386 powerpc ppc64el s390x
yakkety (16.10) (python): A setproctitle implementation for Python
(Python 3)
1.1.8-1ubuntu2: amd64 arm64 armhf i386 powerpc ppc64el s390x
zesty (python): Setproctitle implementation for Python 3
1.1.10-1: amd64 arm64 armhf i386 powerpc ppc64el s390x

So this package is just not part of the trusty version. Is there a
specific reason to stick to trusty?
I gave it a try, no luck though:

==> default: TASK [install-deps : Install MongoDB]
******************************************
==> default: failed: [kernel-ci-backend] (item=[u'mongodb-org',
u'libsnappy1']) => {"failed": true, "item": ["mongodb-org",
"libsnappy1"], "msg": "No package matching 'libsnappy1' is available"}


So I am stuck at the same level as Don is. Anyway, I was wondering how
to be able to have local copies of the kernelci code and use that one as
source instead of the hard coded git urls. It seems rather hard to work
on the setup if you can't change it :)

What about having an option which mounts the local kernelci repos into
the virtual machine? Obviously, the kernelci ansible script would also
need to be modified to allow overwrites.

Thanks,
Daniel


Re: kernelci setup fails

Daniel Wagner <daniel.wagner@...>
 

Hi Don,

On 01/17/2017 05:04 PM, Don Brown wrote:
I have the same problem and worked on it yesterday from several angles.
My notes are logged in Issue #12 in the GitLab repo. The link is:

https://gitlab.com/cip-project/testing/issues/12
I'll create a gitlab account soon, so that I can collaborate via the
ticket system.

Thanks.
Daniel


Re: kernelci setup fails

Don Brown <don.f.brown@...>
 

Hi Daniel,

I have the same problem and worked on it yesterday from several angles. My notes are logged in Issue #12 in the GitLab repo. The link is:



Sincerely,

Don Brown. PMP
don.f.brown@...
Mobile: (317) 560-0513
Here's to Life, Linux and the Pursuit of Happiness

On Tue, Jan 17, 2017 at 7:00 AM, <cip-dev-request@...> wrote:
Send cip-dev mailing list submissions to
        cip-dev@...

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.cip-project.org/mailman/listinfo/cip-dev
or, via email, send a message with subject or body 'help' to
        cip-dev-request@...project.org

You can reach the person managing the list at
        cip-dev-owner@...project.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of cip-dev digest..."


Today's Topics:

   1. kernelci setup fails (Daniel Wagner)


----------------------------------------------------------------------

Message: 1
Date: Tue, 17 Jan 2017 09:49:37 +0100
From: Daniel Wagner <wagi@...>
To: cip-dev@...
Subject: [cip-dev] kernelci setup fails
Message-ID: <f6c5c137-3fc9-c175-4889-5cf16480f167@...>
Content-Type: text/plain; charset=utf-8

Hi,

running 'vagrant up' from a clean setup consistently fails with this:


==> default: TASK [setup] *******************************************************************
==> default: ok: [kernel-ci-backend]
==> default:
==> default: TASK [common : Tweak kernel configuration for performance] *********************
==> default: changed: [kernel-ci-backend]
==> default:
==> default: TASK [install-deps : Install OS dependencies] **********************************
==> default: failed: [kernel-ci-backend] (item=[u'build-essential', u'git', u'lsb-release', u'python-apt', u'python-pip', u'python-pycurl', u'python-virtualenv', u'python2.7-dev', u'sysfsutils', u'python3', u'python3-yaml', u'python3-setproctitle', u'python3-zmq']) => {"failed": true, "item": ["build-essential", "git", "lsb-release", "python-apt", "python-pip", "python-pycurl", "python-virtualenv", "python2.7-dev", "sysfsutils", "python3", "python3-yaml", "python3-setproctitle", "python3-zmq"], "msg": "No package matching 'python3-setproctitle' is available"}


A quick search for setproctitle tells me it is installed via

        kernelci-backend-config/roles/install-deps/tasks/main.yml:    - python3-setproctitle

But ubuntu doesn't seem to know this package:

vagrant@vagrant-ubuntu-trusty-64:~$ apt-cache search setproctitle
libbsd-dev - utility functions from BSD systems - development files
python-setproctitle - A setproctitle implementation for Python
python-setproctitle-dbg - A setproctitle implementation for Python


I haven't spend time to figure out what this package is doing or
how it used. So my quick question how to fix this?

cheers,
daniel


------------------------------

_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


End of cip-dev Digest, Vol 8, Issue 6
*************************************


Re: kernelci setup fails

Daniel Wagner <daniel.wagner@...>
 

Hi Agustin,

On 01/17/2017 03:05 PM, Agustin Benito Bethencourt wrote:
maybe this helps?

https://gitlab.com/cip-project/testing/issues/12

If this is your case, Don is on it. He talked to one of the kernelci
developers, Milo Casagrande about it.
Indeed the second bug report in the ticket is the problem I see too.

http://packages.ubuntu.com/search?keywords=python3-setproctitle

xenial (16.04LTS) (python): A setproctitle implementation for Python
(Python 3) [universe]
1.1.8-1build2: amd64 arm64 armhf i386 powerpc ppc64el s390x
yakkety (16.10) (python): A setproctitle implementation for Python
(Python 3)
1.1.8-1ubuntu2: amd64 arm64 armhf i386 powerpc ppc64el s390x
zesty (python): Setproctitle implementation for Python 3
1.1.10-1: amd64 arm64 armhf i386 powerpc ppc64el s390x

So this package is just not part of the trusty version. Is there a
specific reason to stick to trusty?

Thanks,
Daniel


Re: kernelci setup fails

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi Daniel

On 17/01/17 09:49, Daniel Wagner wrote:
Hi,

running 'vagrant up' from a clean setup consistently fails with this:


==> default: TASK [setup] *******************************************************************
==> default: ok: [kernel-ci-backend]
==> default:
==> default: TASK [common : Tweak kernel configuration for performance] *********************
==> default: changed: [kernel-ci-backend]
==> default:
==> default: TASK [install-deps : Install OS dependencies] **********************************
==> default: failed: [kernel-ci-backend] (item=[u'build-essential', u'git', u'lsb-release', u'python-apt', u'python-pip', u'python-pycurl', u'python-virtualenv', u'python2.7-dev', u'sysfsutils', u'python3', u'python3-yaml', u'python3-setproctitle', u'python3-zmq']) => {"failed": true, "item": ["build-essential", "git", "lsb-release", "python-apt", "python-pip", "python-pycurl", "python-virtualenv", "python2.7-dev", "sysfsutils", "python3", "python3-yaml", "python3-setproctitle", "python3-zmq"], "msg": "No package matching 'python3-setproctitle' is available"}


A quick search for setproctitle tells me it is installed via

kernelci-backend-config/roles/install-deps/tasks/main.yml: - python3-setproctitle

But ubuntu doesn't seem to know this package:

vagrant@vagrant-ubuntu-trusty-64:~$ apt-cache search setproctitle
libbsd-dev - utility functions from BSD systems - development files
python-setproctitle - A setproctitle implementation for Python
python-setproctitle-dbg - A setproctitle implementation for Python


I haven't spend time to figure out what this package is doing or
how it used. So my quick question how to fix this?
maybe this helps?

https://gitlab.com/cip-project/testing/issues/12

If this is your case, Don is on it. He talked to one of the kernelci developers, Milo Casagrande about it.


cheers,
daniel
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


kernelci setup fails

Daniel Wagner <wagi@...>
 

Hi,

running 'vagrant up' from a clean setup consistently fails with this:


==> default: TASK [setup] *******************************************************************
==> default: ok: [kernel-ci-backend]
==> default:
==> default: TASK [common : Tweak kernel configuration for performance] *********************
==> default: changed: [kernel-ci-backend]
==> default:
==> default: TASK [install-deps : Install OS dependencies] **********************************
==> default: failed: [kernel-ci-backend] (item=[u'build-essential', u'git', u'lsb-release', u'python-apt', u'python-pip', u'python-pycurl', u'python-virtualenv', u'python2.7-dev', u'sysfsutils', u'python3', u'python3-yaml', u'python3-setproctitle', u'python3-zmq']) => {"failed": true, "item": ["build-essential", "git", "lsb-release", "python-apt", "python-pip", "python-pycurl", "python-virtualenv", "python2.7-dev", "sysfsutils", "python3", "python3-yaml", "python3-setproctitle", "python3-zmq"], "msg": "No package matching 'python3-setproctitle' is available"}


A quick search for setproctitle tells me it is installed via

kernelci-backend-config/roles/install-deps/tasks/main.yml: - python3-setproctitle

But ubuntu doesn't seem to know this package:

vagrant@vagrant-ubuntu-trusty-64:~$ apt-cache search setproctitle
libbsd-dev - utility functions from BSD systems - development files
python-setproctitle - A setproctitle implementation for Python
python-setproctitle-dbg - A setproctitle implementation for Python


I haven't spend time to figure out what this package is doing or
how it used. So my quick question how to fix this?

cheers,
daniel


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 16/01/17 10:35, Agustin Benito Bethencourt wrote:
Hi,

On 08/12/16 23:56, Ben Hutchings wrote:
This is a backport of changes in 4.5 to extend the range of Address
Space Layout Randomisation for user-space processes. When enabled, this
should make some user-space vulnerabilities harder to exploit, but it
can also cause some applications to fail if they currently use a large
proportion of the virtual address space.

The default ASLR range remains the same, but it can be changed through
kernel config (CONFIG_ARCH_MMAP_RND_BITS) or at run-time through sysctl
(vm.mmap_rnd_bits). (For 32-bit compat tasks, the range is controlled
through CONFIG_ARCH_MMAP_RND_COMPAT_BITS and vm.mmap_rnd_compat_bits.)

This includes support for arm, arm64 and x86 (32- and 64-bit). (arm64
is not currently supported by CIP, but it was easier to include it in
the backport than to leave it out.)

For this and other backports, I'm looking for feedback like:
- Did I miss a follow-up fix or an earlier dependency?
- Does this cause a regression (other than as explained above)?
- Are you likely to use it?
- Are there related features you want in 4.4?
since there is no further feedback, I assume you me merge the patches,
isn't is?
since there is no further feedback, I assume you will merge the patches, isn't is?


Hopefully in a couple or three more weeks we can start testing it with
kernelci tooling.


Ben.

Daniel Cashman (6):
mm: mmap: add new /proc tunable for mmap_base ASLR
arm: mm: support ARCH_MMAP_RND_BITS
arm64: mm: support ARCH_MMAP_RND_BITS
x86: mm: support ARCH_MMAP_RND_BITS
drivers: char: random: add get_random_long()
mm: ASLR: use get_random_long()

Documentation/sysctl/vm.txt | 29 +++++++++++++++++
arch/Kconfig | 68
++++++++++++++++++++++++++++++++++++++++
arch/arm/Kconfig | 9 ++++++
arch/arm/mm/mmap.c | 3 +-
arch/arm64/Kconfig | 29 +++++++++++++++++
arch/arm64/mm/mmap.c | 8 +++--
arch/mips/mm/mmap.c | 4 +--
arch/powerpc/kernel/process.c | 4 +--
arch/powerpc/mm/mmap.c | 4 +--
arch/sparc/kernel/sys_sparc_64.c | 2 +-
arch/x86/Kconfig | 16 ++++++++++
arch/x86/mm/mmap.c | 12 +++----
drivers/char/random.c | 22 +++++++++++++
fs/binfmt_elf.c | 2 +-
include/linux/mm.h | 11 +++++++
include/linux/random.h | 1 +
kernel/sysctl.c | 22 +++++++++++++
mm/mmap.c | 12 +++++++
18 files changed, 240 insertions(+), 18 deletions(-)
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 08/12/16 23:56, Ben Hutchings wrote:
This is a backport of changes in 4.5 to extend the range of Address
Space Layout Randomisation for user-space processes. When enabled, this
should make some user-space vulnerabilities harder to exploit, but it
can also cause some applications to fail if they currently use a large
proportion of the virtual address space.

The default ASLR range remains the same, but it can be changed through
kernel config (CONFIG_ARCH_MMAP_RND_BITS) or at run-time through sysctl
(vm.mmap_rnd_bits). (For 32-bit compat tasks, the range is controlled
through CONFIG_ARCH_MMAP_RND_COMPAT_BITS and vm.mmap_rnd_compat_bits.)

This includes support for arm, arm64 and x86 (32- and 64-bit). (arm64
is not currently supported by CIP, but it was easier to include it in
the backport than to leave it out.)

For this and other backports, I'm looking for feedback like:
- Did I miss a follow-up fix or an earlier dependency?
- Does this cause a regression (other than as explained above)?
- Are you likely to use it?
- Are there related features you want in 4.4?
since there is no further feedback, I assume you me merge the patches, isn't is?

Hopefully in a couple or three more weeks we can start testing it with kernelci tooling.


Ben.

Daniel Cashman (6):
mm: mmap: add new /proc tunable for mmap_base ASLR
arm: mm: support ARCH_MMAP_RND_BITS
arm64: mm: support ARCH_MMAP_RND_BITS
x86: mm: support ARCH_MMAP_RND_BITS
drivers: char: random: add get_random_long()
mm: ASLR: use get_random_long()

Documentation/sysctl/vm.txt | 29 +++++++++++++++++
arch/Kconfig | 68 ++++++++++++++++++++++++++++++++++++++++
arch/arm/Kconfig | 9 ++++++
arch/arm/mm/mmap.c | 3 +-
arch/arm64/Kconfig | 29 +++++++++++++++++
arch/arm64/mm/mmap.c | 8 +++--
arch/mips/mm/mmap.c | 4 +--
arch/powerpc/kernel/process.c | 4 +--
arch/powerpc/mm/mmap.c | 4 +--
arch/sparc/kernel/sys_sparc_64.c | 2 +-
arch/x86/Kconfig | 16 ++++++++++
arch/x86/mm/mmap.c | 12 +++----
drivers/char/random.c | 22 +++++++++++++
fs/binfmt_elf.c | 2 +-
include/linux/mm.h | 11 +++++++
include/linux/random.h | 1 +
kernel/sysctl.c | 22 +++++++++++++
mm/mmap.c | 12 +++++++
18 files changed, 240 insertions(+), 18 deletions(-)
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Ben Hutchings <ben.hutchings@...>
 

On Tue, 2017-01-03 at 15:56 -0800, Kees Cook wrote:
On Mon, Dec 19, 2016 at 2:52 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
On 2016-12-09 13:20, Jan Kiszka wrote:
[...]
Did you try to discuss the back-port topic with the KSPP folks or other
key persons involved in these patches? In the ideal case, the authors
can be CC'ed, do not get annoyed by "these crazy people doing legacy
stuff", and may even do some reviews.
I've chatted with Elena over this last week, and she talked to Kees who
pointed out that the Android people are also doing KSPP backports to 4.4
(thanks, folks!). I didn't check any details, just a heads-up to avoid
duplicate work.
Hi!

The Android common kernel tree is visible here:

https://android.googlesource.com/kernel/common/

In the android-4.4 branch, the backport are these:

b471fcd FROMLIST: mm: ASLR: use get_random_long()
9a3fe39 FROMLIST: drivers: char: random: add get_random_long()
d51891f FROMLIST: x86: mm: support ARCH_MMAP_RND_BITS.
e2240a1 FROMLIST: arm64: mm: support ARCH_MMAP_RND_BITS.
25106ff FROMLIST: arm: mm: support ARCH_MMAP_RND_BITS.
d49d887 FROMLIST: mm: mmap: Add new /proc tunable for mmap_base ASLR.

Hopefully that helps!
Thanks. My backports are identical aside from the placement of some
changes in Kconfig files, which shouldn't make a functional difference.

Ben.

--
Ben Hutchings
Software Developer, Codethink Ltd.


Re: CIP update 07 WK 01

Robert Marshall <robert.marshall@...>
 

Apologies, the following item was missed out of the report:

* build artifacts - we're still unable to retrieve these and get
404 errors on selecting the link:
https://gitlab.com/cip-project/testing/issues/2


Robert Marshall <robert.marshall@codethink.co.uk> writes:

* Virtual machine creation for the testing
We've created 2 vagrant instances one for kernelci and the other for lava
(lava is Debian based and kernelci is ubuntu based) work is in
progress on merging these to make developer setup simpler and reduce
the memory requirements.
https://gitlab.com/cip-project/testing/issues/7

* There's a github branch to allow the host to view the kernelci vm webserver
https://github.com/RobertAJMarshall/kernelci-frontend-config/tree/external-http
this will be merged with the gitlab repository when
https://gitlab.com/cip-project/testing/issues/3
has been resolved.

* The DeviceDictionary error
https://gitlab.com/cip-project/testing/issues/5
is still outstanding

* Devices have been added to lava VM but are not yet running
https://gitlab.com/cip-project/testing/issues/9

* the lava gitlab and the vagrant instance now has been updated
with health check tests and extra documentation.

* The QEMU health check results in a KVM module not found failure
https://gitlab.com/cip-project/testing/issues/8

* Discussions in progress on adding a 'howto'
The developers have a document enumerating the steps involved in
getting the testing environment up and running, this document will
be streamlined and published to the wiki. Meanwhile, we will add the
WIP content to the repository so you can follow it.

* Minor additions to the wiki has been made, specially in the kernel
maintenance page.

Robert
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev


CIP update 07 WK 01

Robert Marshall <robert.marshall@...>
 

* Virtual machine creation for the testing
We've created 2 vagrant instances one for kernelci and the other for lava
(lava is Debian based and kernelci is ubuntu based) work is in
progress on merging these to make developer setup simpler and reduce
the memory requirements.
https://gitlab.com/cip-project/testing/issues/7

* There's a github branch to allow the host to view the kernelci vm webserver
https://github.com/RobertAJMarshall/kernelci-frontend-config/tree/external-http
this will be merged with the gitlab repository when
https://gitlab.com/cip-project/testing/issues/3
has been resolved.

* The DeviceDictionary error
https://gitlab.com/cip-project/testing/issues/5
is still outstanding

* Devices have been added to lava VM but are not yet running
https://gitlab.com/cip-project/testing/issues/9

* the lava gitlab and the vagrant instance now has been updated
with health check tests and extra documentation.

* The QEMU health check results in a KVM module not found failure
https://gitlab.com/cip-project/testing/issues/8

* Discussions in progress on adding a 'howto'
The developers have a document enumerating the steps involved in
getting the testing environment up and running, this document will
be streamlined and published to the wiki. Meanwhile, we will add the
WIP content to the repository so you can follow it.

* Minor additions to the wiki has been made, specially in the kernel
maintenance page.

Robert


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 03/01/17 23:56, Kees Cook wrote:
On Mon, Dec 19, 2016 at 2:52 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
On 2016-12-09 13:20, Jan Kiszka wrote:
On 2016-12-09 00:56, Ben Hutchings wrote:
This is a backport of changes in 4.5 to extend the range of Address
Space Layout Randomisation for user-space processes. When enabled, this
should make some user-space vulnerabilities harder to exploit, but it
can also cause some applications to fail if they currently use a large
proportion of the virtual address space.

The default ASLR range remains the same, but it can be changed through
kernel config (CONFIG_ARCH_MMAP_RND_BITS) or at run-time through sysctl
(vm.mmap_rnd_bits). (For 32-bit compat tasks, the range is controlled
through CONFIG_ARCH_MMAP_RND_COMPAT_BITS and vm.mmap_rnd_compat_bits.)

This includes support for arm, arm64 and x86 (32- and 64-bit). (arm64
is not currently supported by CIP, but it was easier to include it in
the backport than to leave it out.)

For this and other backports, I'm looking for feedback like:
- Did I miss a follow-up fix or an earlier dependency?
- Does this cause a regression (other than as explained above)?
- Are you likely to use it?
- Are there related features you want in 4.4?

Ben.

Daniel Cashman (6):
mm: mmap: add new /proc tunable for mmap_base ASLR
arm: mm: support ARCH_MMAP_RND_BITS
arm64: mm: support ARCH_MMAP_RND_BITS
x86: mm: support ARCH_MMAP_RND_BITS
drivers: char: random: add get_random_long()
mm: ASLR: use get_random_long()

Documentation/sysctl/vm.txt | 29 +++++++++++++++++
arch/Kconfig | 68 ++++++++++++++++++++++++++++++++++++++++
arch/arm/Kconfig | 9 ++++++
arch/arm/mm/mmap.c | 3 +-
arch/arm64/Kconfig | 29 +++++++++++++++++
arch/arm64/mm/mmap.c | 8 +++--
arch/mips/mm/mmap.c | 4 +--
arch/powerpc/kernel/process.c | 4 +--
arch/powerpc/mm/mmap.c | 4 +--
arch/sparc/kernel/sys_sparc_64.c | 2 +-
arch/x86/Kconfig | 16 ++++++++++
arch/x86/mm/mmap.c | 12 +++----
drivers/char/random.c | 22 +++++++++++++
fs/binfmt_elf.c | 2 +-
include/linux/mm.h | 11 +++++++
include/linux/random.h | 1 +
kernel/sysctl.c | 22 +++++++++++++
mm/mmap.c | 12 +++++++
18 files changed, 240 insertions(+), 18 deletions(-)
Did you try to discuss the back-port topic with the KSPP folks or other
key persons involved in these patches? In the ideal case, the authors
can be CC'ed, do not get annoyed by "these crazy people doing legacy
stuff", and may even do some reviews.
I've chatted with Elena over this last week, and she talked to Kees who
pointed out that the Android people are also doing KSPP backports to 4.4
(thanks, folks!). I didn't check any details, just a heads-up to avoid
duplicate work.
Hi!

The Android common kernel tree is visible here:

https://android.googlesource.com/kernel/common/

In the android-4.4 branch, the backport are these:

b471fcd FROMLIST: mm: ASLR: use get_random_long()
9a3fe39 FROMLIST: drivers: char: random: add get_random_long()
d51891f FROMLIST: x86: mm: support ARCH_MMAP_RND_BITS.
e2240a1 FROMLIST: arm64: mm: support ARCH_MMAP_RND_BITS.
25106ff FROMLIST: arm: mm: support ARCH_MMAP_RND_BITS.
d49d887 FROMLIST: mm: mmap: Add new /proc tunable for mmap_base ASLR.

Hopefully that helps!
It does, thanks. I sent Dmitry Shmidt a mail this morning asking for this. Thanks.


-Kees


Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Kees Cook <keescook@...>
 

On Mon, Dec 19, 2016 at 2:52 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
On 2016-12-09 13:20, Jan Kiszka wrote:
On 2016-12-09 00:56, Ben Hutchings wrote:
This is a backport of changes in 4.5 to extend the range of Address
Space Layout Randomisation for user-space processes. When enabled, this
should make some user-space vulnerabilities harder to exploit, but it
can also cause some applications to fail if they currently use a large
proportion of the virtual address space.

The default ASLR range remains the same, but it can be changed through
kernel config (CONFIG_ARCH_MMAP_RND_BITS) or at run-time through sysctl
(vm.mmap_rnd_bits). (For 32-bit compat tasks, the range is controlled
through CONFIG_ARCH_MMAP_RND_COMPAT_BITS and vm.mmap_rnd_compat_bits.)

This includes support for arm, arm64 and x86 (32- and 64-bit). (arm64
is not currently supported by CIP, but it was easier to include it in
the backport than to leave it out.)

For this and other backports, I'm looking for feedback like:
- Did I miss a follow-up fix or an earlier dependency?
- Does this cause a regression (other than as explained above)?
- Are you likely to use it?
- Are there related features you want in 4.4?

Ben.

Daniel Cashman (6):
mm: mmap: add new /proc tunable for mmap_base ASLR
arm: mm: support ARCH_MMAP_RND_BITS
arm64: mm: support ARCH_MMAP_RND_BITS
x86: mm: support ARCH_MMAP_RND_BITS
drivers: char: random: add get_random_long()
mm: ASLR: use get_random_long()

Documentation/sysctl/vm.txt | 29 +++++++++++++++++
arch/Kconfig | 68 ++++++++++++++++++++++++++++++++++++++++
arch/arm/Kconfig | 9 ++++++
arch/arm/mm/mmap.c | 3 +-
arch/arm64/Kconfig | 29 +++++++++++++++++
arch/arm64/mm/mmap.c | 8 +++--
arch/mips/mm/mmap.c | 4 +--
arch/powerpc/kernel/process.c | 4 +--
arch/powerpc/mm/mmap.c | 4 +--
arch/sparc/kernel/sys_sparc_64.c | 2 +-
arch/x86/Kconfig | 16 ++++++++++
arch/x86/mm/mmap.c | 12 +++----
drivers/char/random.c | 22 +++++++++++++
fs/binfmt_elf.c | 2 +-
include/linux/mm.h | 11 +++++++
include/linux/random.h | 1 +
kernel/sysctl.c | 22 +++++++++++++
mm/mmap.c | 12 +++++++
18 files changed, 240 insertions(+), 18 deletions(-)
Did you try to discuss the back-port topic with the KSPP folks or other
key persons involved in these patches? In the ideal case, the authors
can be CC'ed, do not get annoyed by "these crazy people doing legacy
stuff", and may even do some reviews.
I've chatted with Elena over this last week, and she talked to Kees who
pointed out that the Android people are also doing KSPP backports to 4.4
(thanks, folks!). I didn't check any details, just a heads-up to avoid
duplicate work.
Hi!

The Android common kernel tree is visible here:

https://android.googlesource.com/kernel/common/

In the android-4.4 branch, the backport are these:

b471fcd FROMLIST: mm: ASLR: use get_random_long()
9a3fe39 FROMLIST: drivers: char: random: add get_random_long()
d51891f FROMLIST: x86: mm: support ARCH_MMAP_RND_BITS.
e2240a1 FROMLIST: arm64: mm: support ARCH_MMAP_RND_BITS.
25106ff FROMLIST: arm: mm: support ARCH_MMAP_RND_BITS.
d49d887 FROMLIST: mm: mmap: Add new /proc tunable for mmap_base ASLR.

Hopefully that helps!

-Kees


Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux


--
Kees Cook
Nexus Security


Re: [PATCH 4.4-cip 0/6] Extend user-space ASLR range

Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 23/12/16 16:46, Ben Hutchings wrote:
[Sorry for the delay; I haven't been feeling well.]

On Fri, 2016-12-09 at 13:20 +0100, Jan Kiszka wrote:
Did you try to discuss the back-port topic with the KSPP folks or other
key persons involved in these patches? In the ideal case, the authors
can be CC'ed, do not get annoyed by "these crazy people doing legacy
stuff", and may even do some reviews.
I would normally cc the upstream developers, but I was hesitant to do so
for CIP because this is not related to an official stable branch.
Perhaps I should ask on the KSPP list whether a cc for such feature
backports would be appreciated?
Please do so.


On Mon, 2016-12-19 at 11:52 +0100, Jan Kiszka wrote:
I've chatted with Elena over this last week, and she talked to Kees
who
pointed out that the Android people are also doing KSPP backports to
4.4
(thanks, folks!). I didn't check any details, just a heads-up to avoid
duplicate work.
Thanks for letting me know.

Ben.
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk

8261 - 8280 of 8412