-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Friday, July 07, 2017 7:40 PM
To: Daniel Sangorrin; cip-dev@...
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-07 10:50, Daniel Sangorrin wrote:

Hi,

I've been working on a minimal root filesystem made with "deby".
The code is here and it includes BSP meta layers for qemux86-64, Beaglebone black,
Renesas iwg20m board and CycloneV.

https://gitlab.com/cip-playground/project-x/tree/master/deby

The meta-cip-common layer is where we put the packages that will be included
for all targets. The BSP meta-layers contain settings related to the architecture of
the processor, and Linux/u-boot configuration.

Currently, I have tested qemux86-64 and Renesas iwg20m. Both of them worked fine
so in the next step I will try to run tests on them using B@D.
# Beaglebone should work fine, but I haven't tested it yet.

For Cyclone V, I need a kernel repository.
Koguchi-san: could you create it on https://gitlab.com/cip-playground/linux-cip-cyclonev

Finally, for U-boot I need repositories to fetch the sources.
Should we have separate repositories for each board? For example:
https://gitlab.com/cip-playground/u-boot-iwg20m
https://gitlab.com/cip-playground/u-boot-cyclonev

Nice work! Maybe I find the time to model a binary Debian variant as
well, using Isar [1] - ideally prior to DebConf.

Cool. By the way, I put all of the layers under /deby so that you could
create the isar version under the /isar folder.

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Thanks,
Daniel

Jan

[1] https://github.com/ilbers/isar
[2] https://github.com/siemens/kas
[3]
https://github.com/siemens/meta-iot2000/commit/eb4c3df4a62d0e5c32c74082e4c5c467627b9a84

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:20 PM
To: Daniel Sangorrin; cip-dev@...
Cc: kas-devel
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 01:36, Daniel Sangorrin wrote:

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Looks good. You should just set version to 0.10 and use yesterday's
release. You can pull the corresponding docker image from
https://hub.docker.com/r/kasproject/, and there should be a pip package
as well soon.

That was fast. I was using the next branch and built the Dockerfile included in kas.

I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
volume mount the project-x folder and manually set the proxy settings (for some reason using
build-args didn't work for me).

Does it still make a difference to set BB_NUMBER_THREADS and
PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
looking for reasonable defaults we can set (?=) from kas automatically.
BB_DISKMON_DIRS should likely be included, I was adding it as well already.

You know better than me ;)
I just checked the latest ref manual and apparently OE automatically sets them up.

Thanks,
Daniel

Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:42 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 08:34, Daniel Sangorrin wrote:

Hi Jan,

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:20 PM
To: Daniel Sangorrin; cip-dev@...
Cc: kas-devel
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 01:36, Daniel Sangorrin wrote:

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Looks good. You should just set version to 0.10 and use yesterday's
release. You can pull the corresponding docker image from
https://hub.docker.com/r/kasproject/, and there should be a pip package
as well soon.

That was fast. I was using the next branch and built the Dockerfile included in kas.

I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,

We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
it there a way to achieve this in an easier way? Still learning all this
docker stuff.

I was using the -e flag:
docker run -e USER_ID=`id -u $USER` -it kas sh

volume mount the project-x folder and manually set the proxy settings (for some reason using
build-args didn't work for me).

Hmm, they should, I'm using them for builds all the time (along with
corresponding settings for the docker daemon).

Are you sure you didn't set them somewhere else in the docker configuration?
As far as I can see the Dockerfile is not using any ARG instruction at all.

Does it still make a difference to set BB_NUMBER_THREADS and
PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
looking for reasonable defaults we can set (?=) from kas automatically.
BB_DISKMON_DIRS should likely be included, I was adding it as well already.

You know better than me ;)
I just checked the latest ref manual and apparently OE automatically sets them up.

Ah, good to know. The parallel things, or even also DISKMON?

The parallel things. BB_DISKMON_DIRS seems to be disabled by default.

Thanks,
Daniel

Jan

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Wednesday, July 12, 2017 6:13 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 09:00, Daniel Sangorrin wrote:

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:42 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 08:34, Daniel Sangorrin wrote:

Hi Jan,

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:20 PM
To: Daniel Sangorrin; cip-dev@...
Cc: kas-devel
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 01:36, Daniel Sangorrin wrote:

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Looks good. You should just set version to 0.10 and use yesterday's
release. You can pull the corresponding docker image from
https://hub.docker.com/r/kasproject/, and there should be a pip package
as well soon.

That was fast. I was using the next branch and built the Dockerfile included in kas.

I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,

We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
it there a way to achieve this in an easier way? Still learning all this
docker stuff.

I was using the -e flag:
docker run -e USER_ID=`id -u $USER` -it kas sh

Err, right, that's what I meant.

volume mount the project-x folder and manually set the proxy settings (for some reason using
build-args didn't work for me).

Hmm, they should, I'm using them for builds all the time (along with
corresponding settings for the docker daemon).

Are you sure you didn't set them somewhere else in the docker configuration?
As far as I can see the Dockerfile is not using any ARG instruction at all.

Yes, I'm sure. I've just written the doc for a product SDK, and that
uses no further settings as well, see
https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.

hmmm, I think that you made a small mistake in the kas documentation [1]. You wrote that
http_proxy should be passed as a build-arg but as I said there is no ARG instruction in the
Dockerfile.

In the meta-iot2000 link you are setting the proxy variables as environment variables using
'docker run -e http_proxy=$http_proxy' which does work indeed.

[1] https://kas.readthedocs.io/en/latest/devguide.html

Thanks,
Daniel

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Thursday, July 13, 2017 3:37 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-13 02:12, Daniel Sangorrin wrote:

Hi Jan,

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Wednesday, July 12, 2017 6:13 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 09:00, Daniel Sangorrin wrote:

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:42 PM
To: Daniel Sangorrin; cip-dev@...
Cc: 'kas-devel'
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 08:34, Daniel Sangorrin wrote:

Hi Jan,

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@...]
Sent: Tuesday, July 11, 2017 3:20 PM
To: Daniel Sangorrin; cip-dev@...
Cc: kas-devel
Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

On 2017-07-11 01:36, Daniel Sangorrin wrote:

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Looks good. You should just set version to 0.10 and use yesterday's
release. You can pull the corresponding docker image from
https://hub.docker.com/r/kasproject/, and there should be a pip package
as well soon.

That was fast. I was using the next branch and built the Dockerfile included in kas.

I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,

We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
it there a way to achieve this in an easier way? Still learning all this
docker stuff.

I was using the -e flag:
docker run -e USER_ID=`id -u $USER` -it kas sh

Err, right, that's what I meant.

volume mount the project-x folder and manually set the proxy settings (for some reason using
build-args didn't work for me).

Hmm, they should, I'm using them for builds all the time (along with
corresponding settings for the docker daemon).

Are you sure you didn't set them somewhere else in the docker configuration?
As far as I can see the Dockerfile is not using any ARG instruction at all.

Yes, I'm sure. I've just written the doc for a product SDK, and that
uses no further settings as well, see
https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.

hmmm, I think that you made a small mistake in the kas documentation [1]. You wrote that
http_proxy should be passed as a build-arg but as I said there is no ARG instruction in the
Dockerfile.

No, also that works: It won't leave the proxy settings behind in the
image, but it allows to build it behind a proxy.

We don't want to generate "proxified" images that will then only work in
a very specific environment. We are coming from there, it was a mess.
The image we have now can be fed into a CI build, and you only need to
configure the specific proxies at project level. Works nicely with our
corporate gitlab infrastructure.

OK, I understand your approach.

By the way ARG does not leave any settings behind (ENV does).
I was just confused because I thought that all available build-args had
to be declared on the Dockerfile through the ARG instruction.

In the past, not doing so would throw an error. From the current docker
documentation [1], it seems that that has changed and now it should
only throw a warning:
"If a user specifies a build argument that was not defined in the Dockerfile,
the build outputs a warning. [Warning] One or more build-args [foo] were not consumed."

Strange enough, I didn't see any warnings either. I wonder if the documentation
is outdated.

In any case, the kas documentation does not mention anything about setting the
variables on "docker run" so it might be a good idea to provide the information
form the meta-iot2000's README

If you are building from within a proxy-restricted network, make sure the settings are available via the standard environment variables and add

-e http_proxy=$http_proxy -e https_proxy=$https_proxy \
-e ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy

Thanks,
Daniel

[1] https://docs.docker.com/engine/reference/builder/#arg




Sorry I didn't know that you could use --build-arg
without defining an ARG instruction in the Dockerfile. It's not very clearly documented
on the docker website.

On the user's guide you mention "docker run -it kasproject/kas:<version> sh"

or bind-mount the project into the container. See https://hub.docker.com/r/kasproject for all available images."
Still it might be helpful

-e http_proxy=$http_proxy -e https_proxy=$https_proxy \
-e ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy

Jan

In the meta-iot2000 link you are setting the proxy variables as environment variables using
'docker run -e http_proxy=$http_proxy' which does work indeed.>
[1] https://kas.readthedocs.io/en/latest/devguide.html

Thanks,
Daniel

--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux