Introduction


Ben Hutchings <ben.hutchings@...>
 

I've been asked to introduce myself to the list, before getting into
discussions. So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.

--
Ben Hutchings
Software Developer, Codethink Ltd.


Daniel Sangorrin <daniel.sangorrin@...>
 

Hi Ben,

I've been asked to introduce myself to the list, before getting into
discussions. So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!

Best regards,
Daniel

--
IoT Technology center
Toshiba Corp. Industrial ICT solutions,
Daniel SANGORRIN



--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev


Agustin Benito Bethencourt <agustin.benito@...>
 

Hi,

On 15/09/16 08:50, Daniel Sangorrin wrote:
Hi Ben,

I've been asked to introduce myself to the list, before getting into
discussions. So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!
Daniel, can you tell us a little about yourself?

My name is Agustín Benito Bethencourt. I represent Codethink at CIP. I've managed teams-programs-projects for some years ago, the last in the open.

I am looking forward to find out how we are going to deal with this outstanding challenge, keeping a Linux system alive and healthy for many years in mission critical environments. I think it is an exciting one.

Best Regards

Agustin


Best regards,
Daniel

--
IoT Technology center
Toshiba Corp. Industrial ICT solutions,
Daniel SANGORRIN



--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk


Domenico Andreoli <domenico.andreoli@...>
 

Hi all,

On Thu, Sep 15, 2016 at 10:30:59AM +0200, Agustin Benito Bethencourt wrote:
On 15/09/16 08:50, Daniel Sangorrin wrote:
Hi Ben,

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!
Daniel, can you tell us a little about yourself?

My name is Agustín Benito Bethencourt. I represent Codethink at CIP. I've
managed teams-programs-projects for some years ago, the last in the open.
My name is Domenico Andreoli, I've been using Linux since 1995 and
actively contributed to Debian for about a decade. I've always worked
with Linux, on Linux, for Linux, thanks to Linux, mainly in the embedded
context but I learned that without the necessary shared values, Linux
is not fun at all.

Although I'm not really active nowdays, my FOSS imprinting imposes me
to at least observe and keep me informed. So I'm here, just representing
myself, with bags full of 2 cents.

I am looking forward to find out how we are going to deal with this
outstanding challenge, keeping a Linux system alive and healthy for many
years in mission critical environments. I think it is an exciting one.
Yes, very exciting.

Do we recognize that big irons and small IoT leaves are in the same
need also here?

Kind regards,
Domenico

--
3B10 0CA1 8674 ACBA B4FE FCD2 CE5B CF17 9960 DE13


Noriaki Fukuyasu <fukuyasu@...>
 

Hi Ben

Welcome on board! Thank you very much for joining the team.
We are all very excited to have you at this project :)

Hi all:

My name is Noriaki (Nori) Fukuyasu. I work at The Linux Foundation.
I am currently helping the CIP members to operate this project.. 
If anyone has any thoughts on CIP (in terms of the project operation) please let me know.
Also if any company wish to support this project as a corporate member, please let me know.

Thanks!

Nori





On Wed, Sep 14, 2016 at 8:10 PM, Ben Hutchings <ben.hutchings@...> wrote:
I've been asked to introduce myself to the list, before getting into
discussions.  So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time.  Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP.  I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.

--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev



--
Noriaki Fukuyasu

The Linux Foundation
Tel: +81-80-4350-1133
Twitter: nori_fukuyasu
Facebook: linuxfoundationjp

Please visit our web sites:


Paul Sherwood
 

On 2016-09-15 09:30, Agustin Benito Bethencourt wrote:
I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!
Daniel, can you tell us a little about yourself?

My name is Agustín Benito Bethencourt. I represent Codethink at CIP.
I've managed teams-programs-projects for some years ago, the last in
the open.

I am looking forward to find out how we are going to deal with this
outstanding challenge, keeping a Linux system alive and healthy for
many years in mission critical environments. I think it is an exciting
one.
To introduce myself, I'm CEO at Codethink, but to the best of my ability I'll be commenting here from a personal perspective, independent of Codethink's official involvement in CIP.

To start the ball rolling, I'm personally worried about how we are actually going to achieve super-long-term-support and am happy that we can discuss the issues in public.

Greg K-H, Ben Hutchings and others have contributed a huge amount to Long Term Stable and followon initiatives in the community over the years. But when I first started exploring how things like LTS and LTSI can work for embedded and automotive in 2012/2013, I hit some fundamental questions, not least - how in practice can a complex embedded project consume a 'stable' kernel that's being released ** every couple of weeks ** with the words 'users of this series must upgrade'? I presented some work at an automotive GENIVI event in Oct 2013 [1] but the audience at that time literally refused to accept that the idea of whole-of-life updates.

And as Greg said at the time:

"The patches that apply for stuff after 2 years drops off dramatically, and the work involved in keeping stuff working and testing for problems increases greatly.”

Just yesterday there was a very interesting post about backports and long term stable kernels on LWN [2]. Greg is quoted there considering:

"But if we didn't provide an LTS, would companies constantly update their kernels to newer releases to keep up with the security and bugfixes? That goes against everything those managers/PMs have ever been used to in the past, yet it's actually the best thing they could do."

I've been recommending the constant update route route to customers over the last few years, with some success, but many ecosystem members are extremely uncomfortable with the whole idea of aligning with mainline. I think this is broadly because as embedded engineers we've learned over many years that it's best to change the platform as little as possible. I wrote an article trying to challenge this traditional embedded thinking earlier this year [3]

Would be very interested in others' thoughts on this.

br
Paul

[1] http://www.devcurmudgeon.com/pdfs/mainline-lts-ltsi-genivi-20131025.pdf
[2] https://lwn.net/SubscriberLink/700557/091f804480fab479/
[3] http://www.devcurmudgeon.com/technical-debt-for-whole-systems.html


Paul Sherwood
 

On 2016-09-15 17:17, Paul Sherwood wrote:
<snip>
Just yesterday there was a very interesting post about backports and
long term stable kernels on LWN [2]. Greg is quoted there considering:
Oops - I gave the wrong link:

https://lwn.net/SubscriberLink/700530/7f1c21b0fd40489e/

"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever
been used to in the past, yet it's actually the best thing they could
do."


Ben Hutchings <ben.hutchings@...>
 

On Thu, 2016-09-15 at 17:17 +0100, Paul Sherwood wrote:
[...]
[Greg K-H wrote:]
"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever been
used to in the past, yet it's actually the best thing they could do."
Linux does have a very good record of maintaining application binary
compatibility, which should make it safe to upgrade. But every new
release from Linus brings some or all of the following problems:

- feature regressions
- performance regressions
- security regressions
- increased resource consumption
- removal of deprecated features
- driver API churn

Most of the regressions get fixed quickly, but some linger long after
support for the previous stable branch has ended.

So I don't think it's generally sensible to use the latest stable update
in production, and that's why I care about longterm branches (and I
don't know why Greg does).

I've been recommending the constant update route route to customers
over the last few years, with some success, but many ecosystem members
are extremely uncomfortable with the whole idea of aligning with
mainline. I think this is broadly because as embedded engineers we've
learned over many years that it's best to change the platform as little
as possible. I wrote an article trying to challenge this traditional
embedded thinking earlier this year [3]

Would be very interested in others' thoughts on this.
I agree it's possible to do rolling upgrades, but it's very dependent on
the capability and the will (1) to do regular regression testing on the
target systems and (2) to address the regressions that are found without
waiting for them to be fixed upstream. Where a performance regression
or increased resource consumption stretches an existing system so that
it no longer meets its requirements, this might not be practically
possible. This also applies where non-upstream code is broken by
upstream API changes, and that issue is not going away soon.

Ben.

--
Ben Hutchings
Software Developer, Codethink Ltd.


Daniel Sangorrin <daniel.sangorrin@...>
 

Hi Agustín and all,

-----Original Message-----
From: cip-dev-bounces@lists.cip-project.org [mailto:cip-dev-bounces@lists.cip-project.org] On Behalf Of Agustin Benito Bethencourt
Sent: Thursday, September 15, 2016 5:31 PM
To: cip-dev@lists.cip-project.org
Subject: Re: [cip-dev] Introduction

Hi,

On 15/09/16 08:50, Daniel Sangorrin wrote:
Hi Ben,

I've been asked to introduce myself to the list, before getting into
discussions. So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!
Daniel, can you tell us a little about yourself?
Sure.

I've been working for Toshiba Industrial ICT solutions for about 3 years and a half.
That included customizing Linux kernels (mostly driver- and RT-related issues)
and root filesystems [1] for a wide range of embedded systems; as well as
doing research on real-time partitioning, security [2], fast boot, and software updates [3].

# I also have experience developing non-Linux real-time embedded systems
# (e.g. RTOSes, drivers, real-time network protocols, or an ARM Trustzone monitor [4])

[1] http://events.linuxfoundation.org/sites/events/files/slides/LinuxConJapan2016_deby.pdf
[2] https://lwn.net/Articles/673003/
[3] http://events.linuxfoundation.org/sites/events/files/slides/linuxcon-japan-2016-softwre-updates-sangorrin.pdf
[4] https://www.toppers.jp/en/safeg.html

My name is Agustín Benito Bethencourt. I represent Codethink at CIP.
I've managed teams-programs-projects for some years ago, the last in the
open.

I am looking forward to find out how we are going to deal with this
outstanding challenge, keeping a Linux system alive and healthy for many
years in mission critical environments. I think it is an exciting one.
Yeah, I think so too!

Regards,
Daniel





Best Regards

Agustin


Best regards,
Daniel

--
IoT Technology center
Toshiba Corp. Industrial ICT solutions,
Daniel SANGORRIN



--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev
--
Agustin Benito Bethencourt
Principal Consultant - FOSS at Codethink
agustin.benito@codethink.co.uk
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev


Jan Kiszka
 

Hi all,

my name is Jan Kiszka. I'm working for Siemens Corporate Technology on
enabling and enhancing open source components for the use in our
embedded products, and that primarily at lower levels like the kernel.

On 2016-09-16 00:31, Ben Hutchings wrote:
On Thu, 2016-09-15 at 17:17 +0100, Paul Sherwood wrote:
[...]
[Greg K-H wrote:]
"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever been
used to in the past, yet it's actually the best thing they could do."
Linux does have a very good record of maintaining application binary
compatibility, which should make it safe to upgrade. But every new
release from Linus brings some or all of the following problems:

- feature regressions
- performance regressions
- security regressions
- increased resource consumption
- removal of deprecated features
- driver API churn

Most of the regressions get fixed quickly, but some linger long after
support for the previous stable branch has ended.

So I don't think it's generally sensible to use the latest stable update
in production, and that's why I care about longterm branches (and I
don't know why Greg does).
Exactly.


I've been recommending the constant update route route to customers
over the last few years, with some success, but many ecosystem members
are extremely uncomfortable with the whole idea of aligning with
mainline. I think this is broadly because as embedded engineers we've
learned over many years that it's best to change the platform as little
as possible. I wrote an article trying to challenge this traditional
embedded thinking earlier this year [3]

Would be very interested in others' thoughts on this.
I agree it's possible to do rolling upgrades, but it's very dependent on
the capability and the will (1) to do regular regression testing on the
target systems and (2) to address the regressions that are found without
waiting for them to be fixed upstream. Where a performance regression
or increased resource consumption stretches an existing system so that
it no longer meets its requirements, this might not be practically
possible. This also applies where non-upstream code is broken by
upstream API changes, and that issue is not going away soon.
And while we are investing in extending long-term support with our SLTS
versions, we have to improve regression testing as well to ensure the
quality of those releases. But nothing prevents to use the results also
for latest upstream versions. That can also contribute to making rolling
updates more realistic in the future, I we should point this out
whenever folks complain that SLTS would be against the upstream trend.

Jan


Daniel Sangorrin <daniel.sangorrin@...>
 

Hi Paul,

To introduce myself, I'm CEO at Codethink, but to the best of my
ability I'll be commenting here from a personal perspective, independent
of Codethink's official involvement in CIP.

To start the ball rolling, I'm personally worried about how we are
actually going to achieve super-long-term-support and am happy that we
can discuss the issues in public.

Greg K-H, Ben Hutchings and others have contributed a huge amount to
Long Term Stable and followon initiatives in the community over the
years. But when I first started exploring how things like LTS and LTSI
can work for embedded and automotive in 2012/2013, I hit some
fundamental questions, not least - how in practice can a complex
embedded project consume a 'stable' kernel that's being released **
every couple of weeks ** with the words 'users of this series must
upgrade'? I presented some work at an automotive GENIVI event in Oct
2013 [1] but the audience at that time literally refused to accept that
the idea of whole-of-life updates.
As for the embedded systems I deal with, a 2-weeks release is definitely not
required. A 6 six months cycle, complemented with aperiodic patch releases
for really *important* issues, would be good enough.
Of course, different use cases may have different requirements so we
will probably need to reach a consensus on that.

And as Greg said at the time:

"The patches that apply for stuff after 2 years drops off dramatically,
and the work involved in keeping stuff working and testing for problems
increases greatly.”
Just yesterday there was a very interesting post about backports and
long term stable kernels on LWN [2]. Greg is quoted there considering:

"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever been
used to in the past, yet it's actually the best thing they could do."

I've been recommending the constant update route route to customers
over the last few years, with some success, but many ecosystem members
are extremely uncomfortable with the whole idea of aligning with
mainline. I think this is broadly because as embedded engineers we've
learned over many years that it's best to change the platform as little
as possible. I wrote an article trying to challenge this traditional
embedded thinking earlier this year [3]
Thanks, interesting article.

" All of which makes perfect sense for traditional embedded projects."

I just wanted to clarify that these 'traditional embedded projects' are actually
in the scope of the CIP project. I believe embedded systems where
continuous updates are hard to implement, should still benefit from other
CIP activities (e.g. testing, RAS, real-time partitioning support or kernel
self-protection).

Best regards,
Daniel


Would be very interested in others' thoughts on this.

br
Paul

[1]
http://www.devcurmudgeon.com/pdfs/mainline-lts-ltsi-genivi-20131025.pdf
[2] https://lwn.net/SubscriberLink/700557/091f804480fab479/
[3] http://www.devcurmudgeon.com/technical-debt-for-whole-systems.html
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev


Paul Sherwood
 

On 2016-09-16 06:15, Daniel Sangorrin wrote:
Greg K-H, Ben Hutchings and others have contributed a huge amount to
Long Term Stable and followon initiatives in the community over the
years. But when I first started exploring how things like LTS and LTSI
can work for embedded and automotive in 2012/2013, I hit some
fundamental questions, not least - how in practice can a complex
embedded project consume a 'stable' kernel that's being released **
every couple of weeks ** with the words 'users of this series must
upgrade'? I presented some work at an automotive GENIVI event in Oct
2013 [1] but the audience at that time literally refused to accept that
the idea of whole-of-life updates.
As for the embedded systems I deal with, a 2-weeks release is definitely not
required. A 6 six months cycle, complemented with aperiodic patch releases
for really *important* issues, would be good enough.
Of course, different use cases may have different requirements so we
will probably need to reach a consensus on that.
I expect there are multiple usecases/scenarios and a one-size-fits-all approach may not be possible. But note that even with six month cycle and periodic patch releases, it seems to me you imply requirements that

a) updates are relatively easy, low effort, low risk
b) updates may be required for the whole production lifetime of the target

I've seen plenty of examples where the real-world LTSI BSP implementation has made the process of updating the kernel 'a nightmare'.

And I've had lots of pushback from people insisting that no updates will be required 'after the first couple of years, when the bugs have been ironed out'.

I'm not yet sure whether CIP usecases mostly involve devices which are connected to the internet or other third-party services. And I'm not sure whether security and integrity of the software over the longterm is expected to be a key concern or not.

And as Greg said at the time:

"The patches that apply for stuff after 2 years drops off dramatically,
and the work involved in keeping stuff working and testing for problems
increases greatly.”
Just yesterday there was a very interesting post about backports and
long term stable kernels on LWN [2]. Greg is quoted there considering:

"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever been
used to in the past, yet it's actually the best thing they could do."

I've been recommending the constant update route route to customers
over the last few years, with some success, but many ecosystem members
are extremely uncomfortable with the whole idea of aligning with
mainline. I think this is broadly because as embedded engineers we've
learned over many years that it's best to change the platform as little
as possible. I wrote an article trying to challenge this traditional
embedded thinking earlier this year [3]
Thanks, interesting article.

" All of which makes perfect sense for traditional embedded projects."

I just wanted to clarify that these 'traditional embedded projects'
are actually
in the scope of the CIP project.
Yes, they are.

I'm just suggesting that once we are working with a connected device containing more than tens of millions of lines of code, the principles we learned on self-contained device projects with tens or hundreds of thousands of lines, even if they have worked successfully for decades, may no longer apply.

I believe embedded systems where
continuous updates are hard to implement, should still benefit from other
CIP activities (e.g. testing, RAS, real-time partitioning support or kernel
self-protection).
Absolutely.


Yoshitake Kobayashi
 

Hi all,

My name is Yoshitake Kobayashi. I am working for Toshiba.
I've been working on operating system related topics for more than 20 years.
Currently, I've been leading an embedded Linux team in Toshiba to provide Linux
environments for our embedded products.
For open source related projects, I represent Toshiba for three projects which
include CIP, Debian-LTS [1] and Linux Foundation CE Linux project [2].
I've also made several presentations at Embedded Linux Conferences [3].
In retrospect, the most of my presentations relate to start CIP.

For CIP, I am one of the founding member. So, CIP is really exciting and
challenging project for us. I am looking forward to working with you.

----------------------------------------------------------------
[1] https://www.freexian.com/en/services/debian-lts.html
[2] https://www.linuxfoundation.org/projects/core-embedded-linux
[3] http://www.embeddedlinuxconference.com/
----------------------------------------------------------------

Best regards,
Yoshi

On 2016/09/15 17:30, Agustin Benito Bethencourt wrote:
Hi,

On 15/09/16 08:50, Daniel Sangorrin wrote:
Hi Ben,

I've been asked to introduce myself to the list, before getting into
discussions. So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time. Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP. I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.
Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!
Daniel, can you tell us a little about yourself?

My name is Agustín Benito Bethencourt. I represent Codethink at CIP. I've managed teams-programs-projects for some years ago, the last in the open.

I am looking forward to find out how we are going to deal with this outstanding challenge, keeping a Linux system alive and healthy for many years in mission critical environments. I think it is an exciting one.

Best Regards

Agustin


Best regards,
Daniel

--
IoT Technology center
Toshiba Corp. Industrial ICT solutions,
Daniel SANGORRIN



--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev


Daniel Sangorrin <daniel.sangorrin@...>
 

-----Original Message-----
From: Paul Sherwood [mailto:paul.sherwood@codethink.co.uk]
Sent: Friday, September 16, 2016 4:41 PM
To: Daniel Sangorrin
Cc: cip-dev@lists.cip-project.org
Subject: RE: [cip-dev] Introduction

On 2016-09-16 06:15, Daniel Sangorrin wrote:
Greg K-H, Ben Hutchings and others have contributed a huge amount to
Long Term Stable and followon initiatives in the community over the
years. But when I first started exploring how things like LTS and
LTSI
can work for embedded and automotive in 2012/2013, I hit some
fundamental questions, not least - how in practice can a complex
embedded project consume a 'stable' kernel that's being released **
every couple of weeks ** with the words 'users of this series must
upgrade'? I presented some work at an automotive GENIVI event in
Oct
2013 [1] but the audience at that time literally refused to accept
that
the idea of whole-of-life updates.
As for the embedded systems I deal with, a 2-weeks release is
definitely not
required. A 6 six months cycle, complemented with aperiodic patch
releases
for really *important* issues, would be good enough.
Of course, different use cases may have different requirements so we
will probably need to reach a consensus on that.
I expect there are multiple usecases/scenarios and a one-size-fits-all
approach may not be possible. But note that even with six month cycle
and periodic patch releases, it seems to me you imply requirements that

a) updates are relatively easy, low effort, low risk
b) updates may be required for the whole production lifetime of the
target

I've seen plenty of examples where the real-world LTSI BSP
implementation has made the process of updating the kernel 'a
nightmare'.

And I've had lots of pushback from people insisting that no updates
will be required 'after the first couple of years, when the bugs have
been ironed out'.

I'm not yet sure whether CIP usecases mostly involve devices which are
connected to the internet or other third-party services. And I'm not
sure whether security and integrity of the software over the longterm is
expected to be a key concern or not.
Probably there are many different usecases. Sharing the requirements
for each of them could be beneficial for limiting the scope of our work to
things that matter the most.

In the usecases I'm most familiar with, devices are working either standalone
(no physical connection to the Internet) or behind client's security hardened
(e.g. VPNs, firewalls..) gateways that we don't control.
These devices are usually updated (in practice reinstalled) by hand after
stopping the process they are controlling (which can't happen every day)
and run tasks that need high predictability.

For these usecases, we require software (e.g.: kernel, partitioned hypervisor, rootfs)
that has been proved to be stable and reliable for a long-enough period of time, and
will keep being supported as long as our clients require. Note that it's not only about
making updates, but also about making new devices with software that is known
to work reliably across the different companies in the CIP project.

Cheers,
Daniel

And as Greg said at the time:

"The patches that apply for stuff after 2 years drops off
dramatically,
and the work involved in keeping stuff working and testing for
problems
increases greatly.”
Just yesterday there was a very interesting post about backports and
long term stable kernels on LWN [2]. Greg is quoted there
considering:

"But if we didn't provide an LTS, would companies constantly update
their kernels to newer releases to keep up with the security and
bugfixes? That goes against everything those managers/PMs have ever
been
used to in the past, yet it's actually the best thing they could
do."

I've been recommending the constant update route route to customers
over the last few years, with some success, but many ecosystem
members
are extremely uncomfortable with the whole idea of aligning with
mainline. I think this is broadly because as embedded engineers
we've
learned over many years that it's best to change the platform as
little
as possible. I wrote an article trying to challenge this
traditional
embedded thinking earlier this year [3]
Thanks, interesting article.

" All of which makes perfect sense for traditional embedded
projects."

I just wanted to clarify that these 'traditional embedded projects'
are actually
in the scope of the CIP project.
Yes, they are.

I'm just suggesting that once we are working with a connected device
containing more than tens of millions of lines of code, the principles
we learned on self-contained device projects with tens or hundreds of
thousands of lines, even if they have worked successfully for decades,
may no longer apply.
Devices that directly connect to the internet (e.g.: gateways) definitely
need to be security hardened. However, I'm not sure about
the level of security required for devices that are only indirectly
connected (e.g.: behind the gateways). And I'm not sure if all
security mechanisms are compatible with the predictability
required by these systems.



I believe embedded systems where
continuous updates are hard to implement, should still benefit from
other
CIP activities (e.g. testing, RAS, real-time partitioning support or
kernel
self-protection).
Absolutely.


Urs Gleim
 

Hi,

my name is Urs Gleim. I am with Siemens Corporate Technology. I have a history in embedded systems development with all the big proprietary embedded operating systems out there. I worked in projects in the domains automotive (mainly infotainment), industry automation, healthcare and others. Today I am more in the managing role pushing the shift to open source software and (embedded) Linux in particular in the company (lessons learnt from the work with other OS's :-) We run the internal "Corporate Competence Center Embedded Linux" and support businesses units migrating to and maintaining Linux for their products.

The experiences of the last years showed that the way each and every company is building their own customized Linux versions, tool chains, and test infrastructure could be done much more efficiently and more sustainable. Even inside each company there are several solutions for different products.

We have different product cycles and requirements compared to consumer products, data centers, for example. So what every company in the area of systems used in rail, traffic control, healthcare, power generation/transmission, etc. does is: i. adapting Linux for their needs (in terms of build environment, test, features, ...), ii. maintaining these solutions for years.

Furthermore, having standard configurations would lead to a common platform as a basis for others building upon those platform (libraries, network protocols, tools). At the end this is an ecosystem involving and bringing together different communities/projects, suppliers, and the companies creating products. In other domains there are efforts going in the same direction (e.g., AGL, Genivi). So we believe we can do much more by joining forces in the operating system area and Linux will be established even more in the industrial world. One of the strongest pain points today is the effort we spend for (super) long-term maintenance, for each product, in each company. So we start addressing this by creating a  harmonized build and test infrastructure and agreeing on common super long-term maintained kernel versions and patch cycles.

I am looking forward working with you.

Best regards,
Urs


On 16.09.2016 09:53, KOBAYASHI Yoshitake wrote:
Hi all,

My name is Yoshitake Kobayashi. I am working for Toshiba.
I've been working on operating system related topics for more than 20 years.
Currently, I've been leading an embedded Linux team in Toshiba to provide Linux
environments for our embedded products.
For open source related projects, I represent Toshiba for three projects which
include CIP, Debian-LTS [1] and Linux Foundation CE Linux project [2].
I've also made several presentations at Embedded Linux Conferences [3].
In retrospect, the most of my presentations relate to start CIP.

For CIP, I am one of the founding member. So, CIP is really exciting and
challenging project for us. I am looking forward to working with you.

----------------------------------------------------------------
[1] https://www.freexian.com/en/services/debian-lts.html
[2] https://www.linuxfoundation.org/projects/core-embedded-linux
[3] http://www.embeddedlinuxconference.com/
----------------------------------------------------------------

Best regards,
Yoshi

On 2016/09/15 17:30, Agustin Benito Bethencourt wrote:
Hi,

On 15/09/16 08:50, Daniel Sangorrin wrote:
Hi Ben,

I've been asked to introduce myself to the list, before getting into
discussions.  So here's a very brief summary.

I've been working on the Linux kernel for about 10 years, including
co-maintenance of Debian's kernel package for most of that time.  Debian
provides security support for each stable release for 5 years, and in
support of that I backported many fixes to 2.6.32-longterm and took on
maintenance of the 3.2-longterm and 3.16-longterm branches.

I'm now adding another part-time role as lead kernel maintainer for the
CIP.  I look forward to working with you all to establish a project and
process that can extend the support lifetime even further.

Ben.

Welcome to the CIP project. It's great to have such an experienced maintainer
in the project. Looking forward to working with you too!

Daniel, can you tell us a little about yourself?

My name is Agustín Benito Bethencourt. I represent Codethink at CIP. I've managed teams-programs-projects for some years ago, the last in the open.

I am looking forward to find out how we are going to deal with this outstanding challenge, keeping a Linux system alive and healthy for many years in mission critical environments. I think it is an exciting one.

Best Regards

Agustin


Best regards,
Daniel

--
IoT Technology center
Toshiba Corp. Industrial ICT solutions,
Daniel SANGORRIN



--
Ben Hutchings
Software Developer, Codethink Ltd.


_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev



_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev