Date   

Regarding backporting RTC PCF85263 driver to cip 4.4 kernel

Biju Das <biju.das@...>
 

Hi All,

 

RTC PCF85263 is used by iWave RZ/G1C SBC( iwg23s) platform.  We have upstreamed  this driver support  on 4.19 kernel [1]

[1]. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/rtc/rtc-pcf85363.c?h=v5.2&id=fc979933bcf162595b6004d0de4effb64c323152

 

This driver is based  on RTC PCF 85363 driver which has rtc nvmem/alarm support [2] .  Rtc nvmem framework is introduced in 4.13

[2]. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/rtc/rtc-pcf85363.c?h=v5.2&id=a9687aa2764dd2669602bd19dc636cbeef5293d5

 

We are seeing the following options for backporting this driver to 4.4 kernel

 

Option 1:

Backport this driver[1] to 4.4 kernel along with nvmem/alarm support. we have already done this work and patches are available at [3]

[3]. https://gitlab.com/bijud/pcf85263/tree/master

 

Option 2:

Backport RTC PCF85263 driver, removing  Alarm/nvmem functionality from  [2] .

 

Option 3:

Don't backport RTC support at all.

 

Please share your views, so that I can work on this accordingly.

 

Regards,

Biju

 

 


Re: Kernel configurations for 4.19?

daniel.sangorrin@...
 

Hello Ben,

[Note] repeating the reply because I got some rejected mails.

From: Ben Hutchings
I have received new config files from most members, but no-one told me
if the old configs can be deleted. For reference, these are the files
we have for 4.19 that are based on those submitted for 4.4:
You can discard the 4.4 old configurations from Toshiba. At the moment, we only need support for this one:
https://gitlab.com/cip-project/cip-kernel/cip-kernel-config/blob/master/4.19/x86/toshiba_atom_baytrail_cip.config

[Note] we may send more configuration files in the near future.

Thanks,
Daniel

4.19/arm/hitachi_omap_defconfig
4.19/arm/moxa_mxc_defconfig
4.19/arm/siemens_am335x-axm2_defconfig
4.19/arm/siemens_am335x-draco_defconfig
4.19/arm/siemens_am335x-dxr2_defconfig
4.19/arm/siemens_am335x-etamin_defconfig
4.19/arm/siemens_am57xx-pxm3_defconfig
4.19/arm/toshiba_tegra_defconfig
4.19/arm/toshiba_zynq_defconfig
4.19/powerpc/toshiba_defconfig
4.19/x86/plathome_obsvx1.config
4.19/x86/siemens_i386-rt.config
4.19/x86/siemens_iot2000.config
4.19/x86/siemens_server_defconfig
4.19/x86/toshiba_defconfig

Ben.

--
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom

_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


Re: [PATCH v3 4.19.y-cip] Add gitlab-ci.yaml

Pavel Machek
 

Hi!

This is configured to build and test the following configurations:

* BUILD_ARCH: arm
* CONFIG: renesas_shmobile_defconfig
* CONFIG_LOC: cip-kernel-config
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

* BUILD_ARCH: arm64
* CONFIG: renesas_defconfig
* CONFIG_LOC: cip-kernel-config
* DEVICES: r8a774c0-ek874
* DTBS: r8a774c0-ek874.dtb

* BUILD_ARCH: arm
* CONFIG: shmobile_defconfig
* CONFIG_LOC: intree
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

Over time support will be added for all CIP supported architectures and
configurations.

At the moment only simple boot tests are run. Real tests will be added in
the future
I went ahead, and applied this to linux-4.19.y-cip-rt-rebase branch,
too. If it is easy to add that one to testing, it would be nice.

Best regards,
Pavel

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Re: [PATCH v3 4.4.y-cip] Add gitlab-ci.yaml

Pavel Machek
 

On Mon 2019-07-08 14:16:03, Chris Paterson wrote:
This is configured to build and test the following configurations:

* BUILD_ARCH: arm
* CONFIG: renesas_shmobile_defconfig
* CONFIG_LOC: cip-kernel-config
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

* BUILD_ARCH: arm
* CONFIG: shmobile_defconfig
* CONFIG_LOC: intree
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

Over time support will be added for all CIP supported architectures and
configurations.

At the moment only simple boot tests are run. Real tests will be added in
the future
Thanks, applied to 4.4-cip, and pushed out.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Re: [PATCH v3 4.19.y-cip] Add gitlab-ci.yaml

Pavel Machek
 

Hi!

This is configured to build and test the following configurations:

* BUILD_ARCH: arm
* CONFIG: renesas_shmobile_defconfig
* CONFIG_LOC: cip-kernel-config
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

* BUILD_ARCH: arm64
* CONFIG: renesas_defconfig
* CONFIG_LOC: cip-kernel-config
* DEVICES: r8a774c0-ek874
* DTBS: r8a774c0-ek874.dtb

* BUILD_ARCH: arm
* CONFIG: shmobile_defconfig
* CONFIG_LOC: intree
* DEVICES: r8a7743-iwg20d-q7 r8a7745-iwg22d-sodimm
* DTBS: r8a7743-iwg20d-q7-dbcm-ca.dtb r8a7745-iwg22d-sodimm-dbhd-ca.dtb

Over time support will be added for all CIP supported architectures and
configurations.
Thanks, applied and pushed out. Sorry for the delay.

Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Re: [ANNOUNCE] 4.4.176-cip31-rt23

Jan Kiszka
 

Hi Pavel,

On 29.03.19 12:06, Daniel Wagner wrote:
Hello CIP RT Folks!

I'm pleased to announce the 4.4.176-cip31-rt23 stable release.

This release is just an update to the new stable 4.4.176-cip31 version
and no RT specific changes have been made.

You can get this release via the git tree at:

git://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git

branch: linux-4.4.y-cip-rt
Head SHA1: b51a171ad762ba4a78b0ed0c7ec83fb9f6fb135f
Is there a chance to update 4.4-rt based on Daniel' 4.4.179-rt181 release, but
then to 4.4.182 in order to have SACK fixes in?

Thanks,
Jan

PS: Please make it -rt24 then. ;)

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux


Re: About CIP kernel maintenance policy for new hardware support backporting

Hiraku Toyooka
 

Hi,

You can find the following sentence on the page.
an SLTS branch may include larger changes to support new hardware, to bridge the gap between SLTS branches.
Oh, I missed the sentence. Thank you very much for telling it.

In addition to these, we may need to describe the supported hardware (CPU, boards) and
test environment as well.
Is the backporting limited to the supported hardware?

Best regards,
Hiraku Toyooka

2019年7月11日(木) 8:47 <nobuhiro1.iwamatsu@...>:


Hi,


Thanks for pointed out.
You can find the following sentence on the page.
an SLTS branch may include larger changes to support new hardware, to bridge the gap between SLTS branches.
I think we need to write much clear to mention what kind of patches we can accept to CIP SLTS tree and how to approve(?) it by CIP.
Agree.

We think that we need to add the following sentences.
---
We do not apply patches that have not been applied to upstream
(Linus tree:https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git).
If you want to add new features or hardware, they need to be applied into upstream.
Also, you may not be able to easily apply the patches on Upstream. In such a case,
you need to describe in the commit message of patch what modifications have been made
from the original patch.
---

In addition to these, we may need to describe the supported hardware (CPU, boards) and
test environment as well.

Best regards,
Nobuhiro

From: Yoshitake Kobayashi [mailto:yoshitake.kobayashi@...]
Sent: Wednesday, July 10, 2019 9:56 PM
To: hiraku.toyooka@...
Cc: cip-dev@...; pavel@...; iwamatsu nobuhiro(岩松 信洋 ○SWC□OST) <nobuhiro1.iwamatsu@...>
Subject: Re: [cip-dev] About CIP kernel maintenance policy for new hardware support backporting

Hi,

Thanks for pointed out.
You can find the following sentence on the page.
an SLTS branch may include larger changes to support new hardware, to bridge the gap between SLTS branches.
I think we need to write much clear to mention what kind of patches we can accept to CIP SLTS tree and how to approve(?) it by CIP.

Best regards,
Yoshi



2019年7月10日(水) 15:42 <hiraku.toyooka@...>:
Hello,

I have a question about CIP kernel maintenance policy.
Can CIP kernel accept backport patches for new hardware support (not
only fixes)?

I read the following document and it seems to describe only acceptable fixes.
https://wiki.linuxfoundation.org/civilinfrastructureplatform/cipkernelmaintenance

On the other hand, Current linux-4.19.y-cip branch accepts some hardware
support patches such as RZ/G2E(r8a774c0) from upstream.
So I wonder if some policy about new hardware support exists.

--
Hiraku Toyooka
Cybertrust Japan Co., Ltd.
_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


--
Hiraku Toyooka
Cybertrust Japan Co., Ltd.


[cip-kernel-sec][Quickstart v2] docs: add a quickstart with practical information

Daniel Sangorrin <daniel.sangorrin@...>
 

Although the README already contains all the information
that users may need, there are some bits of know-how that
are better expressed through a step-by-step quickstart or
tutorial. This files tries to fill that gap.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
QUICKSTART.md | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 132 insertions(+)
create mode 100644 QUICKSTART.md

diff --git a/QUICKSTART.md b/QUICKSTART.md
new file mode 100644
index 0000000..c79af41
--- /dev/null
+++ b/QUICKSTART.md
@@ -0,0 +1,132 @@
+# Quickstart
+
+## Overview
+
+This project tracks the status of CVEs in mainline and stable kernels. Each CVE is described in YAML format that includes data such as:
+
+```
+$ cat issues/CVE-2019-1999.yml
+description: 'binder: fix race between munmap() and direct reclaim'
+references:
+- https://source.android.com/security/bulletin/2019-02-01
+comments:
+ Debian-bwh: |-
+ Introduced in 4.14 by f2517eb76f1f "android: binder: Add global lru
+ shrinker to binder". Backports of the fix to stable have incorrect
+ metadata.
+ bwh: Backports to stable have incorrect metadata
+introduced-by:
+ mainline: [f2517eb76f1f2f7f89761f9db2b202e89931738c]
+fixed-by:
+ linux-4.14.y: [33c6b9ca70a8b066a613e2a3d0331ae8f82aa31a]
+ linux-4.19.y: [6bf7d3c5c0c5dad650bfc4345ed553c18b69d59e]
+ linux-5.0.y: [bbb19ca082ce27ce60ca65be016a951806ea947c]
+ mainline: [5cec2d2e5839f9c0fec319c523a911e0a7fd299f]
+```
+
+## Quickstart
+
+Clone `cip-kernel-sec` and install its dependencies:
+
+```
+$ git clone https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec
+$ cd cip-kernel-sec/
+$ sudo apt install python3-yaml python3-html5lib python3-cherrypy3 python3-jinja2
+```
+
+Prepare kernel remote repositories according to `conf/remotes.yml`:
+
+```
+$ ./scripts/prepare_remotes.py
+```
+
+Alternatively, you can do that manually:
+
+```
+$ mkdir ../kernel
+$ cd ../kernel
+$ git remote add torvalds https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+$ git remote add stable https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
+$ git remote add cip https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git
+$ cd ../cip-kernel-sec
+```
+
+Download CVE information from [Debian] (https://salsa.debian.org/kernel-team/kernel-sec.git), [Ubuntu] (https://git.launchpad.net/ubuntu-cve-tracker) and Stable:
+
+```
+
+$ ./scripts/import_debian.py
+ -> import/debian
+$ ./scripts/import_ubuntu.py
+ -> import/ubuntu
+$ ./scripts/import_stable.py
+ -> import/stable_branches.yml
+```
+
+Check issues that affect a linux-cip branch:
+
+```
+$ ./scripts/report_affected.py linux-4.4.y
+```
+
+You can show a short description on your report:
+
+```
+$ ./scripts/report_affected.py --show-description linux-4.4.y
+```
+
+Check issues that affect a tag:
+
+```
+$ ./scripts/report_affected.py v4.4.181-cip33
+```
+
+Browse kernel branches and issues interactively:
+
+```
+$ ./scripts/webview.py
+$ firefox http://localhost:8080
+```
+
+[Note] Use Ctr-c to stop the `webview.py` script.
+
+## Kernel maintainer workflow
+
+Import or update the latest CVE information:
+
+```
+$ ./scripts/import_debian.py
+$ ./scripts/import_ubuntu.py
+$ ./scripts/import_stable.py
+```
+
+Edit by hand the newly created issues if you see that some imported information is incorrect or there is missing information:
+
+```
+$ vi issues/CVE-xx.yml
+```
+
+Validate the issue files against the YAML schema.
+
+```
+$ ./scripts/validate.py
+```
+
+YAML allows the same thing to be written in different ways, e.g. bracketed vs bulleted lists. Use `cleanup.py` to make the syntax and ordering of items consistent with the importers, to reduce "noise" in diffs:
+
+```
+$ ./scripts/cleanup.py
+```
+
+Check if the current issues:
+
+```
+$ ./scripts/report_affected.py
+```
+
+## Changelog
+
+- 20190614: First version <daniel.sangorrin@...>
+- 20190618: Add workflow information provided by Ben
+- 20190711: Add tag reporting <daniel.sangorrin@...>
+
--
2.17.1


(No subject)

Daniel Sangorrin <daniel.sangorrin@...>
 

Hello Ben,

Sorry, I realized that there were a few issues in the Quickstart
so I am resending the patch. Please ignore the previous one.

[cip-kernel-sec][Quickstart v2] docs: add a quickstart with practical

Thanks,
Daniel


[cip-kernel-sec] readme: add info about tag_regexp and show-description

Daniel Sangorrin <daniel.sangorrin@...>
 

Probably this should be squashed into the corresponding
patches.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
README.md | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 576cc75..8164826 100644
--- a/README.md
+++ b/README.md
@@ -41,7 +41,8 @@ current or previous year or that are already tracked here.
stable and other configured branches, by reading the git commit logs.

* `scripts/report_affected.py` - report which issues affect the
-specified branches, or all active branches.
+specified branches, or all active branches. You can use --show-description
+to obtain a short description for each CVE ID.

* `scripts/validate.py` - validate all issue files against the
schema.
@@ -72,6 +73,7 @@ keys:
* `base_ver`: Stable version that the branch is based on, e.g.
"4.4". This needs to be quoted so that it's a string not a
number.
+* `tag_regexp`: A regular expression that matches tags on a branch.

### Remotes

--
2.17.1


[cip-kernel-sec][quickstart] docs: add a quickstart with practical information

Daniel Sangorrin <daniel.sangorrin@...>
 

Although the README already contains all the information
that users may need, there are some bits of know-how that
are better expressed through a step-by-step quickstart or
tutorial. This files tries to fill that gap.

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
QUICKSTART.md | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 126 insertions(+)
create mode 100644 QUICKSTART.md

diff --git a/QUICKSTART.md b/QUICKSTART.md
new file mode 100644
index 0000000..4bceb97
--- /dev/null
+++ b/QUICKSTART.md
@@ -0,0 +1,126 @@
+# Quickstart
+
+## Overview
+
+This project tracks the status of CVEs in mainline and stable kernels. Each CVE is described in YAML format that includes data such as:
+
+```
+$ cat CVE-2019-1999.yml
+description: 'binder: fix race between munmap() and direct reclaim'
+references:
+- https://source.android.com/security/bulletin/2019-02-01
+comments:
+ Debian-bwh: |-
+ Introduced in 4.14 by f2517eb76f1f "android: binder: Add global lru
+ shrinker to binder". Backports of the fix to stable have incorrect
+ metadata.
+ bwh: Backports to stable have incorrect metadata
+introduced-by:
+ mainline: [f2517eb76f1f2f7f89761f9db2b202e89931738c]
+fixed-by:
+ linux-4.14.y: [33c6b9ca70a8b066a613e2a3d0331ae8f82aa31a]
+ linux-4.19.y: [6bf7d3c5c0c5dad650bfc4345ed553c18b69d59e]
+ linux-5.0.y: [bbb19ca082ce27ce60ca65be016a951806ea947c]
+ mainline: [5cec2d2e5839f9c0fec319c523a911e0a7fd299f]
+```
+
+## Quickstart
+
+Clone `cip-kernel-sec` and install its dependencies:
+
+```
+$ git clone https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec
+$ cd cip-kernel-sec/
+$ sudo apt install python3-yaml and python3-html5lib python3-cherrypy3 python3-jinja2
+```
+
+Prepare kernel remote repositories according to `conf/remotes.yml`:
+
+```
+$ ./scripts/prepare_remotes.py
+```
+
+Alternatively, you can do that manually:
+
+```
+$ mkdir ../kernel
+$ cd ../kernel
+$ git remote add torvalds https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+$ git remote add stable https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
+$ git remote add cip https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git
+$ cd ../cip-kernel-sec
+```
+
+Download CVE information from Debian (https://salsa.debian.org/kernel-team/kernel-sec.git), Ubuntu (https://git.launchpad.net/ubuntu-cve-tracker) and Stable:
+
+```
+
+$ ./scripts/import_debian.py
+ -> import/debian
+$ ./scripts/import_ubuntu.py
+ -> import/ubuntu
+$ ./scripts/import_stable.py
+ -> import/stable_branches.yml
+```
+
+Check issues that affect a linux-cip branch:
+
+```
+$ ./scripts/report_affected.py linux-4.4.y
+```
+
+Check issues that affect a tag:
+
+```
+$ ./scripts/report_affected.py v4.4.181-cip33
+```
+
+Browse kernel branches and issues interactively:
+
+```
+$ ./scripts/webview.py
+$ firefox http://localhost:8080
+```
+
+[Note] Use Ctr-c to stop the `webview.py` script.
+
+## Kernel maintainer workflow
+
+Import or update the latest CVE information:
+
+```
+$ ./scripts/import_debian.py
+$ ./scripts/import_ubuntu.py
+$ ./scripts/import_stable.py
+```
+
+Edit by hand the newly created issues if you see that some imported information is incorrect or there is missing information:
+
+```
+$ vi issues/CVE-xx.yml
+```
+
+Validate the issue files against the YAML schema.
+
+```
+$ ./scripts/validate.py
+```
+
+YAML allows the same thing to be written in different ways, e.g. bracketed vs bulleted lists. Use `cleanup.py` to make the syntax and ordering of items consistent with the importers, to reduce "noise" in diffs:
+
+```
+$ ./scripts/cleanup.py
+```
+
+Check if the current issues:
+
+```
+$ ./scripts/report_affected.py
+```
+
+## Changelog
+
+- 20190614: First version <daniel.sangorrin@...>
+- 20190618: Add workflow information provided by Ben
+- 20190711: Add tag reporting <daniel.sangorrin@...>
+
--
2.17.1


Add quickstart

Daniel Sangorrin <daniel.sangorrin@...>
 

Hello Ben,

I was planning to put this into the CIP wiki, but I though it might
be better to store it together with the source code so there you go.

[cip-kernel-sec][quickstart] docs: add a quickstart with practical

Thanks,
Daniel


[cip-kernel-sec][RESEND v3] report_affected: add support for reporting on tags

Daniel Sangorrin <daniel.sangorrin@...>
 

Reporting on tags is useful for product engineers that
have shipped a kernel with a specific tag and need to know
which issues affect their product after some time.

Examples:
$ ./scripts/report_affected.py v4.4 v4.4.107 v4.4.181-cip33
$ cd ../kernel
$ git tag myproduct-v1 0f13d9b4d0efa9e87381717c113df57718bc92d6
$ cd ../cip-kernel-sec
$ ./scripts/report_affected.py linux-4.19.y-cip:myproduct-v1 v4.19.50-cip3

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
conf/branches.yml | 2 ++
scripts/kernel_sec/branch.py | 8 +++--
scripts/report_affected.py | 68 +++++++++++++++++++++++++++++++-----
3 files changed, 66 insertions(+), 12 deletions(-)

diff --git a/conf/branches.yml b/conf/branches.yml
index 2ed9db6..8197596 100644
--- a/conf/branches.yml
+++ b/conf/branches.yml
@@ -2,7 +2,9 @@
base_ver: "4.4"
git_remote: cip
git_name: linux-4.4.y-cip
+ tag_regexp: '^v4\.4\.\d+-cip\d+$'
- short_name: linux-4.19.y-cip
base_ver: "4.19"
git_remote: cip
git_name: linux-4.19.y-cip
+ tag_regexp: '^v4\.19\.\d+-cip\d+$'
diff --git a/scripts/kernel_sec/branch.py b/scripts/kernel_sec/branch.py
index 9a7bc3a..cd4f423 100644
--- a/scripts/kernel_sec/branch.py
+++ b/scripts/kernel_sec/branch.py
@@ -23,11 +23,13 @@ from . import version

def get_base_ver_stable_branch(base_ver):
branch_name = 'linux-%s.y' % base_ver
+ esc_base_ver = re.escape(base_ver)
return {
'short_name': branch_name,
'git_remote': 'stable',
'git_name': branch_name,
- 'base_ver': base_ver
+ 'base_ver': base_ver,
+ 'tag_regexp' : r'(^v%s$|^v%s\.\d+$)' % (esc_base_ver, esc_base_ver)
}


@@ -141,7 +143,7 @@ def get_sort_key(branch):
return version.get_sort_key(base_ver)


-def _get_commits(git_repo, end, start=None):
+def iter_rev_list(git_repo, end, start=None):
if start:
list_expr = '%s..%s' % (start, end)
else:
@@ -170,7 +172,7 @@ class CommitBranchMap:
branch['git_name'])
else:
end = 'v' + branch['base_ver']
- for commit in _get_commits(git_repo, end, start):
+ for commit in iter_rev_list(git_repo, end, start):
self._commit_sort_key[commit] \
= self._branch_sort_key[branch_name]
start = end
diff --git a/scripts/report_affected.py b/scripts/report_affected.py
index 0966fe1..27c39ef 100755
--- a/scripts/report_affected.py
+++ b/scripts/report_affected.py
@@ -9,7 +9,9 @@
# Report issues affecting each stable branch.

import argparse
+import copy
import subprocess
+import re

import kernel_sec.branch
import kernel_sec.issue
@@ -22,15 +24,38 @@ def main(git_repo, remotes,
if branch_names:
branches = []
for branch_name in branch_names:
+ tag = None
if branch_name[0].isdigit():
# 4.4 is mapped to linux-4.4.y
name = 'linux-%s.y' % branch_name
+ elif branch_name[0] == 'v':
+ # an official tag, e.g. v4.4.92-cip11
+ # infer branch from tag (regexp's must be specific)
+ for branch in live_branches:
+ if 'tag_regexp' not in branch:
+ # no tag_regexp defined, or mainline
+ continue
+
+ # predefined in branches.yml or a stable branch
+ if re.match(branch['tag_regexp'], branch_name):
+ tag = branch_name
+ name = branch['short_name']
+ break
+ else:
+ raise ValueError('Failed to match tag %r' % branch_name)
+ elif ':' in branch_name:
+ # a possibly custom tag, e.g. linux-4.19.y-cip:myproduct-v1
+ name, tag = branch_name.split(':', 1)
else:
name = branch_name

for branch in live_branches:
if branch['short_name'] == name:
- branches.append(branch)
+ # there could be multiple tags for the same branch
+ branch_copy = copy.deepcopy(branch)
+ if tag:
+ branch_copy['tag'] = tag
+ branches.append(branch_copy)
break
else:
msg = "Branch %s could not be found" % branch_name
@@ -45,6 +70,18 @@ def main(git_repo, remotes,

c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)

+ # cache tag commits and set full_name to show the tag
+ tag_commits = {}
+ for branch in branches:
+ if 'tag' in branch:
+ start = 'v' + branch['base_ver']
+ end = branch['tag']
+ tag_commits[end] = set(
+ kernel_sec.branch.iter_rev_list(git_repo, end, start))
+ branch['full_name'] = ':'.join([branch['short_name'], end])
+ else:
+ branch['full_name'] = branch['short_name']
+
branch_issues = {}
issues = set(kernel_sec.issue.get_list())

@@ -65,15 +102,26 @@ def main(git_repo, remotes,
if not include_ignored and ignore.get(branch_name):
continue

+ # Check if the branch is affected. If not and the issue was fixed
+ # on that branch, then make sure the tag contains that fix
if kernel_sec.issue.affects_branch(
issue, branch, c_b_map.is_commit_in_branch):
- branch_issues.setdefault(branch_name, []).append(cve_id)
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ elif 'tag' in branch and fixed:
+ if fixed.get(branch_name, 'never') == 'never':
+ continue
+ for commit in fixed[branch_name]:
+ if commit not in tag_commits[branch['tag']]:
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ break

for branch in branches:
- branch_name = branch['short_name']
- print('%s:' % branch_name,
- *sorted(branch_issues.get(branch_name, []),
- key=kernel_sec.issue.get_id_sort_key))
+ sorted_cve_ids = sorted(
+ branch_issues.get(branch['full_name'], []),
+ key=kernel_sec.issue.get_id_sort_key)
+ print('%s:' % branch['full_name'], *sorted_cve_ids)


if __name__ == '__main__':
@@ -104,9 +152,11 @@ if __name__ == '__main__':
help='include issues that have been marked as ignored')
parser.add_argument('branches',
nargs='*',
- help=('specific branch to report on '
- '(default: all active branches)'),
- metavar='BRANCH')
+ help=('specific branch[:tag] or stable tag to '
+ 'report on (default: all active branches). '
+ 'e.g. linux-4.14.y linux-4.4.y:v4.4.107 '
+ 'v4.4.181-cip33 linux-4.19.y-cip:myproduct-v33'),
+ metavar='[BRANCH[:TAG]|TAG]')
args = parser.parse_args()
remotes = kernel_sec.branch.get_remotes(args.remote_name,
mainline=args.mainline_remote_name,
--
2.17.1


(Resend v3) move the tag with the other fields

Daniel Sangorrin <daniel.sangorrin@...>
 

Hello Ben,

I think this should be fine.
You need to remove the cached file import/stable_branches.yml

[cip-kernel-sec][RESEND v3] report_affected: add support for

To be honest, I am not 100% sure that the escape code is
perfectly correct. The file stable_branches.yml will have
parameters like this:

tag_regexp: (^v3\.16$|^v3\.16\.\d+$)

Thanks,
Daniel


Re: [cip-kernel-sec][RESEND v2 1/2] report_affected: add support for reporting on tags

daniel.sangorrin@...
 

+ for branch in branches:
+ esc_base_ver = branch['base_ver'].replace('.', re.escape('.'))
+ # example tags: v4.4, v4.19.12
+ tag_regexp = r'(^v%s$|^v%s\.\d+$)' % (esc_base_ver, esc_base_ver)
+ branch['tag_regexp'] = tag_regexp
+
Sorry, I went too fast. I should have moved the regexp to get_base_ver_stable_branch. I will resend.

Thanks,
Daniel


branches.extend(_get_configured_branches('conf/branches.yml'))
branches.extend(
_get_configured_branches(
@@ -141,7 +148,7 @@ def get_sort_key(branch):
return version.get_sort_key(base_ver)


-def _get_commits(git_repo, end, start=None):
+def iter_rev_list(git_repo, end, start=None):
if start:
list_expr = '%s..%s' % (start, end)
else:
@@ -170,7 +177,7 @@ class CommitBranchMap:
branch['git_name'])
else:
end = 'v' + branch['base_ver']
- for commit in _get_commits(git_repo, end, start):
+ for commit in iter_rev_list(git_repo, end, start):
self._commit_sort_key[commit] \
= self._branch_sort_key[branch_name]
start = end
diff --git a/scripts/report_affected.py b/scripts/report_affected.py
index 0966fe1..27c39ef 100755
--- a/scripts/report_affected.py
+++ b/scripts/report_affected.py
@@ -9,7 +9,9 @@
# Report issues affecting each stable branch.

import argparse
+import copy
import subprocess
+import re

import kernel_sec.branch
import kernel_sec.issue
@@ -22,15 +24,38 @@ def main(git_repo, remotes,
if branch_names:
branches = []
for branch_name in branch_names:
+ tag = None
if branch_name[0].isdigit():
# 4.4 is mapped to linux-4.4.y
name = 'linux-%s.y' % branch_name
+ elif branch_name[0] == 'v':
+ # an official tag, e.g. v4.4.92-cip11
+ # infer branch from tag (regexp's must be specific)
+ for branch in live_branches:
+ if 'tag_regexp' not in branch:
+ # no tag_regexp defined, or mainline
+ continue
+
+ # predefined in branches.yml or a stable branch
+ if re.match(branch['tag_regexp'], branch_name):
+ tag = branch_name
+ name = branch['short_name']
+ break
+ else:
+ raise ValueError('Failed to match tag %r' % branch_name)
+ elif ':' in branch_name:
+ # a possibly custom tag, e.g. linux-4.19.y-cip:myproduct-v1
+ name, tag = branch_name.split(':', 1)
else:
name = branch_name

for branch in live_branches:
if branch['short_name'] == name:
- branches.append(branch)
+ # there could be multiple tags for the same branch
+ branch_copy = copy.deepcopy(branch)
+ if tag:
+ branch_copy['tag'] = tag
+ branches.append(branch_copy)
break
else:
msg = "Branch %s could not be found" % branch_name
@@ -45,6 +70,18 @@ def main(git_repo, remotes,

c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)

+ # cache tag commits and set full_name to show the tag
+ tag_commits = {}
+ for branch in branches:
+ if 'tag' in branch:
+ start = 'v' + branch['base_ver']
+ end = branch['tag']
+ tag_commits[end] = set(
+ kernel_sec.branch.iter_rev_list(git_repo, end, start))
+ branch['full_name'] = ':'.join([branch['short_name'], end])
+ else:
+ branch['full_name'] = branch['short_name']
+
branch_issues = {}
issues = set(kernel_sec.issue.get_list())

@@ -65,15 +102,26 @@ def main(git_repo, remotes,
if not include_ignored and ignore.get(branch_name):
continue

+ # Check if the branch is affected. If not and the issue was fixed
+ # on that branch, then make sure the tag contains that fix
if kernel_sec.issue.affects_branch(
issue, branch, c_b_map.is_commit_in_branch):
- branch_issues.setdefault(branch_name, []).append(cve_id)
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ elif 'tag' in branch and fixed:
+ if fixed.get(branch_name, 'never') == 'never':
+ continue
+ for commit in fixed[branch_name]:
+ if commit not in tag_commits[branch['tag']]:
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ break

for branch in branches:
- branch_name = branch['short_name']
- print('%s:' % branch_name,
- *sorted(branch_issues.get(branch_name, []),
- key=kernel_sec.issue.get_id_sort_key))
+ sorted_cve_ids = sorted(
+ branch_issues.get(branch['full_name'], []),
+ key=kernel_sec.issue.get_id_sort_key)
+ print('%s:' % branch['full_name'], *sorted_cve_ids)


if __name__ == '__main__':
@@ -104,9 +152,11 @@ if __name__ == '__main__':
help='include issues that have been marked as ignored')
parser.add_argument('branches',
nargs='*',
- help=('specific branch to report on '
- '(default: all active branches)'),
- metavar='BRANCH')
+ help=('specific branch[:tag] or stable tag to '
+ 'report on (default: all active branches). '
+ 'e.g. linux-4.14.y linux-4.4.y:v4.4.107 '
+ 'v4.4.181-cip33 linux-4.19.y-cip:myproduct-v33'),
+ metavar='[BRANCH[:TAG]|TAG]')
args = parser.parse_args()
remotes = kernel_sec.branch.get_remotes(args.remote_name,
mainline=args.mainline_remote_name,
--
2.17.1

_______________________________________________
cip-dev mailing list
cip-dev@...
https://lists.cip-project.org/mailman/listinfo/cip-dev


Re: [cip-kernel-sec][RESEND 4/6] report_affected: add support for reporting on tags

daniel.sangorrin@...
 

From: Ben Hutchings <ben.hutchings@...>
[...]
+ if 'tag_regexp' in branch:
+ # predefined in conf/branches.yml
+ tag_regexp = branch['tag_regexp']
+ elif branch['git_remote'] == 'stable':
+ # stable format, e.g. v4.19.12
+ esc_base_ver = branch['base_ver'].replace('.', '\.')
This happens to work now, but '\.' is an unrecognised escape sequence
which is deprecated. You presumably meant r'\.', but it might be
clearer to use re.escape().
Thanks, you are right. I have used re.escape('.'). I didn't know this function, it's really useful not having to remember how to escape characters (I figured out I could have used \\.).

I think that the YAML strings in conf/branches.yml are being read as if they were raw strings, but if there is a problem with them let me know.


+ tag_regexp = r'(^v%s$|^v%s\.\d+$)' % (
+ esc_base_ver, esc_base_ver)
I also expected that you would set tag_regexp for stable branches in
the branch module along with all their other fields. Then there's no
need to handle them specially here.
Ah sorry about that. I have moved that code to branch.py now.


+ else:
+ # no tag_regexp defined, or mainline
+ continue
+
+ if re.match(tag_regexp, branch_name):
+ tag = branch_name
+ name = branch['short_name']
+ break
+ else:
+ raise ValueError('Failed to match tag %r' % branch_name)
+ elif ':' in branch_name:
+ # a possibly custom tag, e.g. linux-4.19.y-cip:myproduct-v1
+ name_tuple = tuple(branch_name.split(':'))
+ name = name_tuple[0]
+ tag = name_tuple[1]
[...]

You really can do simply:

name, tag = branch_name.split(':', 1)

(Tuple assignment only requires an iterable, not specifically a tuple,
on the right hand side.) So please use that.
Nice trick. I fixed the code.

Thanks,
Daniel



Ben.

--
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom


[cip-kernel-sec][RESEND v2 2/2] report_affected: add show-description option

Daniel Sangorrin <daniel.sangorrin@...>
 

Rather than looking up each issue file, I would like
to have an overview of what each CVE ID means.

Example:
$ ./scripts/report_affected.py --show-description linux-4.4.y-cip

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
scripts/report_affected.py | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/scripts/report_affected.py b/scripts/report_affected.py
index 27c39ef..22a923b 100755
--- a/scripts/report_affected.py
+++ b/scripts/report_affected.py
@@ -18,8 +18,8 @@ import kernel_sec.issue
import kernel_sec.version


-def main(git_repo, remotes,
- only_fixed_upstream, include_ignored, *branch_names):
+def main(git_repo, remotes, only_fixed_upstream,
+ include_ignored, show_description, *branch_names):
live_branches = kernel_sec.branch.get_live_branches()
if branch_names:
branches = []
@@ -121,7 +121,13 @@ def main(git_repo, remotes,
sorted_cve_ids = sorted(
branch_issues.get(branch['full_name'], []),
key=kernel_sec.issue.get_id_sort_key)
- print('%s:' % branch['full_name'], *sorted_cve_ids)
+ if show_description:
+ print('%s:' % branch['full_name'])
+ for cve_id in sorted_cve_ids:
+ print(cve_id, '=>',
+ kernel_sec.issue.load(cve_id).get('description', 'None'))
+ else:
+ print('%s:' % branch['full_name'], *sorted_cve_ids)


if __name__ == '__main__':
@@ -150,6 +156,9 @@ if __name__ == '__main__':
parser.add_argument('--include-ignored',
action='store_true',
help='include issues that have been marked as ignored')
+ parser.add_argument('--show-description',
+ action='store_true',
+ help='show the issue description')
parser.add_argument('branches',
nargs='*',
help=('specific branch[:tag] or stable tag to '
@@ -162,5 +171,5 @@ if __name__ == '__main__':
mainline=args.mainline_remote_name,
stable=args.stable_remote_name)
kernel_sec.branch.check_git_repo(args.git_repo, remotes)
- main(args.git_repo, remotes,
- args.only_fixed_upstream, args.include_ignored, *args.branches)
+ main(args.git_repo, remotes, args.only_fixed_upstream,
+ args.include_ignored, args.show_description, *args.branches)
--
2.17.1


[cip-kernel-sec][RESEND v2 1/2] report_affected: add support for reporting on tags

Daniel Sangorrin <daniel.sangorrin@...>
 

Reporting on tags is useful for product engineers that
have shipped a kernel with a specific tag and need to know
which issues affect their product after some time.

Examples:
$ ./scripts/report_affected.py v4.4 v4.4.107 v4.4.181-cip33
$ cd ../kernel
$ git tag myproduct-v1 0f13d9b4d0efa9e87381717c113df57718bc92d6
$ cd ../cip-kernel-sec
$ ./scripts/report_affected.py linux-4.19.y-cip:myproduct-v1 v4.19.50-cip3

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@...>
---
conf/branches.yml | 2 ++
scripts/kernel_sec/branch.py | 11 ++++--
scripts/report_affected.py | 68 +++++++++++++++++++++++++++++++-----
3 files changed, 70 insertions(+), 11 deletions(-)

diff --git a/conf/branches.yml b/conf/branches.yml
index 2ed9db6..8197596 100644
--- a/conf/branches.yml
+++ b/conf/branches.yml
@@ -2,7 +2,9 @@
base_ver: "4.4"
git_remote: cip
git_name: linux-4.4.y-cip
+ tag_regexp: '^v4\.4\.\d+-cip\d+$'
- short_name: linux-4.19.y-cip
base_ver: "4.19"
git_remote: cip
git_name: linux-4.19.y-cip
+ tag_regexp: '^v4\.19\.\d+-cip\d+$'
diff --git a/scripts/kernel_sec/branch.py b/scripts/kernel_sec/branch.py
index 9a7bc3a..1922419 100644
--- a/scripts/kernel_sec/branch.py
+++ b/scripts/kernel_sec/branch.py
@@ -121,6 +121,13 @@ def _get_configured_branches(filename):

def get_live_branches():
branches = _get_live_stable_branches()
+ # add regular expressions to infer a stable branch from a stable tag
+ for branch in branches:
+ esc_base_ver = branch['base_ver'].replace('.', re.escape('.'))
+ # example tags: v4.4, v4.19.12
+ tag_regexp = r'(^v%s$|^v%s\.\d+$)' % (esc_base_ver, esc_base_ver)
+ branch['tag_regexp'] = tag_regexp
+
branches.extend(_get_configured_branches('conf/branches.yml'))
branches.extend(
_get_configured_branches(
@@ -141,7 +148,7 @@ def get_sort_key(branch):
return version.get_sort_key(base_ver)


-def _get_commits(git_repo, end, start=None):
+def iter_rev_list(git_repo, end, start=None):
if start:
list_expr = '%s..%s' % (start, end)
else:
@@ -170,7 +177,7 @@ class CommitBranchMap:
branch['git_name'])
else:
end = 'v' + branch['base_ver']
- for commit in _get_commits(git_repo, end, start):
+ for commit in iter_rev_list(git_repo, end, start):
self._commit_sort_key[commit] \
= self._branch_sort_key[branch_name]
start = end
diff --git a/scripts/report_affected.py b/scripts/report_affected.py
index 0966fe1..27c39ef 100755
--- a/scripts/report_affected.py
+++ b/scripts/report_affected.py
@@ -9,7 +9,9 @@
# Report issues affecting each stable branch.

import argparse
+import copy
import subprocess
+import re

import kernel_sec.branch
import kernel_sec.issue
@@ -22,15 +24,38 @@ def main(git_repo, remotes,
if branch_names:
branches = []
for branch_name in branch_names:
+ tag = None
if branch_name[0].isdigit():
# 4.4 is mapped to linux-4.4.y
name = 'linux-%s.y' % branch_name
+ elif branch_name[0] == 'v':
+ # an official tag, e.g. v4.4.92-cip11
+ # infer branch from tag (regexp's must be specific)
+ for branch in live_branches:
+ if 'tag_regexp' not in branch:
+ # no tag_regexp defined, or mainline
+ continue
+
+ # predefined in branches.yml or a stable branch
+ if re.match(branch['tag_regexp'], branch_name):
+ tag = branch_name
+ name = branch['short_name']
+ break
+ else:
+ raise ValueError('Failed to match tag %r' % branch_name)
+ elif ':' in branch_name:
+ # a possibly custom tag, e.g. linux-4.19.y-cip:myproduct-v1
+ name, tag = branch_name.split(':', 1)
else:
name = branch_name

for branch in live_branches:
if branch['short_name'] == name:
- branches.append(branch)
+ # there could be multiple tags for the same branch
+ branch_copy = copy.deepcopy(branch)
+ if tag:
+ branch_copy['tag'] = tag
+ branches.append(branch_copy)
break
else:
msg = "Branch %s could not be found" % branch_name
@@ -45,6 +70,18 @@ def main(git_repo, remotes,

c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)

+ # cache tag commits and set full_name to show the tag
+ tag_commits = {}
+ for branch in branches:
+ if 'tag' in branch:
+ start = 'v' + branch['base_ver']
+ end = branch['tag']
+ tag_commits[end] = set(
+ kernel_sec.branch.iter_rev_list(git_repo, end, start))
+ branch['full_name'] = ':'.join([branch['short_name'], end])
+ else:
+ branch['full_name'] = branch['short_name']
+
branch_issues = {}
issues = set(kernel_sec.issue.get_list())

@@ -65,15 +102,26 @@ def main(git_repo, remotes,
if not include_ignored and ignore.get(branch_name):
continue

+ # Check if the branch is affected. If not and the issue was fixed
+ # on that branch, then make sure the tag contains that fix
if kernel_sec.issue.affects_branch(
issue, branch, c_b_map.is_commit_in_branch):
- branch_issues.setdefault(branch_name, []).append(cve_id)
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ elif 'tag' in branch and fixed:
+ if fixed.get(branch_name, 'never') == 'never':
+ continue
+ for commit in fixed[branch_name]:
+ if commit not in tag_commits[branch['tag']]:
+ branch_issues.setdefault(
+ branch['full_name'], []).append(cve_id)
+ break

for branch in branches:
- branch_name = branch['short_name']
- print('%s:' % branch_name,
- *sorted(branch_issues.get(branch_name, []),
- key=kernel_sec.issue.get_id_sort_key))
+ sorted_cve_ids = sorted(
+ branch_issues.get(branch['full_name'], []),
+ key=kernel_sec.issue.get_id_sort_key)
+ print('%s:' % branch['full_name'], *sorted_cve_ids)


if __name__ == '__main__':
@@ -104,9 +152,11 @@ if __name__ == '__main__':
help='include issues that have been marked as ignored')
parser.add_argument('branches',
nargs='*',
- help=('specific branch to report on '
- '(default: all active branches)'),
- metavar='BRANCH')
+ help=('specific branch[:tag] or stable tag to '
+ 'report on (default: all active branches). '
+ 'e.g. linux-4.14.y linux-4.4.y:v4.4.107 '
+ 'v4.4.181-cip33 linux-4.19.y-cip:myproduct-v33'),
+ metavar='[BRANCH[:TAG]|TAG]')
args = parser.parse_args()
remotes = kernel_sec.branch.get_remotes(args.remote_name,
mainline=args.mainline_remote_name,
--
2.17.1


(Resend v2) report issues for tags

Daniel Sangorrin <daniel.sangorrin@...>
 

Hello Ben,

Thanks again for your detailed reviews. I have fixed the tags
patch with your suggestions. I will reply to them separately.

[cip-kernel-sec][RESEND v2 1/2] report_affected: add support for
[cip-kernel-sec][RESEND v2 2/2] report_affected: add show-description

Thanks,
Daniel


CIP IRC weekly meeting today

SZ Lin (林上智) <SZ.Lin@...>
 

Hi all,

 

Kindly be reminded to attend the weekly meeting through IRC to discuss technical topics with CIP kernel today.

 

*Please note that IRC meeting was rescheduled to UTC (GMT) 09:00 starting from the first week of Apr. according to TSC meeting*

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2019&month=7&day11&hour=9&min=0&sec=0&p1=241&p2=137&p3=179&p4=136&p5=37&p6=248

 

US-West US-East   UK     DE     TW     JP

02:00    05:00   10:00   11:00   17:00   18:00

 

Channel:

* irc:chat.freenode.net:6667/cip

 

Agenda:

 

* Action item

1. Provide the script for CIP kernel config collection - bwh

#link https://lists.cip-project.org/pipermail/cip-dev/2019-June/002506.html

2. List real time kernel questions to ask Daniel Wagner - szlin

3. Try updating CIP RT kernel to 4.19.50 - Pavel

#link https://lists.cip-project.org/pipermail/cip-dev/2019-June/002548.html

4. Work out a solution for LAVA master backups - patersonc

* Kernel maintenance updates

* Kernel testing

* CIP Core

* Software update

* AOB

 

The meeting will take 30 min, although it can be extended to an hour if it makes sense and those involved in the topics can stay. Otherwise, the topic will be taken offline or in the next meeting.

 

Best regards,

 

SZ Lin, Moxa.

7521 - 7540 of 10124