On Tue, 2020-03-17 at 13:30 +0530, vijai kumar wrote:

On Tue, Mar 17, 2020 at 4:19 AM <nobuhiro1.iwamatsu@toshiba.co.jp> wrote:

[...]

I am now implementing LAVA's kselftest. This kselftest uses kseflftest included in the latest kernel.
There are no plans to backport tests to 4.4 or 4.19. I believe that ksefltest included in the CIP kernel
is inadequate in function and difficult to use as it is.
I suggest using a new kernel test that incorporates the latest tests.

Thank you Nobuhiro-San. Our original idea was to use the kselftest
from 4.19/4.4 cip tree. We faced some issues with
out-of-tree compilation while debianizing it. Also we faced some
compilation issues.

If that is the case, that we are going to use kselftest from the
latest kernel, how can we track the kselftest binaries for a
particular
cip kernel release? I am not sure if all the test cases would pass for
4.19/4/4. Do you have any plans to maintain a branch and release
kselftest binaries and/or testcases that have been tested with a
particular cip kernel release?

Some self-tests will fail on older kernel versions due to missing
features or bugs that might not be practically fixable. Test cases can
report "skip" rather than "fail" for missing features, but this is not
done consistently.

The way I've used kselftest with stable branches has been to look for
regressions rather than any failures. The Linaro folks are doing that
(reporting regressions only) when sending test results to the stable
list, but I don't know how or how much they have automated that.

Ben.

--
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom

On Tue, Mar 17, 2020 at 4:19 AM <nobuhiro1.iwamatsu@toshiba.co.jp> wrote:

Hi all,

-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
Sent: Tuesday, March 17, 2020 12:27 AM
To: vijai kumar <vijaikumar.kanagarajan@gmail.com>;
cip-dev@lists.cip-project.org; Chris Paterson
<Chris.Paterson2@renesas.com>; iwamatsu nobuhiro(岩松 信洋 ○ＳＷＣ□
ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>; Pavel Machek
<pavel@denx.de>
Subject: Re: [cip-dev] Linux-cip: Kselftest plans

On 06.03.20 09:21, vijai kumar wrote:

Hi All,

Is kselftest maintained in the cip-kernel tree? I do see some
branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
5.6 linux tree.

The reason being to gather collective thoughts about the plan for
kselftest based tests for cip kernel. Are there any existing or future
plans for using kselftest for testing cip kernels? If so are we going
to use the latest tree from upstream or plan to fix/backport test
cases to current cip kernel versions?

I am now implementing LAVA's kselftest. This kselftest uses kseflftest included in the latest kernel.
There are no plans to backport tests to 4.4 or 4.19. I believe that ksefltest included in the CIP kernel
is inadequate in function and difficult to use as it is.
I suggest using a new kernel test that incorporates the latest tests.

Thank you Nobuhiro-San. Our original idea was to use the kselftest
from 4.19/4.4 cip tree. We faced some issues with
out-of-tree compilation while debianizing it. Also we faced some
compilation issues.

If that is the case, that we are going to use kselftest from the
latest kernel, how can we track the kselftest binaries for a
particular
cip kernel release? I am not sure if all the test cases would pass for
4.19/4/4. Do you have any plans to maintain a branch and release
kselftest binaries and/or testcases that have been tested with a
particular cip kernel release?

Thanks,
Vijai Kumar K

Best regards,
Nobuhiro

Hi all,

On 06.03.20 09:21, vijai kumar wrote:

Hi All,
Is kselftest maintained in the cip-kernel tree? I do see some
branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
5.6 linux tree.
The reason being to gather collective thoughts about the plan for
kselftest based tests for cip kernel. Are there any existing or future
plans for using kselftest for testing cip kernels? If so are we going
to use the latest tree from upstream or plan to fix/backport test
cases to current cip kernel versions?
[1] https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest
Thanks,
Vijai Kumar K

CC'ing our kernel maintainers and Chris in case someone has input on this.

Vijai, maybe you can say a few words about your integration experience and options to move forward.

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Hi All,

Is kselftest maintained in the cip-kernel tree? I do see some
branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
5.6 linux tree.

The reason being to gather collective thoughts about the plan for
kselftest based tests for cip kernel. Are there any existing or future
plans for using kselftest for testing cip kernels? If so are we going
to use the latest tree from upstream or plan to fix/backport test
cases to current cip kernel versions?

[1] https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest

Thanks,
Vijai Kumar K