Re: I need de0-nano testing for -rt release was Re: 4.19.106-cip21-rt8 problems on de0-nano


Jan Kiszka
 

On 09.03.20 11:21, Pavel Machek wrote:
Hi!

I pushed candidate for -cip-rt, but it seems to fail on de0-nano
board. Code under testing is at:

https://gitlab.com/cip-project/cip-kernel/linux-cip/tree/ci/pavel/linux-cip
It is pipeline

https://gitlab.com/cip-project/cip-kernel/linux-cip/pipelines/122762401

I'll reuse the branch for more testing.
I managed to narrow the bad commit to the -rt tree, between:

OK 122904930 pick 69aa73357e6a rcu: Don't allow to change rcu_normal_after_boot on RT
pick 849ef8789077 pci/switchtec: fix stream_open.cocci warnings
pick ad8a5e8279c4 sched/core: Drop a preempt_disable_rt() statement
pick 966f066d96cb timers: Redo the notification of canceling timers on -RT
pick 0393fd5a4f9a Revert "futex: Ensure lock/unlock symetry versus pi_lock and hash bucket lock"
pick 84eb0b64a27a Revert "futex: Fix bug on when a requeued RT task times out"
pick fcc893280f4e Revert "rtmutex: Handle the various new futex race conditions"
pick 2eac93cf9d16 Revert "futex: workaround migrate_disable/enable in different context"
pick 9b8964629f4f futex: Make the futex_hash_bucket lock raw
pick cc1812bf198b futex: Delay deallocation of pi_state
pick f5e115c43100 mm/zswap: Do not disable preemption in zswap_frontswap_store()
pick e0d0d09a08ad revert-aio
pick a0a40bfb4300 fs/aio: simple simple work
pick 0fae581d8c5e revert-thermal
pick c0d95b4a8a1b thermal: Defer thermal wakups to threads
pick 700fbb4afb6e revert-block
pick 4cda50ff12cf block: blk-mq: move blk_queue_usage_counter_release() into process context
pick 9e982f55745b workqueue: rework
pick c0db53dc3bf4 i2c: exynos5: Remove IRQF_ONESHOT
pick 1f160d170203 i2c: hix5hd2: Remove IRQF_ONESHOT
BAD 122882826 eae5a7cab722 sched/deadline: Ensure inactive_timer runs in hardirq context
And something went seriously wrong after these tests. I submitted same
tree twice, and got different results.

First this -- de0-nano succeeds:

https://gitlab.com/cip-project/cip-kernel/linux-cip/pipelines/122904930

Now this -- de0-nano fails (and ipc227e is unfinished for long time):

https://gitlab.com/cip-project/cip-kernel/linux-cip/pipelines/122959477

I'll need some help here.
The logs read like the targets are not (always) coming up, e.g.
https://gitlab.com/cip-project/cip-kernel/linux-cip/-/jobs/457824214#L377
Yes... I don't need that target, but I need de0-nano... and it did not
work last time I checked.
Bikram, could someone on your side check the board status in the Mentor lab? Thanks!

On a related note... it would be good to somehow show difference
between "kernel test failure" and "target failure".
If we see bootloader in the logs, and then test fails/timeouts =>
"kernel test failure", I need to solve it.
If we don't get messages from the bootloader => "target failure",
someone needs to check the power relays or something...
I'm not happy about the parsability of those LAVA logs either, but I have no idea if/how that can be improved best. Maybe Quirin has some idea based on his work with them.

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Join cip-dev@lists.cip-project.org to automatically receive all group messages.