From: Bhola, Bikram <Bikram_Bhola@mentor.com>
Sent: 30 September 2021 12:19
We investigated the failure job and looks like before getting login prompt job
timeout is happening . In the job definition file - job timeout is mentioned
15mins and sometimes due to slow network issue, it takes more time while
downloading, untar and deploying image. So we are seeing timeout during
login prompt or in some cases in earlier stages also. The work in progress to
double up the network bandwidth within a few weeks, which will reduce the
occurrence of this type of issues.
Thank you for your investigation.
I've have increased the timeout as you have suggested:https://gitlab.com/cip-project/cip-testing/linux-cip-ci/-/merge_requests/49
One additional thing I've noticed, the default x86 character delay during boot is 500ms, which seems a long time inbetween each character sent to the platformhttps://lava.ciplatform.org/scheduler/device/x86-simatic-ipc227e-01/devicedict#defline5
Has a lower value for boot_character_delay ever been tried?
Kind regards, Chris
Time being, with an increased job timeout to 20mins, failure is not observed.
We tested 10 times to be working fine.
changes in the job definition file
Need to Modify
From: Chris Paterson <Chris.Paterson2@renesas.com>
Sent: 28 September 2021 15:38
To: Pavel Machek <email@example.com>; Bhola, Bikram
Cc: firstname.lastname@example.org; Jan Kiszka <email@example.com>
Subject: RE: Prompt timeouts on ipc227e board -- randomness related?
From: Pavel Machek <firstname.lastname@example.org>Thank you for reporting the issue.
Sent: 25 September 2021 21:06
It is not first time I see this failure:
Bikram is going to take a look for us (thank you).
Kind regards, Chris
[[0;32m OK [0m] Started Login Service.
[[0m[0;31m* [0m] (1 of 2) A start job is running for…ate sshd host keys
no limit)[K[[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is runningfor…ate
sshd host keys (8s / no limit)[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2)for…evices-
A start job is running for…ate sshd host keys (9s / no limit)[K[
[0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running
eth0.device (8s / 1min 30s)[ 19.855328] systemd: apt-daily-47.041488s
upgrade.timer: Adding 3min 2.027476s random time.
[ 19.864207] systemd: apt-daily.timer: Adding 1h 54min 15.794344s
[ 21.406490] systemd: apt-daily-upgrade.timer: Adding 55min
[ 21.415357] systemd: apt-daily.timer: Adding 11h 48min 4.457495s
[ 22.049807] systemd: apt-daily-upgrade.timer: Adding 3min 54.125406s
[ 22.058500] systemd: apt-daily.timer: Adding 8h 34min 47.388595s
[ 22.511646] systemd: apt-daily-upgrade.timer: Adding 25min
[ 22.520510] systemd: apt-daily.timer: Adding 11h 58min 24.212170s
[K[[0;32m OK [0m] Started Regenerate sshd host keys.
wait for prompt timed out
end: 220.127.116.11 login-action (duration 00:00:24) [common]
Any idea what is going on there? Is it just a test problem, or do we
have kernel regression that only happens sometimes?
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany