Re: [isar-cip-core][PATCH 1/2] ci: Rewrite using extends


Jan Kiszka
 

On 21.08.20 17:04, Jan Kiszka wrote:
On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@...>
---
.gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
scripts/deploy-cip-core.sh | 8 +++--
2 files changed, 60 insertions(+), 18 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3fe7af2..e23345f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2,10 +2,17 @@ image: kasproject/kas-isar:1.1

variables:
GIT_STRATEGY: clone
+ release: buster
+ extention: base
+ use_rt: enable
+ targz: enable
+ dtb: none

-all:
- stage: build
- script:
+stages:
+ - build
+
+default:
+ before_script:
- export http_proxy=$HTTP_PROXY
- export https_proxy=$HTTPS_PROXY
- export ftp_proxy=$FTP_PROXY
@@ -13,20 +20,51 @@ all:
- export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY

- - kas build kas-cip.yml:kas/board/simatic-ipc227e.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster simatic-ipc227e
-
+.build_base:
+ stage: build
+ variables:
+ base_yaml: "kas-cip.yml:kas/board/${target}.yml"
+ script:
- sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/bbb.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster bbb am335x-boneblack.dtb
+ - if [ "${use_rt}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/rt.yml"; fi;
+ - if [ "${extention}" != "base" ]; then base_yaml="${base_yaml}:kas/opt/${extention}.yml"; fi;
+ - if [ "${targz}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/targz-img.yml"; fi;
+ - kas build ${base_yaml}
+ - scripts/deploy-cip-core.sh ${release} ${target} ${extention} ${dtb}

- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/iwg20m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster iwg20m r8a7743-iwg20d-q7-dbcm-ca.dtb
+# base image
+build:simatic-ipc227e-base:
+ extends:
+ - .build_base
+ variables:
+ target: simatic-ipc227e

- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/rzg2m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster hihope-rzg2m renesas/r8a774a1-hihope-rzg2m-ex.dtb
+build:bbb-base:
+ extends:
+ - .build_base
+ variables:
+ target: bbb
+ dtb: am335x-boneblack.dtb

- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/security.yml
+build:iwg20m-base:
+ extends:
+ - .build_base
+ variables:
+ target: iwg20m
+ dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
+
+build:hihope-rzg2m-base:
+ extends:
+ - .build_base
+ variables:
+ target: rzg2m
+ dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
+
+build:qemu-amd64-base:
+ extends:
+ - .build_base
+ variables:
+ target: qemu-amd64
+ extention: security
+ use_rt: disable
+ targz: disable
diff --git a/scripts/deploy-cip-core.sh b/scripts/deploy-cip-core.sh
index 4c8d4c9..5b7eab9 100755
--- a/scripts/deploy-cip-core.sh
+++ b/scripts/deploy-cip-core.sh
@@ -16,9 +16,13 @@ fi

RELEASE=$1
TARGET=$2
-DTB=$3
+EXTENSION=$3
+DTB=$4

BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET
+if [ "${EXTENSION}" != "base" ] ; then
+ BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET-$EXTENSION
+fi

echo "Compressing cip-core-image-cip-core-$RELEASE-$TARGET.wic.img..."
xz -9 -k $BASE_PATH.wic.img
@@ -38,6 +42,6 @@ fi
aws s3 cp --no-progress $KERNEL_IMAGE s3://download.cip-project.org/cip-core/$TARGET/
aws s3 cp --no-progress $BASE_PATH-initrd.img s3://download.cip-project.org/cip-core/$TARGET/

-if [ -n "$DTB" ]; then
+if [ "$DTB" != "none" ]; then
aws s3 cp --no-progress build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB s3://download.cip-project.org/cip-core/$TARGET/
fi
Unfortunately, this scale out to multiple jobs seem to cause download
issues, see e.g.

- https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
- https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649

First thought it was a sporadic hic-up, but it reoccurs, now with
master. Is kernel.org throttling us here?
And I'm afraid there is more broken, namely in the deployment that is
only triggered over master. Please have a look at the failing jobs.

Thanks,
Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Join cip-dev@lists.cip-project.org to automatically receive all group messages.