Previously, these tests had the potential for timing out with the
default 5 second timeout value if initializing /dev/urandom took too
long.
Now the tests use a 10 second timeout value.
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 9bc3bbce38)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Prior to b7d251293a, txaio would test both asyncio and twisted.
Add back the twisted sample and include the twisted package in the
config so both modes of the package are tested.
Fixes: b7d251293a ("package/python-txaio: drop python 2 support")
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 41b08a779b)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since commit [1], test_atf is failing. See [2].
Passing -no-pie to ld does not produce the bl31.elf file (and does not
generate an error). TF-A 2.11 reworked its makefiles and switched to
gcc to link files. This commit fixes this failing test by updating to
this newer TF-A version.
Fixes: [2].
[1] 248c2d045e
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/8609307891
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit b0ff88423f)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The commit 6889056f1e (support/scripts/genimage.sh) adds support for
creating bmap images.
Since then, the script genimage.sh looses the error returned by the
genimage command.
As a consequence, the target-finalize target looses the error as well,
and so on up to make.
This adds the shell option -e to exit immediately if a command ends with
an error, so the error is not ignored.
Fixes:
gportay@archlinux ~/src/rtone-br2-external $ make
(...)
>>> Executing post-image script support/scripts/genimage.sh
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"" (stderr):
INFO: cmd: "rm -rf "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"/*" (stderr):
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"" (stderr):
INFO: cmd: "cp -a "/tmp/tmp.EclKGwtA4R" "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp/root"" (stderr):
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/images"" (stderr):
INFO: vfat(efi-part.vfat): cmd: "mkdosfs '/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat'" (stderr):
INFO: vfat(efi-part.vfat): adding file 'efi-part/EFI' as 'EFI' ...
INFO: vfat(efi-part.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -sp -i '/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat' '/home/gportay/src/rtone-br2-external/output/images/efi-part/EFI' '::EFI'" (stderr):
Disk full
INFO: vfat(efi-part.vfat): cmd: "rm -f "/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat"" (stderr):
ERROR: vfat(efi-part.vfat): failed to generate efi-part.vfat
INFO: cmd: "rm -rf "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp/"" (stderr):
make[1]: Leaving directory '/home/gportay/src/rtone-br2-external/buildroot'
gportay@archlinux ~/src/rtone-br2-external $ echo $?
0
Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 294d13bb85)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since commit d10d22221f (utils/get-developers: read patch from stdin
when it's not a tty), get-developers accepts to read a patch fromn its
stdin when it is not a tty.
Add a test for this.
Reported-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 29e1af8430)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
get-developers will check its stdin to decide whether it is a tty or
not, and behave differently whether it is or not. So, when we run the
tests, we need an actual tty.
However, when running in a CI pipeline, like on Gitlab-CI, there is no
tty available on stdin.
Fake one. We don't need anything too fancy, so just a slave pty will
suffice.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/8830671800
Fixes: d10d22221f (utils/get-developers: read patch from stdin when
it's not a tty)
Reported-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 3778f704cd)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since commit 9c0c7846cd (support/dependencies: don't check for python
on the host), we no longer check for a host python interpreter installed
on the system.
Drop the comment in support/dependencies/check-host-python3.sh, as it is
now confusing.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 3722998a3d)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
parser.error() reports a nice error message, that also displays a short
reminder of the available options.
Adapt the test-suite accordingly: previously, the error string was an
exact string in the stdout list, while it now is a substring in one of
the strings in stderr. The exit code changes, too.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 35f381b93e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This is a basic test for Xvisor RISC-V 64bit. It is running few
management and status commands. It does not start a Linux kernel.
RISC-V 64bit was chosen for this test because it was the simplest
solution to run xvisor in a qemu emulator.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit e14380b3c4)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In some specific situations, there is subtle bugs which depends on a
specific Qemu emulator version and the code it runs.
For example, EDK2 on Aarch64 could work with specific versions of Qemu,
EDK2 and ATF. See commit cc0823c2d "boot/edk2: bump to version
edk2-stable202405" [1].
Also, some Qemu bugs made the guest OS crash. See for example commit
9534b9c00 "package/qemu: fix qemu 9.x issue for AArch32 Secure
PL1&0" [2].
Commit 0d4177598 "support/testing/infra/emulator.py: add build host
dir to qemu search path" added the ability for a runtime test to
select host-qemu in order to use it. It is also possible for a user
to use the "utils/run-tests" script on its host system providing its
own version of Qemu. The Buildroot CI can also use its Qemu version
included in the reference Docker image.
This means the Qemu emulator for running a runtime test can be from
several sources:
- Buildroot Docker reference image,
- Buildroot host-qemu package version,
- Developer host OS qemu version.
Those versions can also change in time.
In order to help debugging of those subtle emulator bugs, this commit
adds a recording of the actual Qemu version used to run a test.
[1] cc0823c2d1
[2] 9534b9c00c
[3] 0d4177598c
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit ed9da08944)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The SBSA Reference Platform was updated to use the neoverse-n1 cpu
(armv8.2a) in Qemu v8.1 [1].
Commit c58a2a968 "configs/qemu_aarch64_sbsa_defconfig: switch to
neoverse-n1 (armv8.2a)" [2] reflected this change in
qemu_aarch64_sbsa_defconfig.
This commit updates the test_fwts runtime test the same way.
This update is possible with the help of commits [3] and [4].
[1] 1877272bad
[2] c58a2a9687
[3] 0d4177598c
[4] 027ddcbff6
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit c2f91e9046)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The SBSA Reference Platform was updated to use the neoverse-n1 cpu
(armv8.2a) in Qemu v8.1 [1].
Commit c58a2a968 "configs/qemu_aarch64_sbsa_defconfig: switch to
neoverse-n1 (armv8.2a)" [2] reflected this change in
qemu_aarch64_sbsa_defconfig.
This commit updates the test_edk2 runtime test the same way.
This update is possible with the help of commits [3] and [4].
[1] 1877272bad
[2] c58a2a9687
[3] 0d4177598c
[4] fba62f2ab1
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
(cherry picked from commit 36c7a5366a)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The weston runtime test is unreliable, depending on the execution speed
of its runner. Example of failure is [1], and success is [2]. This
commit improves the test in several ways, to make it more robust to
execution speed variations:
- The command started in background (weston, weston-simple-egl) are
now started in a subshell. This suppresses the job control messages
when they are stopped. Those messages could interfere with the parsing
of the output;
- Wait time are moved outside the emulator;
- The kernel argument vt.global_cursor_default=0 is added, to make sure
cursors are globally disabled, since the test use vkms display CRCs;
- The memory of the emulator is increased to 512M. This test uses cpio
initramfs, and the filesystem size increased;
- The vkms driver emulates a "vsync" event, but can generate a warning
when the system is too slow. This warning is printed on the console by
the klogd daemon in its default configuration. This commit adds the
overlay file /etc/default/klogd to limit only kernel emergency
messages to be printed on the console. This change fixes the failure
seen in [1] ;
- Some sleep time values were adjusted to run on a "performant" idle
host. Those values are not suitable in all situations. In the
meantime a generic retry mechanism is added in the test infra, this
commit adds a retry logic inspired from the test_flutter runtime
test.
Fixes: [1]
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/8562483474
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/8435236652
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The mosquitto runtime test can randomly fail on slow
runners, see [1].
This commit improves this test in the following ways:
- the mosquitto_sub subscriber process is now started in a subshell
to suppress the job control messages (to prevent any spurious
messages when the job stops),
- the standard error is redirected to /dev/null, to prevent the
printing of any messages,
- the mosquitto_pub publisher process is started later, by increasing
the sleep time,
- finally, a new sleep time is introduced between the mosquitto_pub
publisher process and the check of the mosquitto_sub subscriber, to
make sure it will have time to write its output and exit.
Fixes: [1]
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/8453386454
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit f439b47ed "boot/grub2: add RISC-V 64bit EFI support" [1]
introduced the grub2 RISC-V UEFI support.
At that time, it was not possible to have a runtime test for this.
qemu-system-riscv64 is not in the Buildroot Docker reference image,
and the base distribution (Debian 11) does not provide a
sufficiently recent version. A qemu-system-riscv64 >= 8.0.0 is needed.
Commit 0d4177598 "support/testing/infra/emulator.py: add build host
dir to qemu search path" [2] added the capability to use the
Buildroot host-qemu package inside runtime test.
With this support, it is now possible to add a runtime test for a
grub2 UEFI boot on RISC-V 64bit.
Also, due to an upstream edk2 git submodule issue, this commit
includes a workaround to use https://sources.buildroot.net for this
test. See [3].
[1] f439b47ed6
[2] 0d4177598c
[3] 47fc9e5509
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit enables host-qemu in the build, to be used by the runtime
test infrastructure.
Since this commit switch to a more recent host-qemu, it also removes
firmware options to workaround old qemu bugs.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit updates ATF to 2.11, and enables host-qemu in the build,
to be used by the runtime test infrastructure.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit updates ATF to 2.11 and kernel to 6.6.58, and enables
host-qemu in the build, to be used by the runtime test infrastructure.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The relevant qemu emulator is started with pexpect.spawn() with a
default environment (and PATH). This will always use the qemu available
on the host. When running in the reference Docker image, it is using an
old Qemu v5.2.0. This version is getting too old to run with more
recent components, like EDK2 or ATF.
The main reason to use a qemu version preinstalled in the Docker image
was to save some build time. Also, this behavior is opposite to the
"start-qemu.sh" helper script [1], which tries to execute the
Buildroot host-qemu first (and then fallback the the version on the
host system, if available). An option was added in [2] to let the user
have a better control on which version is used.
Also, updating the Buildroot reference Docker image has a larger
impact on the whole project (i.e. updating the image would bring
a newer qemu version, but might break many other things, at a time
which is not always convenient).
We now need some kind of control in runtime tests, to choose between a
"stock" qemu, and a more recent version with all the bells and
whistles.
This commit adds the Buildroot host build directory in the path. This
will make the runtime test infrastructure to use a host-qemu, if
selected by a runtime test configuration.
For now, most of the test can continue to use the qemu version provided
in the Docker image. Only the problematic tests relying on a recent
version will select BR2_PACKAGE_HOST_QEMU=y. The host-qemu package is
usually maintained to a recent version, so this should be sufficient.
[1] https://gitlab.com/buildroot.org/buildroot/-/blob/master/board/qemu/start-qemu.sh.in
[2] 5de78f181a
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
EDK2 version edk2-stable202405 (not yet in Buildroot) has dependencies
on specific Qemu version.
Buildroot guidelines also recommend that "each patch must be complete".
See: [1]. This means, a patch in a series is not allowed to break
something knowingly, even in the same patch series. This rule is
mainly for using git bisect.
We also want to avoid bumping EDK2, update the runtime test infra,
and few tests in a big single commit, for readability. This would mix
too many changes in the same commit.
As a trade off, this commit disables the runtime tests which are known
to be broken by the upcoming edk2 update. Then, those tests will be
individually fixed and re-enabled in followup commits.
This EDK2/Qemu incompatibility only affects runtime tests using EDK2
on the Arm Aarch64 architecture. This commit disable the runtime
tests by adding a "skipTest()" call in:
- tests.boot.test_edk2
- tests.boot.test_grub.TestGrubAArch64EFI
- tests.package.test_fwts
[1] https://buildroot.org/downloads/manual/manual.html#submitting-patches
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently the test TestPythonTwisted is not broken [1] since it doesn't
test all modules provided by python-twisted package.
But TestPythonTwisted should have failed due to missing xml in
twisted/web module.
Due to this, TestPythonPy3MagicWormhole runtime test is currently
broken.
Add missing python3-xml dependency and update
sample_python_twisted.py to load twisted/web that uses xml
module.
Fixes (partially due to missing python setuptool in
python-magic-wormhole-mailbox-server):
https://gitlab.com/buildroot.org/buildroot/-/jobs/8199992646
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Generating Bitcoins to an address can take longer than the current
timeout, on slow runners. This commit fixes this issue by increasing
the timeout on specific commands. This issue was also observed more
frequently on newer bitcoin-core version 28.0.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/7782083081
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When I initially made the CVE cells collapsible, I neglected to
count the unsure CVEs. This patch adds unsure CVEs to the cell collapsing
calcualation to ensure that cells with lots of unsure CVEs actually get collapsed.
This patch also removes the "+ 1" from the cve_total calculation,
which fixes the cve_total being off-by-one.
I'm not sure *why* I did that in the first place.
demo:
https://sen-h.codeberg.page/pkg-stats-demos/@pages/add-unsure_cves-to-cve_total-calc.html
Signed-off-by: Sen Hastings <sen@hastings.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a test that runs the dtc commandline tools. To test devicetree
compilation, we use an example devicetree from the dtc project. The
example source is GPL-2.0+ licensed.
Signed-off-by: Brandon Maier <brandon.maier@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
With the introduction of ARM FDPIC support, we need to make sure that
the existing ARM FLAT external toolchains are not made
visible/available when ARM FDPIC is selected. This commit updates the
gen-bootlin-toolchains script to take this into account.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
