Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the TestXvisor fails when building xvisor with gcc 15.1.0.
Since xvisor is now fixed, opensbi 1.5 fail to build with gcc-15 for
the same reasons.
opensbi 1.6 include a fix for C23 enabled compilers [2]
[1] 947dbc92a2
[2] d539d64a23
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This reverts commit 559bb33ae7, which
was disabling BR2_BACKUP_SITE in our tests with the following
reasoning:
support/testing: do not use s.b.o
Currently, the runtime tests will use the sources.buildroot.org backup
mirror, which is the default setup.
However, in some cases we do not want to use the backup site, because we
want to ensure that the download actually works.
This is the case for vendored packages, like cargo or golang packages,
for whbich we want to check that gthe download still works when we
update the rust or go versions, or when our download helpers change.
So, disable the use of the backup site in all runtime tests, and drop
the no-longer needed special cases.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
However, not using the backup site is causing a LOT of random failures
in Gitlab CI, making the results barely usable. Since it's more
important to get real failures than random ones due to download
issues, let's re-enable BR2_BACKUP_SITE in our testing infrastructure.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the tests.package.test_dmidecode fails when building its Kernel 6.6.39
with gcc 15.1.0.
This commit fixes the issue by updating the test Kernel to the
latest 6.6.y version (6.6.102 at the time of this commit) which
includes the fix for gcc-15.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10984686001
[1] 947dbc92a2
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the tests.package.test_msr_tools fails when building its Kernel 5.15.55
with gcc 15.1.0.
This commit fixes the issue by updating the test Kernel to the latest
LTS version (6.12.42 at the time of this commit) which includes the
fix for gcc-15. Note: the 5.15.y series does not include this fix for
the x86_64 architecture, which is why this commit switches to 6.12.y.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10984686294
[1] 947dbc92a2
Cc: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the tests.package.test_dpdk fails to build its Kernel 6.6.58 with
gcc 15.1.0.
This commit fixes the issue by updating the test Kernel version to
the latest 6.6.y version (6.6.102 at the time of this commit) which
includes the fix for gcc-15.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10984686010
[1] 947dbc92a2
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the tests.package.test_numactl fails to build its Kernel 6.1.75 with
gcc 15.1.0.
This commit fixes the issue by updating the test Kernel version to
the latest 6.1.y version (6.1.148 at the time of this commit) which
includes the fix for gcc-15.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10984686336
[1] 947dbc92a2
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since Buildroot commit [1] "update to Bootlin toolchains 2025.08-1",
the tests.package.test_wine fails to build its Kernel 6.6.27 with
gcc 15.1.0.
This commit fixes the issue by updating the test Kernel version to
the latest 6.6.y version (6.6.102 at the time of this commit) which
includes the fix for gcc-15.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10984686766
[1] 947dbc92a2
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Fiona Klute (WIWA) <fiona.klute@gmx.de>
[Julien:
- reword commit log title
- replace python '%' formatting with '+' concatenation
- remove IP address assertion in test controller as it
is already in sample script
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
The toolchains have been rebuilt with a recent Buildroot, and are
based on:
- Stable toolchains:
GCC 14.3
Binutils 2.43.1
GDB 15.2
Linux headers 5.4
glibc 2.41
musl 1.2.5
uclibc-ng 1.0.45
- Bleeding edge toolchains
GCC 15.1
Binutils 2.44
GDB 16.3
Linux headers 5.15
glibc 2.41
musl 1.2.5
uclibc-ng 1.0.45
The microblaze bleeding-edge toolchains are back, since we integrated
the "atomic issue" fix.
All tests are successful:
https://gitlab.com/tpetazzoni/buildroot/-/pipelines/1976388296
(note: the test branch had one more commit doing tweaks on the wget
option and re-enabling the backup site for unit tests, which are
needed to avoid intermittent failures of Gitlab CI jobs)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
On my laptop (i5-8365U), running cross-tesseract-ocr in QEMU takes
longer than the default timeout (5s), causing the test to fail:
# time tesseract frame10.png output
Error in pixReadMemTiff: function not present
Error in pixReadMem: tiff: no pix returned
Error in pixaGenerateFontFromString: pix not made
Error in bmfCreate: font pixa not made
Estimating resolution as 157
real 0m 8.17s
user 0m 5.36s
sys 0m 2.79s
So extend the assertRunOk() timeout with ~2x that similar to how it is done
for the gst-launch-1.0 pipeline.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Commit [1] introduced an Armv7 test case for Xen named TestXenArmv7,
next to the original Aarch64 test case which was simply named TestXen.
The test list shows, for example with the command
"support/testing/run-tests -l":
tests.package.test_xen.TestXen
tests.package.test_xen.TestXenArmv7
In order to make this test list a bit more explicit, this commit
renames the TestXen to TestXenAarch64. With that change, the list
becomes:
tests.package.test_xen.TestXenAarch64
tests.package.test_xen.TestXenArmv7
[1] 5346824a83
Cc: Vincent Stehlé <vincent.stehle@arm.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Add a TestXenArmv7 class with its related files in an arm/ subfolder
under test_xen/, to test Xen on 32-bit Arm v7.
We cannot boot with UEFI in this case; we use a custom U-Boot script,
which creates the Xen configuration Devicetree during boot.
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
In preparation of adding a test for Xen on 32-bit Arm v7:
- Introduce an architecture-agnostic TestXenBase class where we move
most of the Xen test scenario and bits of the configuration.
- Re-organise the test_xen/ folder with the architecture-agnostic files
under common/ and the 64-bit Arm specific files under aarch64/.
Make the 64-bit Arm TestXen class inherit from the base class and leave
in there only the architecture-specific parts:
- The 64-bit Arm configuration bits.
- The test function, which passes the proper 64-bit Arm simulator
options to the generic test function.
No functional change intended.
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Via the docker-compose runtime test, we can exercise the full suite of
the docker-related packages: docker-compose, of course, but also
docker-engine, which in turn allows exercising containerd. The latter
by defualt uses runc as the container runtime, but can alternatively use
crun.
Extend the docker-compose runtime test with a variant that enables crun.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The syslinux runtime test (which is in fact a build-only test) ensures
that syslinux does get build at least once a week (via the gitlab-CI
weekly pipeline). Runtime testing would need much more work, though, but
nothing in syslinux is currently runtime tested anyway.
Reported-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This package provides a reference, functional implementation of a
registry for distributing container images. Formerly the Docker
registry, it now also handles OCI images, so can be used to distribute
all kinds of container related images: classic Docker images, OCI
images, but also helms [0] and the likes, which are also packages as OCI
images in fact.
The upstream name of the package is just "distribution" [1], but this is
a bit too generic and unsearchable; distributions like Debian and Ubuntu
traditionally provides it as the "docker-registry" package, but this is
now misleading as it is not just for Docker anymore. So we decided to
name it "distribution-registry": it contains the upstream name and will
be searchable with the 'registry' part as well.
The built executable is named "registry", which is also a bit too
generic. Again, for the same reasons as above, we name it
"distribution-registry"; this also incidentally makes our golang-package
infra "just work" and do the right thing when it installs the package.
The upstream package provides a wrapper Makefile to build the package,
but it does not do more than our infra already does; the only little nit
is that it allows updating the version file (with git infra et al.). We
can just ignore that and update the version ourselves.
We also provide a minimalist configuration file: it exposes the registry
to localhost only, as a security measure. This file will have to be
overridden (e.g. in a rootfs-overlay) with proper authentication, and
with further locally meaningful setup anyway.
Finally, we also implement a run-time test that validates that we can
push and pull images to/from the registry. Since the registry is a big
go package, it takes time to start, so wait for it a little bit (and yet
a little bit more after it starts logging), and increase the timeouts
for commands that deal with the registry.
For this simple test, the registry is directly exposed with plain HTTP,
not HTTPS, so we must declare it as insecure, so that container tools
can reach it without custom options (e.g. without using skopeo's
--{src,dest}-tls-verify=false options).
We re-use skopeo to talk to the registry. This allows us to grab an
image from the Docker hub registry, and push it to our local registry,
then retrieve it back. Since we want to diffferentiate failures to grab
the image from the Docker Hub [2], from failures to push to our
registry, we do it in two steps: copy from Docker Hub to a local OCI
image, then push that to our registry.
Since both distribution-registry and skopeo are big go-based, statically
linked binaries, running both in the standard virtual machine does not
work: the 256MiB emulated in vexpress are not enough and cause segfaults
when running both at once. The vexpress also only accepts at most
256MiB, so we must use another machine; we decided to use the same as is
used by the podman teest case, which allows at least 1Gib, which is now
enough.
[0] https://helm.sh/
[1] https://github.com/distribution/distribution
[2] in case we reach the 100-pulls-a-day limitation
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Christian Stewart <christian@aperture.us>
Signed-off-by: Julien Olivain <ju.o@free.fr>
... and extend the skopeo runtime test to make use of those.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The new test requires a br2-external directory because we compile a
small test program on the host and install it on the target, but it's
not useful to have it in the main Buildroot package tree.
The test program loads and parses a sample HTML document. Taking
inspiration from 'examples/get_title.c' in gumbo-parser, it also
searches for the title of the document just to check that we can do
more than the parsing.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This test verifies that we can run nginx with the modsecurity
directives.
It also checks a very simple rule that blocks requests containing the
keyword "blockme".
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
[Julien:
- add / at directory end in DEVELOPERS
- sort DEVELOPERS entries alphabetically
- remove unneeded test configs already present in
BASIC_TOOLCHAIN_CONFIG
- sort test config directives alphabetically
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
For release notes since edk2-stable202411, see:
https://github.com/tianocore/edk2/releases/tag/edk2-stable202502https://github.com/tianocore/edk2/releases/tag/edk2-stable202505
Note: the edk2-stable202505 release note mention a security fix.
This fix was already backported in Buildroot in a previous commit.
For that reason, this bump is not marked as securiy, and also
removes the backported patch and its associated _IGNORE_CVES entry.
Those versions include fixes when using gcc 15 (host and target).
This commit also updates the edk2-platforms and edk2-non-osi packages
with the last commit merged at the edk2 release date (2025-05-23).
In the case of edk2-platforms, the commit id used in this update
is few days later, to include a fix for the Arm Sgi575 platform.
Exact commit should have been [1] (to be exactly aligned to the edk2
tag date), but commit [2] is used instead. The package edk2-non-osi is
updated to commit [3].
Also, the SocioNext SynQuacer platforms (including the DeveloperBox)
were removed upstream in edk2-platforms commit [4]. This commit
reflect that change by removing those platforms choices and associated
runtime tests.
This commit has been runtime tested with tests using EDK2 package,
with commands:
support/testing/run-tests \
-d dl -o output_folder \
tests.boot.test_edk2 \
tests.boot.test_grub.TestGrubAArch64EFI \
tests.boot.test_grub.TestGrubi386EFI \
tests.boot.test_grub.TestGrubRiscV64EFI \
tests.boot.test_grub.TestGrubX8664EFI \
tests.fs.test_iso9660.TestIso9660Grub2EFI \
tests.fs.test_iso9660.TestIso9660Grub2Hybrid \
tests.package.test_fwts
It has also been runtime tested (by booting in qemu) with defconfigs
using EDK2 package:
qemu_aarch64_sbsa_defconfig
qemu_riscv64_virt_efi_defconfig
[1] 2bfe3fa99e
[2] 92f7d69e8a
[3] ea2040c2d4
[4] aaba5807f1
Cc: Dick Olsson <hi@senzilla.io>
Cc: Romain Naour <romain.naour@smile.fr>
Cc: Vincent Stehlé <vincent.stehle@arm.com>
Tested-by: Vincent Stehlé <vincent.stehle@arm.com>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Back port four upstream fixes for those issues:
- gpsprof could not log or dump GNSS messages; gpsprof is useful
to calbrate a GNSS receiver, especially when building a static
RTK base;
- the JSON blurb reported by gpsd was sometimes broken, causing
decoding errors in clients;
- the detection of 64-bit time on 32-bit systems was borked;
- UDP mode was borked;
The runtime test is extended to test all known transports: TCP, UDP,
and PTY.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This test verifies that clinfo correctly detects the rustiCL platform
and llvmpipe as device.
We check:
- exit code,
- Platform name matches rusticl,
- Device name matches llvmpipe.
Signed-off-by: El Mehdi YOUNES <elmehdi.younes@smile.fr>
[Julien:
- rename file to test_mesa3d.py
- add Medhi in DEVELOPERS for this test file
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
For change log, see:
https://gitlab.com/gpsd/gpsd/-/blob/release-3.26.1/NEWS
Support for RTCM104v2 and v3, as well as for ublox, are now
unconditionally built, so drop our corresponding options. No
needfor legacy options: users that had those enabled will
still get them.
Support for Oceanserver has been dropped upstream, so drop our
option and add legacy handling to infrom users they should
complain upstream if they needed it (very unlikely).
Also, a bug introduced in gpsd 3.26 makes the Buildroot gpsd runtime
test failing. See [1]. As a workaround, this commit also changes
this runtime test to emulate a gps over TCP, instead of UDP.
[1] https://gitlab.com/gpsd/gpsd/-/issues/337
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
[Julien:
- rebased on branch next
- add gpsd version removing OceanServer option in Config.in.legacy
- add link to NEWS in commit log
- switch test_gpsd to use tcp instead of udp
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
The aarch64 virt platform doesn't have any default VGA devices so we
don't need to configure them here.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Julien Olivain <ju.o@free.fr>
pydantic pins an exact version of pydantic-core in it's pyproject.toml
[1]. This was missed during the last pydantic bump where pydantic-core
was erroneously updated to the latest release instead, in commit [2]
and [3].
Modify the pydantic runtime test to catch this error in the future.
[1] https://github.com/pydantic/pydantic/blob/v2.11.4/pyproject.toml#L49
[2] e1c939e426
[3] e42de820cd
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
[Julien: add reference to commits introducing the issue]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Since Buildroot commit [1], the runtime test emulator infra is setting
the emulated system date to the host date.
While this is desired in general, this behaviour is introducing a
variability in the test execution. Depending if the test is executed
during winter or summer time, the output of the command "date +%Z"
will produce a different output.
This commit fixes the issue by setting a fixed date and time on the
emulated system. The date is fixed to Unix Epoch plus one hour. This
is because Linux cannot set the system date to a value less than the
system uptime. So we cannot set the time back to Unix Epoch with the
command "date -s @0" (this would result to a EINVAL Invalid argument).
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/9922589073https://gitlab.com/buildroot.org/buildroot/-/jobs/9922589081
[1] cf8641b73e
Signed-off-by: Julien Olivain <ju.o@free.fr>
Add a kernel fragment to enable
CONFIG_ARM64_VA_BITS_48 for the AARCH64 QEMU platform in the
clang compiler -rt runtime test.
Without this setting, this sanitize-based test fails with clang-20
due to insufficient virtual adress space.
"This might potentially affect ARM platforms with 39-bit address space." from [1]
[1] a588cfe37e
Signed-off-by: El Mehdi YOUNES <elmehdi.younes@smile.fr>
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The test_xen.py runtime test, introduced in [1] and improved in [2],
is calling a "stty raw" command, just after the emulated machine
login, to avoid double-cooking the consoles. This double-cooking
prevents the test controller to correctly get the command error codes.
Buildroot commit [3] "support/testing: set date in emulated machine"
introduced an invocation of the date command to set time on the
emulated machine, just after the login. The returned error code is also
checked. Since this commit [3], the test_xen runtime test is failing
while attempting to set the date. This is because it is invoked before
the test script executes this "stty raw" command.
The need of executing a command just after the login, and just
before we set the emulated machine date is very limited. It is almost
specific to this test. So, rather than changing the test
infrastructure, this commit simply moves this "stty raw" invocation
from the runtime test script to a custom /etc/profile.d/stty-raw.sh
file on target rootfs overlay, to do this call just at the login.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/10000011350
[1] 055f82ebbd
[2] cd0ffd598c
[3] cf8641b73e
Cc: Vincent Stehlé <vincent.stehle@laposte.net>
Tested-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Note: this test was not working in Buildroot test infrastructure
before commit [1] was merged, because dieharder has the string "# "
in its output.
[1] 0cad947b96
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
While in theory, the fastapi tests finds problems with the pydantic
package, it's not obvious that this test should be run when the pydantic
package is updated.
Add a new test that just covers pydantic.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Add a runtime test for the 'dust' package to verify that the binary
executes correctly in a minimal buildroot rootfs. The test checks that:
- 'dust --version' runs without error
- 'dust' can analyze a directory structure with files
- The output includes the expected directory names
Signed-off-by: El Mehdi YOUNES <elmehdi.younes@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Add a runtime test for the 'bat' package to verify that the binary executes
correctly in a minimal Buildroot rootfs.The test cheks that:
- 'bat --version' runs without error
- 'bat' can read and display a text file
- the displayed content matches the expected string
Signed-off-by: El Mehdi YOUNES <elmehdi.younes@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The test timeout is no longer sufficient, so double it.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Importing urllib3 already allows us to check that for example zlib can
be loaded at runtime.
For good measure, also create the PoolManager object mentioned in the
user guide ([1]), and check that we can normalize a URL like it is
done in urllib3's 'test/test_util.py'.
[1]: https://urllib3.readthedocs.io/en/stable/user-guide.html
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Julien Olivain <ju.o@free.fr>
