Commit [1] "support/testing: improve weston test reliability" moved
out the wait time from the emulator (to run on the test controller).
While doing so, the sleep time which was initially _after_ the
"killall weston" invocation to in stop_weston() was incorrectly
moved before the command invocation. In this state, the test can
succeed on fast host computer running the test. But it will most
likely fail on an average computer.
This commit fixes this issue by moving the sleep time after
the command invocation.
[1] 6561a5d773
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some EDK II configurations have complex dependencies on several packages
and additional build options; build tests help keeping track of those
more easily.
Factorize some code common to all the build tests into a new
TestEdk2BuildBase class, which defines a base configuration and a method
to assert that binaries do indeed exist after the build.
While at it, add myself in DEVELOPERS.
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Dick Olsson <hi@senzilla.io>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Switch the test assert to use 'Europe/Berlin' instead of
'posix/Europe/Berlin'. The tz-name is an internal implementation detail
anyway and the lookup changed to prefer the stated name in /etc/timezone
again, if it matches the configured timezone via /usr/share/zoneinfo.
Ref: 4ffac9232b
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Since Buildroot commit [1], the test_gstreamer1 runtime test is
failing to build the tesseract-ocr package. The root cause is that
the test uses the default arm external toolchain, which is the Linaro
ARM 2018.05 based on gcc 7.3.1.
Since [1], tesseract-orc no longer compiles with gcc 7 (it requires at
least gcc 8).
This commit fixes the issue by switching the toolchain to the
bootlin versions (based on gcc 14 at the time of this commit).
Also, changing the compiler version slightly slowed down the video
encoding. This commit also increases the encoding command timeout.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/9407846232
[1] f32da8b984
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The busybox date applet accepts the following:
date @1234567
but this confuses the coreutils version which doesn't implicitly set
time. As some tests might need coreutils binaries we should ensure the
emulator login will work with both. Fix this by passing the -s (set)
option to the command.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Julien Olivain <ju.o@free.fr>
In commit ba70f94e62 (subversion makes it so we do not need a separate
tarball to store empty directories and symlinks for the target_skeleton.)
/etc/resolv.conf was made a symlink to ../tmp/resolv.conf. There was no
rationale behind this change, but back at the time (2006!), the only
location that Buildroot guaranteed to be writable was /tmp, and we
needed a writable location to create resolv.conf from a DHCP reply for
example, so the change made sense.
Commits 2f67d9e2fe (preparation for BR2_PACKAGE_BUSYBOX_SKELETON), in
2007, then d9119c0bc5 (move default skeleton to fs/ and drop busybox
skeleton), in 2010, and finally 6c3e3ad419 (New top-level directory:
system), in 2012, propagated that /etc/resolv.conf without any change in
the target of the symlink.
However, since commit d380dde1d2 (skeleton: make /run a proper
directory/filesystem), in 2015, a tmpfs has been mounted on /run, so it
is now guaranteed to be writable as well for SYS-V-based systems. That
has also been the case for openrc since commit a840e485d1
(package/skeleton-init-openrc: new package), in 2019. This has also
always been the case for systemd as well.
resolv.conf is not a temporary file [0]; it is really more of a runtime
data; its location in /tmp is just a historical artefact of a long-wrong
assertion.
Nowadays, distributions either have an actual file as /etc/resolv.conf
(when /etc is writable), or it is a symlink to a location below /run.
Although resolv.conf is used by the C library resolver, a lot of
applications now will directly use it; this is the case for some
container management tools, such as podman (and its helpers, like
netavark), that expect /etc/resolv.conf to either be a file or a symlink
to somewhere in /run, and fail to provide DNS services if that is not
the case.
So, we move the target of /etc/resolv.conf to ../run/resolv.conf.
As a consequence, we can drop the workaround in podman's runtime test.
To be noted: there are tow custom fstab files in Buildroot that do not
explicitly mount a tmpfs on /run:
- one for a systemd runtime test, so /run would also be a tmpfs there
anyway, as systemd does mount one;
- one for the mender_x86_64_efi_defconfig; that's a not a problem:
/ is an ext4 that is mounted r/w (and /tmp is not a tmpfs either, so
the symlink to /tmp would not not have worked if / was not r/w
anyway).
This leaves people with their own fstab, to adapt. Cater to them by
adding a little blurb in the documentation.
Note: the following 43 tests all built and ran successfully:
tests.init
tests.package.test_apache
tests.package.test_docker_compose
tests.package.test_dropbear
tests.package.test_iputils
tests.package.test_podman
tests.package.test_skopeo
tests.package.test_wget
(no other runtime test were attempted)
[0] temporary, as in relatively short-lived.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
passt/pasta is the officially supported network backend, with slirp4netns
only considered a legacy solution. However, some people have experienced
corner cases when using passt/pasta (as it still is a young stack), so we
do not want to do a blanket replace of slirp4netns just yet. Still, we
make passt/pasta the default option.
As a consequence, the existing runtime tests will now test passt/pasta,
so we introduce two new tests for slirp4netns as a network backend.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Podman is a container manager not unlike Docker, but is daemon-less.
Similarly to docker-engine, quite a few kernel config options are
required; as they are very similar in goals and features, the options
from docker-engine have been duplicated for podman. As that was not
enough, a few additional options have been added after trial-and-error
testing (they are not explicitly listed in the documentation).
The documentation [0] states that seccomp can be disabled (i.e. not
enabled). However, without seccomp support, starting containers requires
--security-opt=seccomp=unconfined to be specified; it does not look
trivial to make that the default, though [1]. Furthermore, containers
are about security, so disabling a security measure does not sound too
good. So we make seccomp support mandatory.
Podman needs netavark as a the network backend (it missing is a hard
error at runtime). It is supposed to also require aardvark-dns, an
authoritative DNS resolver, but it missing does not look to adversely
affect networking, so it is not used (as not even packaged in Buildroot
yet).
Podman can run either as the root user, or it can run rootless, i.e. by
a non-root user, which requires a bit of setup (see below, in the
runtime test) and a few other dependencies: slirp4netns [2] (to provide
network connectivity in userland), and support for subordinate UIDs and
GIDs with the shadow library. Rootless mode is one of the main selling
point for podman, so we decided that this would not be configurable in
Buildroot.
Similar to Docker, podman can inject a minimalist init as PID1 in
containers, and like Docker, this is optional; podman however by default
uses catatonit as such an init [3]. As for Docker-engine, we offer a
choice of which init package to use to provide podman-init.
Podman requires at least three config files; they can be either per-user
or system-wide:
- containers.conf [4]: defines various settings for the container
runtimes;
- policy.json [5]: defines what signature to accept to validate
images; without one such file, podman just refuses to pull images;
- registries.conf [6]: defines where to pull images from; without it,
podman does not know how to pull un-qualified images (i.e. images
where the registry is not specified in the path, and which Docker
would fetch from the Docker Hub, e.g. "busybox:latest").
For those three files, we provide a very minimal default that (in the
same order as above):
- uses the slirp4netns network backend for rootless operation (the
default in podman is to use pasta [1], so we need to explicitly
configure it to use slirp4netns);
- allows pulling images which signature can't be verified;
- pulls unqualified images from the Docker Hub, as is traditional.
Providing actual files is going to be use-case dependent, and interested
parties will have to provide their own config files, e.g. in a rootfs
overlay.
Finally, we add a runtime test for podman. Podman is a huge binary, and
may call other huge binaries (netavark...); this can be quite slow in
the emulated machine (even when running on a very fast host machine), so
we use a huge timeout for all commands involving podman, even those that
exit the containers, as that may need to tear down podman setup.
The default kernel used in runtime tests is missing a lot of features,
so we need to build our own; we use the same version as the bundled
kernel. We can't use cpio either, because we need a filesystem that can
be used as a lower and upper of overlayfs, which is not possible with
the filesystem the cpio is extracted into; ext2 fits the bill, so we use
that. We need a bit of space to store images and stuff, so let's be
generous and allocate 256M.
To test rootless operation, we need a non-root user that has some
special setup [7]; it is easier to run the commands from the infra
rather than carry a user-definition table and a rootfs overlay. We need
that user to have the same prompts (main and continuation) so that the
REPLWrapper still detects those; it has the unfortunate side effect that
it is not immediately obvious whether a command was run as root or not,
and one has to look back up in the run-log to see whether there was a
transition to another user earlier.
Still for rootless containers, podman/netavark expect /etc/resolv.conf
to be either a plain file, or a symlink that points either deeper in
/etc or anywhere in /run; if resolv.conf resolves to any other location,
DNS in rootless containers does not work. This is reasonable, and is
what already happens on a systemd-based system (and thus all major
distributions nowadays. However, in Buildroot, we put the actual file in
/tmp; this is historical, and dates back to the days where Buildroot did
not have a guaranteed-writable /run. So, we work around this limitation
in the test (for now).
The official busybox image on the Docker Hub supports a lot of
architectures, of which armv7 which we use for this runtime test.
Finding a small image that also supports armv7 on other registries was
a bit of a challenge; we eventually found one busybox image on quay.io,
but it is not an official busybox image; still, it fits the bill, so we
use it.
There is no runtime test with systemd, as this requires quite some
additional setup that does not look very trivial to do; when it detects
it is running under systemd in rootless mode, podman expects that a full
user session exists, or it whines about it every time it is started,
reverting to non-systemd behaviour; getting a full user session does not
look to be that trivial (PAM?), so this would not exercise the actual
integration with systemd, so the test would not be meaningful, so it is
not provided. This is left as an exercise to an interested party to
extend the tests.
PS: Hat-tip to Raphael, who provided some pointers and hints on this
change, especially for rootless mode. Thanks! 👍
[0] https://podman.io/docs/installation#get-source-code
[1] it looks like we can provide a custom seccomp profile, by specifying
seccomp_profile="PATH" in containers.conf; that would still require
seccomp support to use that file, though, so that does not change the
outcome.
[2] it is possible to use another backend, but it is not packaged in
in Buildroot yet: https://passt.top/passt/about/#pasta-pack-a-subtle-tap-abstraction
[3] podman expects a 'catatonit' helper in /usr/libexec/podman, so even
if tini would be usable instead, it would not feel right to use it to
impersonate catatonit. So let's assume that only catatonit is supported.
[4] https://github.com/containers/common/blob/main/docs/containers.conf.5.md
[5] https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md
[6] https://github.com/containers/image/blob/main/docs/containers-registries.conf.5.md
[7] https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Raphael Pavlidis <raphael.pavlidis@gmail.com>
Cc: Christian Stewart <christian@aperture.us>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Skopeo can be used to manipulate container images, and so can be useful
on target devices that are going to be used as a host to run containers.
So, add a target variant for skopeo.
... with a runtime test while at it.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Christian Stewart <christian@aperture.us>
Cc: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When time comes to check certificates, the date and time in the emulated
machine should be close enough to the actual values, so that certificate
validity can be checked.
Some Qemu machines have an RTC (e.g. arm vexpress-a9 has a pl031), and
the kernel needs a driver for those RTC. It is not guaranteed that the
machine used for a test meets those two conditions; in such a case, the
time in the machine starts way back in the past (1970-01-01T00:00:00Z on
sysv, or the release date of systemd). This is the case with the default
kernel, so such tests do not have the proper time.
Set the date to the date of the host system. This is going to be accurate
to the second, which is, by far, enough for our purpose.
To avoid having to consider what combination of emulated machine and kernel
configuration are being used, we always set the date, as this is a
generic step that should be done by the infra (like login in as root is).
The Emulator() class doesn't inherit from unittest.TestCase, so we can't
call any of the usual self.assertXXX() methods; instead, we just raise
a standard exception, like is done a few lines above to detect the login
prompt.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@datacom.com.br>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This removes warnings in editors/IDEs with MyPy typechecking
integration. Test classes override "config" with strings (different
type than None).
Signed-off-by: Fiona Klute (WIWA) <fiona.klute@gmx.de>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Check a save/start/stop cycle based on the rules created by direct
commands in the pre-existing test.
Signed-off-by: Fiona Klute (WIWA) <fiona.klute@gmx.de>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The new test checks that a pre-defined rules file can be loaded and
works as expected, and that after flushing the blocked IP responds to
ping again.
Signed-off-by: Fiona Klute (WIWA) <fiona.klute@gmx.de>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Previously, these tests had the potential for timing out with the
default 5 second timeout value if initializing /dev/urandom took too
long.
Now the tests use a 10 second timeout value.
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Prior to b7d251293a, txaio would test both asyncio and twisted.
Add back the twisted sample and include the twisted package in the
config so both modes of the package are tested.
Fixes: b7d251293a ("package/python-txaio: drop python 2 support")
Signed-off-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
flake8 5.0.4 provided by Debian 12 (bookworm) now detect a missing
whitespace after 'assert' keyword:
E275 missing whitespace after keyword
Signed-off-by: Romain Naour <romain.naour@smile.fr>
[Peter: fix subject, drop extra paranthesises]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since commit [1], test_atf is failing. See [2].
Passing -no-pie to ld does not produce the bl31.elf file (and does not
generate an error). TF-A 2.11 reworked its makefiles and switched to
gcc to link files. This commit fixes this failing test by updating to
this newer TF-A version.
Fixes: [2].
[1] 248c2d045e
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/8609307891
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Like Yann E. Morin did for
support/testing/tests/package/test_python_django.py, adjust the logic
to avoid the weird "self.assertTrue(False, ...)" construct, and
instead test exit_code after the loop.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The test runs the flask sample app through the waitress wsgi server
instead of the flask development server.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Since the bump of NetworkManager to 1.50.0 in Buildroot commit
489d1f9249, the
tests.package.test_python_sdbus_networkmanager.TestPythonPy3SdbusNetworkmanager
is failing to build as NM >= 1.50.0 needs headers >= 4.20.
We fix this test by using a newer toolchain. This means we can't use
anymore the basic toolchain config, so we use a fully specific
defconfig for this test.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/9019173666
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Updating directly to 4.x is too difficult and complex, as mender 4.x is
a complete rewrite in C++, with several new dependencies and changes.
As such, update to the last version that was written in Go, which is
3.5.3. The following changes are necessary:
The artifact_info file is no longer supported. Instead, mender now expects a
bootstrap.mender artifact created by mender-artifact /var/lib/mender.
See the following for more information:
https://northerntech.atlassian.net/browse/MEN-2585https://northerntech.atlassian.net/browse/MEN-2583https://docs.mender.io/release-information/release-notes-changelog/mender-client#mender-3-5-0-1https://github.com/mendersoftware/mender/blob/3.5.3/Documentation/automatic-bootstrap-artifact.md
- Remove all instances of creating or copying an artifact_info file
from board examples and package/mender.
- Add a generate_mender_bootstrap_artifact method to
board/mender/x86_64/post-image-efi.sh which creates a proper
bootstrap.mender file and places it in the data partition.
- Add a post-build.sh script to support/testing/tests/package/test_mender
with the same generate_mender_bootstrap_artifact method with one change:
As the rootfs.ext4 file is not yet created; we omit the optional
--provides "rootfs-image.checksum:${img_checksum}" argument when
generating the bootstrap.mender file.
Mender expects the device_type file to exist in /var/lib/mender/device_type.
This is further supported by the following line in tests/Dockerfile.daemon:
`echo device_type=docker-client > /var/lib/mender/device_type`
Add a migration section in docs/manual/migrating.adoc
Update the package/mender/readme.txt to include a quick mention of the
bootstrap.mender artifact change.
License changes:
New:
vendor/github.com/klauspost/compress/internal/snapref/LICENSE (BSD-3-Clause)
vendor/github.com/klauspost/compress/zstd/internal/xxhash/LICENSE.txt (MIT)
Removed:
vendor/github.com/klauspost/compress/internal/snapref/LICENSE
vendor/github.com/klauspost/compress/zstd/internal/xxhash/LICENSE.txt
Modified:
LICENSE: Update year from 2022 to 2024
LIC_FILES_CHKSUM.sha256: Sum of all Licenses changed
vendor/github.com/mendersoftware/mender-artifact/LICENSE
- Update year from 2022 to 2023
Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
[Arnout:
- fix LIC_FILES_CHKSUM.sha256 hash
- Better URL for bootstrap artifact doc
- Migrate to 2025.02 instead of 2024.11
- Improve migrating text (editorial changes)
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
- Move the check for /var/run/mender.pid to before self.run_mender_test()
for the two tests using sysvinit.
- Instead of checking for /var/run/mender.pid during the systemd test,
check to ensure "systemctl is-active mender-client" returns "active"
Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
[Arnout: fix flake8 issue]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Revision 0 is very old and misses a number of features. From man mkfs.ext2:
Set the file system revision for the new file system. Note that 1.2
kernels only support revision 0 file systems. The default is to create
revision 1 file systems.
We are unlikely to have a lot of Linux 1.2.x users, so drop support for it.
As revision 1 is the default, drop the -E revision argument to mkfs.ext2 for
simplicity.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Current test runs a Xen hypervisor and a Linux dom0, from which we can
detect that we run under Xen but not much more. Indeed, with current
test's setup the Xen tools are not functional.
We add back networking to our stripped-down kernel config to repair Xen
tools.
With Xen tools functionals, we can enhance the test with the creation of
a VM. To be able to use the rootfs that we build in both the dom0 and in
this new dom1, we switch to an initramfs; no more rootfs partition in
the image. After stripping the ramdisk a bit we can even reduce Qemu's
memory to 1 GB.
Suggested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Previously, ZFS tests were limited to x86-64 due to lack of cross-platform support.
However, upstream efforts have improved cross-platform compatibility.
Update the ZFS test platform from x86-64 to aarch64, aligning it with other tests.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The previous version, Linux 5.15.35, is outdated.
Upgrade to the latest LTS release.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Since commit d10d22221f (utils/get-developers: read patch from stdin
when it's not a tty), get-developers accepts to read a patch fromn its
stdin when it is not a tty.
Add a test for this.
Reported-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
get-developers will check its stdin to decide whether it is a tty or
not, and behave differently whether it is or not. So, when we run the
tests, we need an actual tty.
However, when running in a CI pipeline, like on Gitlab-CI, there is no
tty available on stdin.
Fake one. We don't need anything too fancy, so just a slave pty will
suffice.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/8830671800
Fixes: d10d22221f (utils/get-developers: read patch from stdin when
it's not a tty)
Reported-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
We are going to remove nios2 support, so remove the Bootlin nios2
external toolchain.
Remove this toolchain from the Buildroot testsuite.
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
parser.error() reports a nice error message, that also displays a short
reminder of the available options.
Adapt the test-suite accordingly: previously, the error string was an
exact string in the stdout list, while it now is a substring in one of
the strings in stderr. The exit code changes, too.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
