Enhance the Xen python tests to exercise block devices: this boils down
to switching from ramdisks to disk partitions for the domains rootfs.
(Refer to the comments in the python script for block devices details.)
- Add support for PCI and Xen block to the Linux kernel configurations.
- Add a few commands to list the xvda block device for good measure.
- Generate two partitions with the rootfs in the disk images; we use the
same rootfs contents twice, once for each domain.
- Add a paravirtualized block device to the Xen dom1 configurations and
adjust both domains kernel command lines, to specify the rootfs
locations.
- Build host-qemu for Arm v7, to workaround an issue with 32b Arm and
old Qemu versions, which is what we have on CI currently.
- While at it, bump Linux kernel to 6.18.4 and U-Boot to 2026.01.
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 09baeb4653)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
The run log of this ltp-testsuite test shows:
INFO: runltp script is deprecated, try kirk
https://github.com/linux-test-project/kirk
This commit updates this test to replace this deprecated runltp
shell script with the newer kirk Python script.
The logic of this runtime test remains the same: it runs a small number
of 'read' system call tests, and checks there is no failures and at
least one test succeed.
Cc: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Acked-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 43e254a646)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
The ddresue runtime test is using the `dmsetup` command provided by the
dmraid package. This package is outdated and will be removed. This
command is also provided by the lvm2 package, which is still maintained.
This commit replaces the dmraid package by lvm2 in the test config.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 4fc6e8637b)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
The tests check if all supported hash algorithms are usable in
mkimage, for both host and target packages. Additionally, as a
necessary tool, it verifies the previous fix for FIT output from
dumpimage.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
[Julien: use builtin kernel for faster testing]
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit 2dbe71dba6)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Some host commands need to call other host commands: For example,
"mkimage" from host-uboot-tools needs to run "dtc". This would fail or
call system commands without adding the host bin dir to PATH.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
[Julien: use python functions/constants to build path]
Signed-off-by: Julien Olivain <ju.o@free.fr>
(cherry picked from commit befb6ae81d)
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
The NVD database has CVE entries that are not present but may be
referenced in other security trackers.
For instance the CVE-2024-12455 is documented in the Debian security
tracker [1]. However, the NVD page is empty [2] and this entry is not
present in the NVD database mirror.
The following command would make the script fail:
```
echo '{
"vulnerabilities": [
{
"id": "CVE-2024-12455"
}
]
}' | support/scripts/cve-check --enrich-only
```
No CVEs present in Buildroot ignored CVEs are affected. But when
enriching an SBOM with legitimate CVE not present on NVD, the script
will fail.
This patch change the behavior to just log to stderr unknown CVEs
instead of making the script fail.
[1] https://security-tracker.debian.org/tracker/CVE-2024-12455
[2] https://nvd.nist.gov/vuln/detail/CVE-2024-12455
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
[Peter: Tweak warning message]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The 5.4.x series is now EOL upstream, so drop the linux-headers
option and add legacy handling for it.
Bump kernel version in test_zfs.py.
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
As requested by the gnu.org admins:
"
Since August 2024, we've been under DDoS attacks from common command line
tools.
To fix this, we would need to change the user-agent from "Python/3.11
aiohttp/3.8.4" to "buildroot.org pkg-stats" instead.
"
It indeed probably makes sense to use an unique user-agent string, so rework
the script to do that.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The gnu.org admins have been blocking the IP address of machines running
pkg-stats as the GET requests for the (many) packages with gnu.org URLs are
seen as abusive.
The resource body is not used, so use a HTTP HEAD request instead of a GET
to limit server load and bandwidth use.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When using a custom skeleton where the merged symlinks are missing,
the build fails with errors like:
support/scripts/check-merged -t skeleton -u -b /usr/src/simplek8s/rootfs-skeleton
The skeleton in -t is not properly setup:
- /usr/bin should exist, be a directory, and not be a symlink
- /usr/lib should exist, be a directory, and not be a symlink
The skeleton in skeleton is not properly setup:
- /usr/bin should exist, be a directory, and not be a symlink
- /usr/lib should exist, be a directory, and not be a symlink
[...]
Commit 793ebd5d28 (support/scripts/check-merged: use getopts instead of
getopt) intoduced a flawed use of getopts: unlike getopt, getopts does not
conume the positional arguments. This causes the check for directory
validity to also check each option as if they were directories.
For overlays, this is transparently ignored, because the checks are only lax
for overlays (missing symlinks are OK).
However, for skeletons, the checks are strict. Because of that, a missing
symlink is considered an error, when it should be considered as being OK.
The fix is to actually consume the positional args to only keep the list of
directories to validate, like is done for example in
support/download/dl-wrapper.
Fixes: 793ebd5d28
Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit 1187c34d88 (support/scripts: move merged-usr errors message
into check-merged-usr.sh) introduced the use of getopt to parse its
options; doing so allowed to use long option (with two leading dashes),
which is more descriptive than the usual one-character options.
However, getopt is part of util-linux; it is not a shell built-in.
util-linux is not a prerequisite of Buildroot, so we may end up running
on a system where it is missing.
We could add host-util-linux as a dependency when the system does not
provide getopt, but that's not very nice; even though host-skeleton does
not need to check for merged-bin for now, it does not need getopt, and
thus we could add host-util-linux (which depends on host-skeleton) as a
dependency of skeleton-custom. But that will not be tenable over the
long run, especially if/when we do a merged-bin in host dir.
Requiring that util-linux be installed system-wide is not nice either;
it's an additional requirement on the host.
We can do like we do in the oter scripts, though: use the shell built-in
getopts. Its usage is slightly different, and does not support long
options. As it's just for use in an internal script, we can live with
the less descriptive options, though.
Switch to using getopts, it removes the need for a new host dependency.
Fixes: 1187c34d88
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Enriches the input CycloneDX SBOM with vulnerability information and
analysis from the NVD database.
The NVD database is cloned using a mirror of it and the content is compared
locally. By default the path 'dl/buildroot-nvd' is used.
Example usage to analyse vulnerabilities of an input CycloneDX SBOM:
$ make show-info | utils/generate-cyclonedx | support/script/cve-check
The 'cve-check' can also be used to only enrich the vulnerabilities
present on the input SBOM with a set metadata (description, cvss,
references, ...) without applying an analysis.
With the following command the vulnerabilities ignored by Buildroot
present in the CycloneDX SBOM are enriched with description, cvss, etc
...
$ make show-info | utils/generate-cyclonedx | support/script/cve-check --enrich-only
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
[Peter: fix minor flake8 issues]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch move the 'download_nvd' call to the 'pkg-stats' script
instead of automatically calling 'read_nvd_dir'.
Since the cve.py file can be used as a library it's up to the caller to
decide whether or not to update the NVD database.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since version 256, systemd will taint the system if /usr/bin and
/usr/sbin are not merged, known as merged-bin:
# systemctl --no-pager status
● buildroot
State: running
Units: 166 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Mon 2025-07-07 19:48:05 UTC; 19s ago
systemd: 257.7
Tainted: unmerged-bin
CGroup: /
├─init.scope
[...]
Although this is not yet an error, it will be in the future.
To be as ready as we can be when that happens, forcibly enable
merged-bin, like we did when we initially added merged-usr (except
this time we carry the select from systemd, even though it is not
yet strictly required, rather than from the init entry).
Extend the runtime test to catch any tainted flag.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Norbert Lange <nolange79@gmail.com>
Cc: Sen Hastings <sen@hastings.org>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently, we partially accept that a skeleton or a rootfs overlay be
merged:
- for unmerged, we accept all kind of situations: unmerged, partially
merged, badly merged, merged-usr or merged-bin, arbitrary relative
or absolute symlinks, and whatnots;
- for merged-usr, we strictly require a properly set up merged-usr,
and we refuse a merged-bin;
- for merged-bin, we stricty require a properly set up merged-bin.
The unmerged case is inconsistent with the other cases, especially it
allows for arbitrary symlinks that may point to arbitrary locations that
may even not belong to $(TARGET_DIR) at all...
We fix that by ensuring that the skeleton and overlays strictly adhere
to the merge-level of the configuration; i.e. for an unmerged config, we
require that the skeleton and overlays be strictly unmerged, that is,
/bin, /lib, and /sbin, and their counterparts in /usr, are actual
directories.
Thus, for all three types of merge level, the skeleton and overlays must
match the configured merge level.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Starting with version 256 [0], systemd warns when /usr/bin and
/usr/sbin are different directories; in the future, it may even
refuse to boot in such a situation.
Add support for merged-bin, not unlike the support we have for
merged-usr; we also make merged-bin a sub-case of merged-usr
(i.e. it is not possible to do merged-bin without merged-usr).
[0] https://github.com/systemd/systemd/blob/v256/NEWS#L265
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Acked-by: TIAN Yuanhao <tianyuanhao3@163.com>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
We're going to need it to check merged-bin, so the naming would be
misleading as it would no longer be just about merged-usr.
Also drop the extension, it's useless.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently, we accept that the merged usr is backward, i.e. that the
/usr/bin, /usr/sbin, and /usr/lib entries be symlinks to, resp., /bin,
/sbin, and /lib. We also allow either entries to be symlinks to other
parts of the root. Both are accepted despite the comment at the top of
the script explaining what should be accepted.
However, a properly merged usr is the other way around: /bin, /sbin, and
/lib are the symlinks to resp. /usr/bin, /usr/sbin, and /usr/lib, which
are actual directories.
Fix the check-merged-usr script accordingly: implement the test as we
mean it, by testing the conditions rather than resorting to a convoluted
and incorrect use of stat(1).
Even though the split between test_dir() and test_merged() seems
superfluous, it'll come useful when we introduce support for merged-bin
in a later patch.
For skeletons, we require that the directories do exist, while we allow
them to be missing for overlays; indeed, it is perfectly legit to
provide an overlay that only contains totally unrelated directories
(e.g. /var/www to populate a webroot for example).
Extend the heading-comment to be more explicit (and drop '/' as there is
nothing to say about it).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
By moving the loop over the overlays into the script, we can generate
better error messages about how and why a skeleton or a specific overlay
is improperly setup for merged-usr.
We can also now rely on its exit code to decide whether the skeleton or
the overlays are properly setup, rather than stash the stdout/stderr to
a Makefile variable and test the emptiness thereof.
Introduce a --type option to pass the type of root to verify, for better
error reporting. This will incidentally be usefull in a future commit,
when we need to take different actions based on whether the root is a
skeleton or an overlay.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Rather than repeat the same canned sequence over and over again, move it
to a function that we can reuse as many times as needed. This will come
handy when we later need to check merged-bin.
Switch to using bash as it allows for nicer functions (local variables),
but we anyway require it globally already so that's not an additional
dependency.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Given the following criteria: `cpe:2.3:a:oneidentitty:syslog-ng:*:*:*:*:-:*:*:*`.
The former `cpe_matches` implementation would match with the following
CPE: `cpe:2.3:a:oneidentitty:syslog-ng:4.71:*:*:*:premium:*:*:*`.
The 'hyphen' ('-') meaning is "Not Attributed" (NA) a criteria with no
attributed software edition shouldn't match with a CPE with an attributed
software edition:
https://csrc.nist.gov/pubs/ir/7695/final
This patch also create a distinct 'CPE' object that aggregate the
function specifics to CPEs like it's done for 'CVE'.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The `support/script/cve.py` file is used as a library. Depending on how
you use this library you might not want to write content to stdout when
calling its function.
This patch move the 'updating' log to the 'pkg-stats' script and write
the alert when LooseVersion doesn't have a version to stderr.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The test tests.package.test_xen.TestXenArmv7 has become unstable after
commit 24ff258905 ("support/testing: test_xen: add networking"),
leading to a ~50% chance of failure on gitlab CI.
This seems to happen only with 32b (not with the 64b TestXenAarch64),
only with Qemu < 9.1, and when this happens the test will hang during a
step involving the simulated network and the dom1 (DHCP or ping).
The root cause is suspected to be a bug in the virtio-net hardware
emulation of the qemu version 7.2.15 included in the Buildroot docker
image.
As a workaround, remove all network interactions between dom1 and the
simulated network:
- Instead of the gateway, ping dom0 from dom1.
- Instead of using DHCP, configure the network with fixed IP addresses,
so that we know the IP address of the dom0.
A possible solution could have been to compile a host-qemu within the
test, by adding BR2_PACKAGE_HOST_QEMU=y and
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y in the test configuration.
This would have used the current Buildroot version (10.1.0) not affected
by this issue, but this would also have significantly increased the test
compilation time.
Since the DHCP is not really needed here to test the Xen networking
capability, this is why the network configuration simplification was
preferred here.
After those changes both Xen tests are stable on CI and locally.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/11871454648
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a package for Snagboot, the open-source
vendor-neutral recover and reflashing tool for embedded targets. We
install the dependencies needed for snagrecover and snagflash, but we
don't bother installing the dependencies of the GUI-based snagfactory,
as that would require Kivy on the host, which is Qt based.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a new host package for python-pyfatfs, which is
needed by Snagboot.
Homepage: https://pypi.org/project/pyfatfs/
A small test doing some minimal verification that pyfatfs works is
also added.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a host package for the Python 'fs' module, needed as
a dependency of pyfatfs, itself needed by Snagboot.
Homepage: https://pypi.org/project/fs/
In addition, a very simple test case is adding to check that
host-python-fs is minimally working.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The host variant of python-xmodem will be needed by Snagboot, so this
commit enables it. We also add a very simple test case that simply
makes sure we can import the module. Testing xmodem more extensively
is difficult without an actual serial port to open.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The host variant of python-pyusb will be neded for Snagboot, so it is
enabled with this commit. In addition, we're adding a very simple test
case to validate that host-python-pyusb minimally works.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
A host variant of python-tftpy will be needed for Snagboot, so enable
it, and add a very simple test for the host package. We considered
adding a test that actually starts a TFTP server with tftpy and then
downloads from that server with the client-side of tftpy, but
allocating an available UDP port reliably is tricky, so we kept the
test case very simple for now.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Colin Foster <colin.foster@in-advantage.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This quickly tests that the host-python-serial package at least
minimally works.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a new host only package for the crccheck Python
module, which is a dependency of Snagboot.
Homepage on PyPi: https://pypi.org/project/crccheck/
This commit also adds a very basic test case that allows to verify
that the Python module can be imported and that some minimal
functionality works.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Runtime test cases are currently mostly used to test target packages,
and the BRTest class is perfect for that as we can easily feed a
Buildroot configuration, boot it under Qemu and run commands inside
the emulator.
A few tests use the more basic BRConfigTest when they have special
needs that don't match with BRTest.
However, as we are going to add a number of tests to verify the proper
functionality of some host only packages, which have no visible
menuconfig option, it makes sense to add another BRHostPkgTest class
that allows to easily trigger the build of "make host-<foo>
host-<bar>", and then run some commands on the host machine. Such
tests could be done by using BRConfigTest, but that would require a
bit of duplicated boilerplate, which BRHostPkgTest allows to remove.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When run_cmd_on_host() runs a command that fails, we only get an
exception with no details to debug what happened. Let's improve that
by catching the exception, and printing the command output. This
requires redirecting stderr to stdout (instead of /dev/null) and
asking to get the output in text format.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The getunconfirmedbalance rpc call was marked as deprecated
in [1] (included in v0.19.0, released on 2019-11-08) and removed
in [2] (included in v30.0, released on 2025-10-09).
This commit replaces the old getbalance/getunconfirmedbalance rpc
calls with the new call getbalances (plural) returning a json
object containing all the data.
This commit is needed before updating bitcoin to v30.0.
[1] facfb4111d
[2] c3fe85e2d6
Cc: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When the test was introduced in [1], a comparison of the source and
the destination file was added, with a typo (the method ".cmp" name is
missing).
The test is failing with error such as in [2]:
Traceback (most recent call last):
File "/root/sample_python_scp.py", line 15, in <module>
assert filecmp("/etc/hostname", "/tmp/hostname")
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'module' object is not callable
This commit fixes that.
[1] 7470587cfa
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/11781378510
Cc: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Add a basic test involving a local SSH connection (based on dropbear
server). The test validates that we are able, through python-scp, to
fetch the /etc/hostname file.
The test needs some basic key initialization to allow connecting to the
dropbear server, hence the custom run_sample_scripts method.
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
[Julien:
- remove unneeded os import in test script
- execute setup commands with assertRunOk()
- create /root/.ssh with 0700 mode
- run sample scripts using super()
- check in the sample file that copied files are the same
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
