Runtime tests running on test runners are subject to a high
variability in term of performance and timing. Most or the runtime
test commands are executed with a timeout, in pexpect.
Slow or very loaded test runners can use the timeout_multiplier to
globally increase those timeouts.
Some runtime test commands sometimes needs to poll or query a state,
rather than having purely sequential actions. It is sometimes hard to
know, from the test writer point of view, the maximum timeout to set, or
if a retry logic is needed.
In order to help debugging runtime tests failing due very slow
execution, this commit adds extra information on the host test runner
about its load in the run log. Relevant information are: number of
cpus, the load average at the moment the emulator is started and the
current timeout_multiplier.
Note: this change was discussed in:
https://lists.buildroot.org/pipermail/buildroot/2024-July/759119.html
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The previous commit removes s6-test from the runtime tests. This
commit adds the same test for eltest, which replaced s6-test.
Signed-off-by: J. Neuschäfer <j.neuschaefer@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
s6-test was removed in s6-portable-utils 2.3.0.0, and replaced with
eltest (from execline). This change broke the existing runtime tests
with commit 42fad03182 ("packages: bump skalibs-related packages").
Instead of s6-test simply use the shell built-in "test", which provides
sufficient functionality.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/7338388010
Signed-off-by: J. Neuschäfer <j.neuschaefer@gmx.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The TestInitSystemNone bypasses the normal init and instead launches
directly into a shell. So it needs to bypass the login code and go
directly to a shell.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This test started failing at commit
0cad947b96
"support/testing/infra/emulator.py: fix qemu prompt detection" with the
error message
AttributeError: 'NoneType' object has no attribute 'run_command'
This is because we changed emulator.run() so that emulator.login() must
be called first. But this test skips the login and goes directly to a
shell. Use the new emulator.connect_shell() function which prepares the
shell without logging in.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The patch adds the possibility to create, in addition to the usual
image, an image of type bmap that drastically reduces the amount of
data that needs to be written to an SD card, resulting in time
savings.
It looks at whether BR2_PACKAGE_HOST_BMAP_TOOLS is enabled to decide
whether it should or not generate bmap tool images. It generates bmap
images for all images referenced in the genimage configuration file,
as long as they exist in $(BINARIES_DIR).
Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Note: this test was not working in Buildroot test infrastructure
before commit [1] was merged, because dmidecode has the string "# "
in its output.
[1] 0cad947b96
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Lots of people are using broken text editors that 1. do not naturally
terminate text files with a final \n as is customary in UNIX text files,
and 2. do not respect our .editorconfig settings, which explicitly
require adding that final newline. See this nice summary of what a text
file is (with references to applicable standards):
https://stackoverflow.com/questions/12916352/shell-script-read-missing-last-line/12916758#12916758
So, it is not surprising that read does not read the last "line" of a
file, when said "line" does not end with a newline, because it is thus
not really a line.
Even though we do mandate actual text files, let's be a little bit lax
in this respect, because people may write packages, and their hash
files, in a br2-external tree, and they may not have our .editorconfig
in the directory heierarchy (e.g. if buildroot is a submodule of their
br2-external tree, or whatever).
mapfile does not suffer from this limitation, though, and correctly
reads all lines from a file, even the final line-that-is-not-a-line.
mapfile was introduced in bash 4.0, released on 2009-01-20, more than
15 years ago. Debian squeeze, released in 2011 already had bash 4.1.
Those are really ancient. So, it means we can indeed expect bash
version 4.0 or later; which means mapfile is available.
"It should be fine!"
Fixes: #15976
Reported-by: masonwardle@gmail.com
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Teach the generic package handling code how to extract zstd (.tar.zst)
archives. When zstd is not installed on the host, host-zstd gets built
automatically.
Signed-off-by: Matt Staveley-Taylor <matt.stav.taylor@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The current implementation of buildroot depe dependency graphing
either does forward- or reverse-dependency traversal.
This patch enables buildroot to graph forward and reverse dependencies
on the graph for the same package: (Diagram Credit: Yann E. MORIN)
$ make pkg-d-graph-both-depends
pkg A -. .-> pkg E
\ /
pkg B ----> pkg D ----> pkg F
/ \
pkg C -' '-> pkg G
In the above example a single graph shows pkg {A,B,C} are needed
by pkg D, and pkg D is a dependency of pkg {E,F,G}.
Makefile help and manual are also updated.
Signed-off-by: Steve Hay <me@stevenhay.com>
[Arnout:
- remove DEPTH and RDEPTH, their functionality is already covered by
BR2_GRAPH_DEPS_OPTS;
- remove --rdepth, it was felt to not add sufficient added value;
- add the new target to the manual;
- fix flake8 errors.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Instead of having done_deps as a global variable, pass it as an argument
to the print_pkg_deps function. This will allow the recursive search to
be started multiple times.
Signed-off-by: Steve Hay <me@stevenhay.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Since we copied a piece from code from pexpect, we need to also follow
the license of pexpect. Effectively, that makes the file licensed under
both our default GPL-2.0 and pexpect's ISC license. This is already
hinted at in the comment inside _repl_sh_child, but let's make it
explicit by adding a license header to the file.
Ideally, of course, each file in Buildroot would have a license file.
This was done with the reuse tool:
reuse annotate -l GPL-2.0 -l ISC support/testing/infra/emulator.py
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The qemu.run() method can break when a command happens to output the
string "# " to stdout. This is because qemu.run() detects when a command
has completed by searching for the shell prompt, which by default is
"# ". It then captures everything before the "# " as the commands
output, causing the rest of output to be lost.
Instead use the pexpect libraries REPLWrapper to handle running
commands. It has hooks to set a custom prompt and avoid some other
pitfalls of wrapping a shell.
We unfortunately can't reuse replwrap._repl_sh directly, because it
tries to spawn a command while we already have a qemu started. So we
need to copy that code into _repl_sh_child. While we're at it, also
define our own prompt strings.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
[Arnout:
- Make all arguments to _repl_sh_spawn non-optional.
- Move non_printable_insert to a local variable instead of an argument,
we don't need to override it.
- Copy the comment from _repl_sh that explains why non_printable_insert
is needed.
- Add a comment about timeouts.
- Rename spawn to child (we don't actually spawn anything so this felt
more natural, even though the class
- Use single quotes instead of triple quotes, and explicitly escape the
nested quotes.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Now we have the post-process options, we can use them to pass the
manifest path instead of passing it through an environment variable.
For now, we support both. The environment variable approach will be
removed again later.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Currently, we have no explicit, dedicated way to pass information to the
download post-process scripts, even though we do already need to pass
the path to the manifest when it is not the default, which we do with an
environment variable.
We'll soon need to be able to pass additional information to
post-process scripts, like instructing cargo to not use a lock file when
the package is not locked.
Extend the dl-wrapper with a new option with an argument, where the
argument will be passed as-is to the post-process script.
We explicitly do not document this new variable, as it is expected to
only be used in our package infrastructures (although there is currently
exactly one package that will directly need it, and we hope to drop that
in the future).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The python-pymupdf and mupdf packages do not follow the exact same
version numbers anymore. There is no specific mention of version
compatibilities in the upstream package, so update our comment to just
say that both should be "compatible" (to try to avoid one being
updated without the other).
The hardcoded paths we used to remove from setup.py are no longer
present, so the post-patch hook is removed. The new setup.py instead
uses new environment variables which we now provide.
This new python-pymupdf version fails at runtime when mupdf builds
static libraries.
Note also that python-pymupdf is gradually switching to a new
implementation that requires mupdf to be built with python
bindings. For now, both implementations are still available but we
only compile the old one. The runtime test is adapted accordingly as
the legacy implementation has to be imported with "fitz_old".
While at it, the dependencies are also split to one per line to make
them easier to diff in the future.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
[Raphaël:
- fix cross-compilation
- remove unneeded dependencies
- update to 1.23.22
- update the commit message
]
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
To give us a chance to catch runtime issues (such as missing
dependencies) more easily, add a test that writes a sample PDF file,
read it back and verify the text that was read.
Like similar packages that lead to a big
rootfs (e.g. python-botocore), this test requires a separate ext2
rootfs to avoid filling the default amount of RAM available
entirely (which would cause missing files from the root filesystem and
in turn, test failures).
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add optional c extension for python-ruamel-yaml which can speed up yaml
loading/parsing.
Extend the ruamel-yaml runtime test to check if the c extension works
correctly.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Recent versions of wget, starting with wget 2.0, aka wget2 thereafter,
no longer support FTP (nor FTPS, aka FTP-over-SSL). wget2 is packaged in
Fedora 40, recently released; F40 does not even have the old wget
available in its repository anymore.
Introduce cURL as a download backend, that we use for FTP and FPTS
protocols.
Note that the -q flag does not means being quiet; it means that a curlrc
file should not be parsed. The long option is --disable, which meaning
is not much more obivous than the short -q. It also has to be the first
option on the command line.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Version 2.1.0 is a major bump, with an API change that should be
backward compatible with version up to 1.6.1.
The runtime test failed in timeout because the crng init did not have
time to complete in the default 5-second timeout; tests showed that the
trip-off would be between 6s and 7s here, so increase it enough to leave
some margin on less-capable machines.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Asaf Kahlon <asafka7@gmail.com>
Cc: Davide Viti <zinosat@gmail.com>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This patch was created as a fix to a problem that occurred during the
compilation of QEMU:
>>> qemu 8.1.1 Patching
Applying 0001-tests-fp-disable-fp-bench-build-by-default.patch using patch:
patching file tests/fp/meson.build
Hunk #1 succeeded at 138 with fuzz 2 (offset -502 lines).
Applying 0002-softmmu-qemu-seccomp.c-add-missing-header-for-CLONE_.patch using patch:
patching file softmmu/qemu-seccomp.c
Applying 0004-tracing-install-trace-events-file-only-if-necessary.patch using patch:
patching file trace/meson.build
With the bump to version 8.1.1, the patch that disabled the compilation
of the fp-bench test does not report any errors, even though the patch
itself is no longer applicable. The only noticeable message is:
"Hunk #1 succeeded at 138 with fuzz 2 (offset -502 lines)."
As reported by the patch man page:
"With context diffs, and to a lesser extent with normal diffs, patch can
detect when the line numbers mentioned in the patch are incorrect, and
attempts to find the correct place to apply each hunk of the patch.
As a first guess, it takes the line number mentioned for the hunk, plus
or minus any offset used in applying the previous hunk. If that is not
the correct place, patch scans both forwards and backwards for a set of
lines matching the context given in the hunk. First patch looks for a
place where all lines of the context match. If no such place is found,
and it's a context diff, and the maximum fuzz factor is set to 1 or more,
then another scan takes place ignoring the first and last line of
context. If that fails, and the maximum fuzz factor is set to 2 or more,
the first two and last two lines of context are ignored, and another
scan is made. The default maximum fuzz factor is 2.
If the hunk is installed at a different line from the line number
specified in the diff, you are told the offset. A single large offset
may indicate that a hunk was installed in the wrong place. You are also
told if a fuzz factor was used to make the match, in which case you
should also be slightly suspicious."
By setting the maximum fuzz factor to 0, we avoid that patches which
cannot be applied are incorrectly reported as valid, with positive
side-effects on version bumps.
Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Commit f77d698f83 changed apply-patches to
use $TAR instead of `tar`, but did not define a fallback if $TAR is not
defined. This results in an error when calling apply-patches.sh from
outside Buildroot's Makefile. Our team uses this script to setup local
checkouts of Buildroot package's with patches for development.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
