Commit 9696d27756 "package/gpsd: condition python stuff to the proper
kconfig option" changed the condition in which the gpsd python scripts
are installed. After that change, the "gpsfake" command (which is a
python script) is no longer found and the runtime test is failing.
This commit fixes the issue by reflecting the change in the runtime
test Buildroot configuration.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit b6f4d79df2)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The mysql virtual package was removed in commit 8708f3a23a
"package/mysql: drop virtual package".
The mariadb runtime test was authored before this mysql virtual
package removal, but was merged after it, in commit 5356754d1e
"support/testing: add mariadb runtime test". Due to this, this test
always failed with the error:
Makefile.legacy:9: *** "You have legacy configuration in your .config! Please check your configuration.". Stop.
This commit fixes the issue by removing the legacy
BR2_PACKAGE_MYSQL=y configuration directive.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/7540345406
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 3da3361a1b)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Runtime tests running on test runners are subject to a high
variability in term of performance and timing. Most or the runtime
test commands are executed with a timeout, in pexpect.
Slow or very loaded test runners can use the timeout_multiplier to
globally increase those timeouts.
Some runtime test commands sometimes needs to poll or query a state,
rather than having purely sequential actions. It is sometimes hard to
know, from the test writer point of view, the maximum timeout to set, or
if a retry logic is needed.
In order to help debugging runtime tests failing due very slow
execution, this commit adds extra information on the host test runner
about its load in the run log. Relevant information are: number of
cpus, the load average at the moment the emulator is started and the
current timeout_multiplier.
Note: this change was discussed in:
https://lists.buildroot.org/pipermail/buildroot/2024-July/759119.html
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 7a6edbc7b9)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Lots of people are using broken text editors that 1. do not naturally
terminate text files with a final \n as is customary in UNIX text files,
and 2. do not respect our .editorconfig settings, which explicitly
require adding that final newline. See this nice summary of what a text
file is (with references to applicable standards):
https://stackoverflow.com/questions/12916352/shell-script-read-missing-last-line/12916758#12916758
So, it is not surprising that read does not read the last "line" of a
file, when said "line" does not end with a newline, because it is thus
not really a line.
Even though we do mandate actual text files, let's be a little bit lax
in this respect, because people may write packages, and their hash
files, in a br2-external tree, and they may not have our .editorconfig
in the directory heierarchy (e.g. if buildroot is a submodule of their
br2-external tree, or whatever).
mapfile does not suffer from this limitation, though, and correctly
reads all lines from a file, even the final line-that-is-not-a-line.
mapfile was introduced in bash 4.0, released on 2009-01-20, more than
15 years ago. Debian squeeze, released in 2011 already had bash 4.1.
Those are really ancient. So, it means we can indeed expect bash
version 4.0 or later; which means mapfile is available.
"It should be fine!"
Fixes: #15976
Reported-by: masonwardle@gmail.com
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
(cherry picked from commit ac2e6b3927)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To give us a chance to catch runtime issues (such as missing
dependencies) more easily, add a test that writes a sample PDF file,
read it back and verify the text that was read.
Like similar packages that lead to a big
rootfs (e.g. python-botocore), this test requires a separate ext2
rootfs to avoid filling the default amount of RAM available
entirely (which would cause missing files from the root filesystem and
in turn, test failures).
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 115e9493b8)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit f77d698f83 changed apply-patches to
use $TAR instead of `tar`, but did not define a fallback if $TAR is not
defined. This results in an error when calling apply-patches.sh from
outside Buildroot's Makefile. Our team uses this script to setup local
checkouts of Buildroot package's with patches for development.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 118c824b56)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When running a test that uses host-python-setuptools using the Buildroot
Docker image, for example running the following command,
> ./utils/docker-run ./support/testing/run-tests -o output -s -k tests.package.test_python_pytest.TestPythonPy3Pytest
The build fails with the following error,
> File "/home/blmaier/buildroot/output/TestPythonPy3Pytest/build/host-python-setuptools-69.2.0/setuptools/_distutils/dist.py", line 354, in _gen_paths
> yield pathlib.Path('~').expanduser() / filename
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/home/blmaier/buildroot/output/TestPythonPy3Pytest/host/lib/python3.11/pathlib.py", line 1385, in expanduser
> raise RuntimeError("Could not determine home directory.")
> RuntimeError: Could not determine home directory.
>
> ERROR Backend subprocess exited when trying to invoke get_requires_for_build_wheel
Python setuptools is looking for $HOME but failing to find it.
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Note that here, it is in fact not really relevant. We only extract a
tarball, and we don't use any "modern" or GNU-only options like
--strip-components. However, for consistency it's better to use the same
tar everywhere.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: "Maier, Brandon L Collins" <Brandon.Maier@collins.com>
Reviewed-by: brandon.maier@collins.com
[Arnout: quote TAR="..."]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
No functional change as we install a python symlink, but use python3 for
consistency with the other scripts.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The python-magic-wormhole runtime test can randomly fail on slow
runners, see [1].
The issue is that the sending command is started first in background
_without_ redirecting its output to /dev/null. The receiving command
is started after, expecting the message to be printed on the first
standard output line. On slower systems, the sending command still
print messages while the test controller expect output from the
receiving command. The expected string finally appear, but not on the
first line. This makes the test fail.
This commit fixes the issue by redirecting all outputs (stdout, stderr)
of the sending command to /dev/null. To help even more, the sleep time
is moved from the emulator to the test controller. The sleep time is
also multiplied by the timeout_multiplier.
Fixes: [1]
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/6888691508
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The python-hid package got tweaked when applied in commit 4432b08790
(package/python-hid: new package) so it now selects
BR2_PACKAGE_HIDAPI, and we therefore do not need to explicitly enable
hidapi in the test.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Our git download backend switches the remote of our local clone, and
fetches all refs and tags from that remote.
When the local clone has a tag fetched from another remote, and the new
remote also has a tag by the same name, and that tag points to another
commit, then git refuses to fetch the new tag and exits in error, as it
considers that the new tag would clobber the existing one. This is safe
and sane behaviour when run interactively with a human that can take a
decision.
However, in our case, we don't care about any tags that were present
before, as only the last one makes sense in our case: the one from the
remote the user has requested for the current build.
Tell git to forcefully pull tags, even if they would clobber existing
ones.
Note that, although this changes the git backend, it does not change the
content of generated archives, so we do not need to bump the suffix
version.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
We can't stay in the past forever and ever...
Since tar 1.35, the way some fields (devmajor and devminor) are stored
has changed. These fields exist for each file in the tarball, but only
used for device nodes. In previous versions of GNU tar, they were set to
zero; since 1.35, they are set to empty.
Although this doesn't change anything about the content of the tarball,
and it will be extracted in exactly the same way regardless of the tar
version used for extracting, it does change the hash of the tarball.
Therefore, we have to
- make sure that the correct version of tar is used;
- update the format version so that the filename is different from
before.
Increment all BR_FMT_VERSION by one.
Require tar >= 1.35 instead of < 1.35.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
[Arnout: also increment BR_FMT_VERSION and extend the commit message]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Replace the names of the tarballs in the hash files to -git3.
We don't have any symlinks in the tests, so the hashes themselves don't
change.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Currently, when we generate archives, we rely on a few assumptions and
mechanisms to ensure reproducilibity. So far, we mostly accounted for
the content (i.e. content, filenames, and path) of the files we
archived, and this is OK (git and svn should provide reproducilbe
content by design, and cargo and go vendoring are also supposed to be
generating reproducible content.
However, tarballs do not only contain the content of the files; they
also have a few metadata about those files. Beyond filenames and paths,
which are already reproducible, there is the timestamp, the user and
group name and ID. Those are also accounted for and made reproducible.
The final touch (so far!) is that files have access rights (aka mode),
and those too are stored in tarballs. So far we accounted for those by
ensuring that Buildroot would always run under a known umask, thus
generating files with reproducible modes.
That falls short in one case that we did not envision, though: a shared
download directory, where extended attributes are set to provide a
default ACL that is permissive, to allow two or more users (with
different uid and gid) to all read and write to such a directory. This
is trivially achieved with something like:
$ mkdir -p "${BR2_DL_DIR}"
$ setfacl -m 'default:user::rwx' "${BR2_DL_DIR}"
$ setfacl -m 'default:group::rwx' "${BR2_DL_DIR}"
$ setfacl -m 'default:other::rwx' "${BR2_DL_DIR}"
This has the effect that:
- files below BR2_DL_DIR are all set with user, group, and world read
and write access,
- files executable by the owner will also be group and world
executable,
- directories are user, group, and world readable, writable, and
searchable.
This means that all the archives we generate from files in BR2_DL_DIR
will have modes that are different from those generated on other systems,
where only the traditional umask is used.
There are various solutions to solve that issue:
- detect the situation and abort: that's not nice, because users have
a legitimiate reason to want to share that directory,
- find a solution for each affected download mechanism: git, svn, hg,
cvs, bzr... and for each of the affected vendoring mechanism: go and
cargo [0]; this is not nice, because it means a lot of repetition,
with the risk that they diverge over time (e.g. one is fixed for a
newer issue, while the others are left out due to an oversight...)
- find a single, common solution that works in all cases, whatever the
download mechanism and/or vendoring: this is the best, because we
can extend and fix it once and everything else benefits from it.
We obviously go for the third option.
The common solution is rather simple. When creating the tarball in
support/download/helpers, give an option to tar to set the group and
other permissions to those of the user, but without write permission.
This implies that we must bump the version-suffix for the download
backends [1] and for the vendoring post-processes. It also implies that
the hash may change, under the following circumstances:
- Symlinks normally have permissions 0777 (because symlink permissions
are in fact meaningless). They will now have permission 0755 in the
tarball.
- If the original tarball (for vendored go and cargo packages) contained
files that are readable or executable by owner but not by group or
other, they will now be readable resp. executable by group and other
too. Note that for writeable it is not the case, because those were
already handled by our 0022 umask (which makes them not writeable by
group and other).
Because the hash may change, we need to update the BR_FMT_VERSION for
everything that creates tarballs. Go and cargo didn't have one up to
now, the the previous commit added the possibility to give one. The ones
for git and svn have to be updated. Since it is now possible to have a
suffix for both the VCS and the post-processing, change the suffix to
something more descriptive than "-brX", i.e. -git3 for git, -go1 for
golang, etc.
The hash updates and filename changes will be handled in a follow-up
commit.
[0] Note however that the vendoring is currently not done in a
sub-directory of BR2_DL_DIR, but the cargo and go caches are located
there. Files that get copied from there to the vendoring area would be
tainted as well, and thus we want to address that situation as well.
[1] we currently do not have a CVS version suffix, because we do not
guarantee the reproducilibity of CVS archives (we can't); for hg, we are
currently using hg's own archive tool, and presumably that does not have
the mode issue because it is not using the checked-out files. Still,
doing the mode fix in a single location will help extend those two
backends in the future (if that ever happens...).
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The version suffix for the git-generated archives has changed, so update
the filenames accordingly in the hash files. The content of the archives
has not changed, though.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
