When calling 'printvars', the 'suitable-host-package' macro is printed
(a macro is just a variable like the others, after all, just with some
parameters). Because it is printed as a variable, it is missing its
parameters, but it still tries to evaluate the $(shell) construct.
This causes spurious warning:
make[1]: support/dependencies/check-host-.sh: Command not found
Only try and call the script if there is actually a tool to check for.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 77304e5143)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
On a properly setup machine, it is totally useless to use sudo to run
docker; it is very bad practice. Instead, users really should add
themselves to the docker group.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 9f6a2dbe03)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The current stretch image we are using on the maintenance branches still
has the old Let's Encrypt certificate that expired a few weeks ago,
causign a lot of download errors:
Suggested-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The heuristic to extract the various variables of interest is pretty
crude: we filter on variables ending with certain suffixes (like
'%_VERSION' to get the version strings).
However, in doing so, we may dump variables that are not actual package
versions (especially with br2-external trees), and those may contain one
or more equal sign. And anyway, an actual package version string may
very well contain an equal sign too.
But the current situation is that the output of 'printvars' is split on
all equal signs, which will not fit in the 2-tuple we assign the result,
thus causing an exception.
Fix that by limiting to a single split.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit b919d5dbba)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently when a tag is added to the Buildroot git tree, the gitlab-ci
create a pipeline with several hundred of jobs (~750) to build all
defconfigs and execute the Buildroot testsuite.
However, there is only a limited number of gitlab-ci runner (9 runners)
and some jobs reach the timeout limit (24h) while waiting for a runner
[1]. Indeed, the Buildroot project doesn't use the Gitlab's shared
runners.
In addition to the pipeline created when a new tag is added to the
git repository, two pipelines are created each weeks to execute the
Buildroot testsuite (on monday [2]) and build all defconfigs (on
Thursday [3]).
At some point there are too many jobs waiting in gitlab due board
defconfigs builds. Indded a board defconfig requires a lot of time
(~30min) compared to other jobs in order to build a toolchain and a
kernel linux along with a basic rootfs. There is currently 262
defconfigs.
This is even worse when several pipelines are trigged at the same
time (new git tag and scheduled pipeline trigger).
In order to reduce the number of long jobs, don't build board
defconfigs with pipelines trigged on tag, keeping only the runtime
tests and the Qemu's defconfigs.
[1] https://gitlab.com/buildroot.org/buildroot/-/jobs/1758966541
[2] https://gitlab.com/buildroot.org/buildroot/-/pipelines/404035190
[3] https://gitlab.com/buildroot.org/buildroot/-/pipelines/401685550
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 8ea6eead60)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The current Gitlab CI mechanism allows to trigger all tests in a CI
pipeline by pushing a branch named <something>-runtime-tests, or to
trigger a single test in a CI pipeline by pushing a branch name
<something>-tests.<name of test>.
However, there are cases where it is useful to run a suite of tests,
for example to run all tests in tests.init.test_busybox.
This commit makes that possible by extending the current semantic of
<something>-tests.<name of test> to not expect a complete test name,
but instead to accept all tests that starts with the given pattern.
This allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox.TestInitSystemBusyboxRo
like it was the case before. But it now also allows to do:
git push gitlab HEAD:foobar-tests.init.test_busybox
to run all Busybox tests.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 23186356a1)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The pkg-stats script queries release-monitoring.org to find the latest
upstream versions of our packages. However, up until recently,
release-monitoring.org had no notion of stable
vs. development/release-candidate versions, so for some packages the
"latest" version was in fact a development/release-candidate version
that we didn't want to package in Buildroot.
However, in recent time, release-monitoring.org has gained support for
differentiating stable vs. development releases of upstream
projects. See for example
https://release-monitoring.org/project/10024/ for the glib library,
which has a number of versions marked "Pre-release".
The JSON blurb returned by release-monitoring.org has 3 relevant
fields:
- "version", which we are using currently, which is a string
containing the reference of the latest version, including
pre-release.
- "versions", which is an array of strings listing all versions,
pre-release or not.
- "stable_versions", which is an array of string listing only
non-pre-release versions. It is ordered newest first to oldest
last.
So, this commit changes from using 'version' to using
'stable_versions[0]'.
As an example, before this change, pkg-stats reports that nfs-utils
needs to be bumped to 2.5.5rc3, while after this patch, it reports
that nfs-utils is already at 2.5.4, and that this is the latest stable
version (modulo an issue where Buildroot has 2.5.4 and
release-monitoring.org has 2-5-4, this will be addressed separately).
Note that part of this change was already done in commit f7b0e0860, but
it was incomplete.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 11efcb39b2)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The pkg-stats scripts tries to match packages against
release-monitoring.org in two ways:
- First by using the "Buildroot" distribution registered on
release-monitoring.org, in which we have added a lot of mappings
between Buildroot package names and release-monitoring.org package
names. If there is a match using this distribution, the package
status is RM_API_STATUS_FOUND_BY_DISTRO, which means that the
resulting HTML has a "found by distro" statement.
- Then, if the first solution didn't work, by using the pattern
matching, as done in the check_package_get_latest_version_by_guess()
function.
However, there is a bug in this later case: it sets the package status
to RM_API_STATUS_FOUND_BY_DISTRO as well, while it should have been
RM_API_STATUS_FOUND_BY_PATTERN. Due to this bug, in the resulting HTML
file from a pkg-stats run, all packages are marked as "found by
distro" even the ones that are "found by guess".
This commit fixes that by setting the correct package status.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 9602fd94e7)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
For example with libpng: 1.6.37 instead of 1.7.0beta89
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: coalesce into a single line]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit f7b0e08605)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Adding the Image format on the Qemu command line avoid this warning:
"WARNING: Image format was not specified for 'output/TestUbi/images/rootfs.ubi' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions."
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 1ab2dd6aa5)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Traditional VT-10x terminals (and their emulators) [0] have a "magic
margins" feature that enables the last character position to be updated
without scrolling the screen: whenever a character is printed on the
last column, the cursor stays over the character, instead of moving to
the next line.
The Busybox shell, ash, attempts to defeat this feature by printing
CR,LF right after echoing a character to the last column.[1] This
doesn't play well with emulator.py. The run() method of the Emulator
class captures the output of the emulated system and assumes the first
line it reads is the echo of the command, and all subsequent lines are
the command's output. If the line made by the command + shell prompt is
longer than 80 characters, then it is echoed as two or more lines, and
all but the first one are mistaken for the command's output.
We fix this by telling the emulated system that we are using an
ultra-wide terminal with 29999 columns. Larger values would be ignored
and replaced by the default, namely 80 columns.[2]
[0] https://vt100.net/docs/vt100-ug/chapter3.html - DECAWM
[1] https://git.busybox.net/busybox/tree/libbb/lineedit.c?h=1_34_0#n412
[2] https://git.busybox.net/busybox/tree/libbb/xfuncs.c?h=1_34_0#n258
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Co-authored-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit eb3ee3078a)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Compiling on Ubuntu 20.04 generates this:
./util.c: In function ‘file_write_dep’
./util.c:54:18: warning: ‘..config.tmp’ directive writing 12 bytes into a region of size between 1 and 4097 [-Wformat-overflow=]
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~
./util.c:54:2: note: ‘sprintf’ output between 13 and 4109 bytes into a destination of size 4097
54 | sprintf(buf, "%s..config.tmp", dir);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
and similar warnings on confdata.c, lines 778, 989, 995, 1000, 1007,
1040, 1046 and 1054. Avoid the warnings by enlarging the destination
buffer of fprintf().
Normally, we want changes to kconfig to be reflected by patches in
support/kconfig/patches. This makes it easier to resync with upstream
kconfig. However, in this case, everything that is changed here is
already changed completely (and differently) upstream, so there is no
added value in keeping the patch.
Signed-off-by: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 324612d68e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.
However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:
Traceback (most recent call last):
[...]
emulator.login()
File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.
We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.
[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- don't change encoding, use codec_errors
- rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit d6d7cbb8e0)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To avoid spending some time to build the x86_64 toolchain (~20min),
switch to corei7 cpu (Nahalem) and use the prebuilt Bootlin toolchain.
We have to use the "stable" Bootlin toolchain to use the same kernel version
for the toolchain kernel headers and the running kernel.
With the "bleeding-edge" toolchain we have the "kernel too old" issue
(running kernel 4.19 vs kernel headers 5.4)
Runtime tested locally.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 521b6f8550)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Switch from the Buildroot internal toolchain for armv5 to
the prebuilt Bootlin external toolchain.
The test doesn't require to build a toolchain, there was
no prebuilt glibc toolchain recent enough at the time this
test has been introduced.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 184d20404e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This test already use builtin kernel provided by the testsuite infra:
self.emulator.boot(arch="armv7",
kernel="builtin",
options=["-initrd", img])
But a second kernel is build from the its defconfig. This second kernel
is not used by the test.
The TestRust (using BR2_PACKAGE_HOST_RUST=y) is really long to build,
save some cpu time by removing the kernel build.
This unused kernel (based on 4.11.3 release) doesn't even build with
host gcc >= 10.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit f6d438d59f)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The kernel 5.5.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
But we can't just update this test to the next linux kernel LTS 5.10.y since
the minimum gcc version has been updated to gcc 4.9 since 5.8 kernel [2]
and the Sourcery CodeBench ARM 2014.05 is used (gcc 4.8 based).
Enable arm cortex A9 and VFP support to switch to the ARM arm prebuilt
toolchain (the Bootlin toolchain could be used).
While at it use the prebuilt buildin kernel for the vexpress target
recently updated to 5.10.7.
Fixes:
https://gitlab.com/kubu93/buildroot/-/jobs/1564202094
[1] https://gcc.gnu.org/gcc-10/porting_to.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6ec4476ac82512f09c94aff5972654b70f3772b2
[3] 3cf2782906
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit e0ad7c6411)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 73278c8a70)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The kernel 4.16.7 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit e0a64dfc0e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 69de111c46)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The kernel 4.11.3 curently used by the test doesn't build with host
gcc >= 10 due the gcc default -fno-common. See GCC 10 porting guide [1].
/usr/bin/ld: scripts/dtc/dtc-parser.tab.o:(.bss+0x20): multiple definition of `yylloc'; scripts/dtc/dtc-lexer.lex.o:(.bss+0x0): first defined here
Bump to the next LTS release.
[1] https://gcc.gnu.org/gcc-10/porting_to.html
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit 322b40405e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This version bump is needed to pass the ATF test with
hardening option enabled (-fstack-protector-strong)
With the version v2.2, ATF fail due to undefined references:
./build/juno/release/bl2u/arm_tzc400.o: In function `arm_tzc400_setup':
arm_tzc400.c:(.text.arm_tzc400_setup+0x10): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0x18): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xb8): undefined reference to `__stack_chk_guard'
arm_tzc400.c:(.text.arm_tzc400_setup+0xcc): undefined reference to `__stack_chk_fail'
Since commit ccac9a5bbb, Buildroot no
longer forces ENABLE_STACK_PROTECTOR. However, we rely on the ATF build
system to handle it correctly, and this wasn't the case in v2.2.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/1524842591
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
(cherry picked from commit e5494f1fac)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
gcc 10.x is now used by default but the kernel 4.19 used by
test_docker_compose doesn't build with it.
Bump the kernel to 4.19.204 release that contains a lot of
fixes for newer gcc.
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 5d60e07e27)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit 2eaa6d0f36 (boot/uboot: fix uboot building host tools on x86
architecture) added use of $(PKG_CONFIG_HOST_BINARY), but forgot to add
the corresponding build-ordr dependency.
Add this missing depenency now.
Additionally, the associated test had an explicit host pkgconf enbled in
its configuration. This is superfluous now that uboot properly depends
on host-pkgconf, so drop that from the test.
Note: it hapenned to work, because host-pkgconf, when explicitly enabled
in the configuration, and without per-package directories, would build
before uboot and thus be available. This would fail with PPD, though,
and thus would break for TLPB.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit d0edfec1e2)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The make all command run the tools/makefile on the process.
This makefile use "pkg-config" command to support static link.
The issue is the use of pkg-config configured for crosscompiling
to build binaries tools for host architecture.
To fix it, I add pkg-config environment variable to configure it for host.
Add a test to avoid future regress on the build of U-boot.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- fix mixed space-TAB indentation
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 2eaa6d0f36)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit aa31d10808)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 4915b692c8)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.
Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit ab2d472cde)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
ijson < 2.5 (as available in Debian 10) use the slow python backend by
default instead of the most efficient one available like modern ijson
versions, significantly slowing down cve checking. E.G.:
time ./support/scripts/pkg-stats --nvd-path ~/.nvd -p avahi --html foobar.html
Goes from
174,44s user 2,11s system 99% cpu 2:58,04 total
To
93,53s user 2,00s system 98% cpu 1:36,65 total
E.G. almost 2x as fast.
As a workaround, detect when the python backend is used and try to use a
more efficient one instead. Use the yajl2_cffi backend as recommended by
upstream, as it is most likely to work, and print a warning (and continue)
if we fail to load it.
The detection is slightly complicated by the fact that ijson.backends used
to be a reference to a backend module, but is nowadays a string (without the
ijson.backends prefix).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit f31227e628)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When a --transform expression is provided, it is by default also applied
to the target of a symlink.
When we create tarballs (from git or svn checkouts), we use a --transform
expression to replace the leading ./ with the package name and version.
This causes issues when a package contains symlinks that points to
./something, as the leading './' is also replaced.
Fix that by using the 'S' transformation scope flag, as described in the
tar manual:
https://www.gnu.org/software/tar/manual/html_node/transform.html#transform
In addition, several transformation scope flags are supported, that
control to what files transformations apply. These are:
‘r’ Apply transformation to regular archive members.
‘R’ Do not apply transformation to regular archive members.
‘s’ Apply transformation to symbolic link targets.
‘S’ Do not apply transformation to symbolic link targets.
‘h’ Apply transformation to hard link targets.
‘H’ Do not apply transformation to hard link targets.
Default is ‘rsh’ [...].
Fixes: #13616
This has been checked to not change any of the existing hash for any of
our git-downloaded package (some are host-only, hence the few fixups):
---8<---
$ m="$( git grep -l -E -- -br[[:digit:]]+.tar.gz boot package/ \
|awk -F/ '{print $(NF-1)}' \
|sed -r -e 's/(imx-mkimage|netsurf-buildsystem|prelink-cross|qoriq-rcw|vboot-utils)/host-\1/g' \
-e 's/$/-source/'
)"
$ make defconfig; make clean; BR2_DL_DIR=$(pwd)/trash-me make ${m}
---8<---
Note: it is unclear what the 'H' flag does nor how it works, because the
concept of "target of a hardlink" is not obvious; probably it has to do
with how tar internally detects and stores hardlinks. Since we do not
yet have any issue with hardlinks, just ignore the problem for now, and
postpone until we have an actual issue with a real test-case.
Signed-off-by: Jean-pierre Cartal <jpcartal@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
[yann.morin.1998@free.fr:
- re-indent commit log
- add scriptlet to test existing hashes
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit c9f27fdc1b)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In Makefiles, variables are split, filtered, and otherwise mangled on
a space as a separator. In a shell, they will also be split on TABs.
We split and filter and iterate on variables in a lot of places, and
most importantly, spaces in PATH is very seldom tested, if at all, so
a lot of packages will not be working properly in such a situation.
For example, the config.guess contains constructs that are not resilient
to a space in PATH:
PATH=$PATH:/.attbin ; export PATH
Also, our fakedate will iterate over PATH:
for P in `echo $PATH | tr ':' ' '`; do
Those are only two cases, but the first means basically all
autotools-based packages are susceptible to subtle breakage.
Furthermore, Buildroot itself does not support that the top-level or
output directories are in a path with spaces anyway.
So, instead of chasing all cases that might be potentially broken,
let's just detect the case and bail out, like we already do when PATH
contains a \n, or when it contains the current working directory.
Reported-by: Dan Raymond <draymond@foxvalley.net>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit e36974d9e8)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since c043ecb20c (support/download: change format of archives
generated from svn), the svn backend uses the generic helper to
create reproducible archives.
That helper really does its job as expected, but the svn backend
is flawed in two ways:
- the first, most obvious breakage happens with versions older
than 1.9, as they do not support the '--show-item' option
for the 'info' action;
- the second is more involved, in that svn will by default
expand the old, legacy, deprecated, cumbersome CVS-style
keywords, in the form of revision marks like '$Date$' in a
C-style comment in a source file. These replacements are
done on checkout as well as on export, and they use local
settings, like the local locale and timezone.
This means that two people with different settings, will get
different sources when the svn-checkout or svn-export the same
revision from the same tree...
Needless to say that this is not very reproducible...
While the first is easily solved, the second is more involved.
We need to ensure that what source is used initially to compute
the hash, will also be the source that are used to check the hash.
There are basically two solutions:
1. we ensure the same environment, by forcing the timezone and
the locale to arbitrary values
2. we disable keyword expansion
For the first solution, this still leaves the possibility that we
miss some environment settings that have an impact on the keyword
expansion. It would mean that Yann's settings be used, as he did
introduce the hash for the only svn-downloaded package we have,
avrdude, settings which are:
TZ=Europe/Paris
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="fr_FR.utf8"
LC_NUMERIC="fr_FR.utf8"
The second option means that the generated archives change. That
means we'd have to bump the archive version for svn downloads, and
that we update the hashes for all the svn-downloaded packages.
We chose to go with the second option, because this is what really
makes more sense, rather than hard-coding arbitrary values in the
environment. And we also have only one svn-downloaded package,
avrdude.
And thus, we're reaching the trigger for this change: avrdude is
impacted by the CVS-keyword expansion issue:
https://svn.savannah.gnu.org/viewvc/avrdude/trunk/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js?revision=1396&view=markup
which would give two different files when checked out on different
machines:
diff -durN foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js
--- foo/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
+++ bar/avrdude-r1450/avrdude/atmel-docs/EDBG/common/jquery/layout/jquery.layout.js 2020-09-22 09:36:45.000000000 +0200
@@ -1,6 +1,6 @@
/**
* @preserve jquery.layout 1.3.0 - Release Candidate 30.51
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
* $Rev: 303005 $
*
* Copyright (c) 2012
@@ -4718,7 +4718,7 @@
/**
* jquery.layout.state 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5074,7 +5074,7 @@
/**
* jquery.layout.buttons 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2010
* Kevin Dalman (http://allpro.net)
@@ -5356,7 +5356,7 @@
/**
* jquery.layout.browserZoom 1.0
- * $Date: 2015-11-02 22:13:28 +0100 (Mon, 02 Nov 2015) $
+ * $Date: 2015-11-02 21:13:28 +0000 (Mon, 02 Nov 2015) $
*
* Copyright (c) 2012
* Kevin Dalman (http://allpro.net)
So we also update the hash for avrdude.
Fixes:
http://autobuild.buildroot.org/results/e3b/e3b0508047f32008ebfa83c5255ec5994b6af120/ (time issue)
http://autobuild.buildroot.org/results/48e/48e78e84b425e79cdb98c16ab40247a0fa7e9676/ (keyword expansion issue)
Reported-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Reviewed-by: Alexander Sverdlin <alexander.sverdlin@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit c92be85e3a)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
