Previously, Python builds could intermittently fail when removing files
if `find` removed a directory before searching within it [0].
An arbitrary example:
/tmp/tmp.h9KMUse5zO $ find . -type d -name __pycache__ -exec rm -rvf {} \; ; echo -e "RETURN CODE: $?"
removed directory './a/b/c/d/e/__pycache__'
find: ‘./a/b/c/d/e/__pycache__’: No such file or directory
removed directory './a/b/c/d/e/f/__pycache__'
find: ‘./a/b/c/d/e/f/__pycache__’: No such file or directory
removed directory './a/b/c/d/e/f/g/__pycache__'
RETURN CODE: 1
Now, pass the `-depth` argument to `find` so a depth-first search is
performed to avoid trying to search within an already deleted folder.
Fixes: 54d48c8cad ("package/python3: miscellaneous fixups")
Fixes: http://autobuild.buildroot.net/results/ba1d4213ae9912d53412ded6d8e257b67a4f7c0e/
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
evemu ships with an ancient py-compile script (from 2011) depending
the imp module, which was finally removed in Python 3.12.
Set autoreconf in order to get an up to date version of py-compile
which automatically handles the different python versions.
This is a similar problem as in
42eae73ef1 package/cracklib: python needs autoreconf
Fixes:
http://autobuild.buildroot.org/results/a52/a520583be6e86993d268d6535f727a352bd92028/
and many other configs where Python3 is selected and thus evemu
is configured to build the python-bindings.
Signed-off-by: Andreas Naumann <anaumann@ultratronik.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Dependencies changed: inih dependency was removed, glib2 and lua added.
Tio's meson build system fails to find luajit, so exclude that,
otherwise all versions of lua >= 5.1 are supported.
Retested the build for sparc and sparc64 and the compile issues have
been resolved upstream by reworking the baudrate selection code quite a
while ago[1].
Disable installation of man pages with new build option.
[1] a23be7f2c2
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Release Notes: https://nodejs.org/en/blog/release/v20.15.1
Fixes the following CVE's:
CVE-2024-36138 - Bypass incomplete fix of CVE-2024-27980 (High)
CVE-2024-22020 - Bypass network import restriction via data URL (Medium)
CVE-2024-22018 - fs.lstat bypasses permission model (Low)
CVE-2024-36137 - fs.fchown/fchmod bypasses permission model (Low)
CVE-2024-37372 - Permission model improperly processes UNC paths (Low)
Also these additional CVE's were fixed in the v20.12.1 and v20.12.2 releases [1][2]:
CVE-2024-27983 - Assertion failed in node::http2::Http2Session::~Http2Session() leads to HTTP/2 server crash- (High)
CVE-2024-27982 - HTTP Request Smuggling via Content Length Obfuscation - (Medium)
CVE-2024-27980 - Command injection via args parameter of child_process.spawn without shell option enabled on Windows
NodeJS tests are passing:
$ ./support/testing/run-tests -o ./outputs/ -k tests.package.test_nodejs -d dl
12:02:58 TestNodeJSModuleHostSrc Starting
12:02:58 TestNodeJSModuleHostSrc Building
13:17:15 TestNodeJSModuleHostSrc Building done
13:17:23 TestNodeJSModuleHostSrc Cleaning up
.13:17:23 TestNodeJSModuleHostBin Starting
13:17:23 TestNodeJSModuleHostBin Building
14:06:15 TestNodeJSModuleHostBin Building done
14:06:20 TestNodeJSModuleHostBin Cleaning up
.14:06:20 TestNodeJSBasic Starting
14:06:20 TestNodeJSBasic Building
14:55:40 TestNodeJSBasic Building done
14:55:45 TestNodeJSBasic Cleaning up
LICENSE hash changed due to changes in vendored components:
* copyright year update and adding spdx identifier [1]
[1] https://nodejs.org/en/blog/release/v20.12.1
[2] https://nodejs.org/en/blog/release/v20.12.2
[3] d5a316f5ea
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Expose --with-dns-delay=SEC configure option to fine tune retry interval
to match that of the system resolver.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- New command line flag -l to control klogctl(2) on/off and allow raw
kernel logs to console, use with 'quiet' kernel cmdline option
- Add reload command for systemd service
- Add *.conf support for rotate_size and rotate_count
- Add *.conf support for listen addr:port
- Fix blocking delay for unresolvable DNS names when logging to remote
- Fix failure to bind to not-yet-set IP address by retrying every 5 sec
Full ChangeLog at https://github.com/troglobit/sysklogd/releases/tag/v2.6.0
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The meson build system will be useful in future commits to have a
proper installation in staging with pkg-config files.
Drop Makefile patch which is no longer relevant.
Backport a patch removing _LARGEFILE64_SOURCE which can result in a
redefinition build error.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add the raspberry pi picotool for interacting with rp2040 binaries and
devices.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add the pi-pico sdk as a dependency for picotool. This is a bit of a
weird package, because you normally clone the git repository and point
the picotool (or rp2040 firmware project) build to it.
For picotool specifically, only headers are used from the sdk, for
building a firmware project, it is included as source files via cmake,
so this pretty much has the same semantics for that as well.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Sources of packages which are using the 'local' _METHOD_SITE or
which declare an _OVERRIDE_SRC_DIR are not archived when generating
the legal-info target.
This is problematic since those packages are the most likely to
require being published, because it's highly probable that their
sources have been modified. A typical use case is a package (e.g. linux)
that is maintained as a git submodule of the BR2_EXTERNAL and with the
LINUX_OVERRIDE_SRCDIR set appropriately, or with a tool like repo
putting it in a well-known location.
This patch generates a tar.gz archive on the fly, containing the
result of the rsync of the package's source directory, using the
same arguments as during the preparation of a build.
Note that is not possible to archive the builddir, since it may
contain artifacts of previous compilations.
In order to create a tarball that is reproducible, we reuse the
mk_tar_gz function from the download helpers. We have to provide it with
the TAR variable. We also have to cd to the build directory first,
because the script generates a temporary file in the current directory.
That script needs a date. We unforuntately don't have a good
reproducible data. The timestamps of the files in the override srcdir
itself aren't useful either: if they come from git, they typically have
the time of checkout. Therefore, we fix the date at Jan 1, 1970 (which
we can do with @0).
The mk_tar_gz function could actually work directly in the override
srcdir without the need for an rsync - except for one limitation: it
doesn't have the equivalent of RSYNC_VCS_EXCLUSIONS. It would be nice to
refactor the script so it could accept a list of exclusions instead of a
single one and also work at any level instead of just the toplevel. But
that is for future improvement.
It is not possible to consider rsync as a "normal" download method
either (which would have solved the initial problem), because if we
do that:
* the DL dir's content would be altered improperly
* the rsync + tar gz overhead would be paid on compilation and is
significant, for example, for a linux kernel
* a hash file would be necessary, which would be a burden to
maintain, for sources frequently modified
Signed-off-by: Nicolas Carrier <carrier.nicolas0@gmail.com>
[Arnout:
- Drop the auxiliary variables: their use is contained within one code
block and it's easier to read with being explicit.
- Put it in an additional subdirectory .legal-info-rsync of the build
directory, to reduce the chance of conflict with an existing
file/directory.
- Archive the directory (e.g. linux-custom) rather than its contents,
so that it matches a normal tarball.
- Use the mk_tar_gz function from download helpers to create a
reproducible tarball.
- Split long lines.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
By default U-Boot builds a default environment from its own configuration
and a board-specific set of variables. However it also allows to bypass
this entirely and define the entire default environment from a text file.
Expose this feature to Buildroot. This allows to have a file e.g. in
board/.../uboot.env which contains an easy to maintain text file with the
wanted default environment, without patching the U-Boot source code.
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Add "Upstream: N/A" tag for patches that were not sent upstream. These
are all patches that would not be acceptable for upstream.
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Commit 9a66711be1 ("package/tar: fix
linking with libiconv") added two patches that patch Makefile.am
files, but it forgot to add TAR_AUTORECONF = YES.
However, a gotcha is that TAR_AUTORECONF = YES would apply to both the
target tar, but also the host tar. But for host-tar, we can't accept
to do an autoreconf, because this would cause host-tar to have a
dependency on host-{autoconf,automake,libtool}, which in turn would
have a dependency on host-tar.
Since anyway the patches 0002/0003 that modify Makefile.am are only
useful to fix a libiconv problem that occurs when building target tar,
we disable autoreconf for host-tar, and we prevent automatic
autoreconf by touching the right Makefile.in files. This mess can be
dropped next time we update tar, as 0002/0003 will be part of the next
tar release.
Fixes:
http://autobuild.buildroot.net/results/efdf7cf9ede810ed7f766cc4138b16054bc0c18a/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some specific versions of OP-TEE need cmake on the host to build the scmi
firmware, like the newly release STMicroelectronics 3.19.0-stm32mp-r2
version.
To allow building such OP-TEE OS versions, this commit adds the option
BR2_TARGET_OPTEE_OS_NEEDS_CMAKE which when enabled ensures that host-cmake
is built before OP-TEE OS.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[Arnout: use BR2_CMAKE_HOST_DEPENDENCY]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Add initial support for STM32MP257 GPU drivers.
gcnano-binaries will be configured for STM32MP1 platforms on ARM 32-bits
and STM32MP2 on ARM 64-bits.
Update to the latest version of the gcnano-binaries blobs, which are
compatible with the last v6.1-stm32mp-r2 kernel from the
STMicroelectronics BSP.
Signed-off-by: Romain Gantois <romain.gantois@bootlin.com>
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Add missing ' to fix `make help'.
This fixes the following error:
/bin/bash: -c: line 1: unexpected EOF while looking for matching `''
make: *** [Makefile:1180: help] Error 2
Fixes: ca6c896bdd
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Steve Hay <me@stevenhay.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The patch adds the possibility to create, in addition to the usual
image, an image of type bmap that drastically reduces the amount of
data that needs to be written to an SD card, resulting in time
savings.
It looks at whether BR2_PACKAGE_HOST_BMAP_TOOLS is enabled to decide
whether it should or not generate bmap tool images. It generates bmap
images for all images referenced in the genimage configuration file,
as long as they exist in $(BINARIES_DIR).
Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This is needed to support a host build of gnupg2. It is not directly
selectable.
Signed-off-by: Ben Hutchings <ben.hutchings@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This is needed to support a host build of gnupg2. It is not directly
selectable.
Signed-off-by: Ben Hutchings <ben.hutchings@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This variable was added in commit
62c7857dfa ("package/libassuan: add host
variant"), but in fact does not make sense for a host
package. Indeed, config scripts only need to be tweaked for target
packages, and in any case, the <pkg>_CONFIG_SCRIPTS variable is only
used during staging installation (to tweak the config scripts) and
during target installation (to remove them), and neither staging nor
target installation are relevant for host packages.
Therefore, let's remove this small piece of dead code.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This variable was added in commit
6a4ad23a07 ("package/libgpgme: add host
variant"), but in fact does not make sense for a host package. Indeed,
config scripts only need to be tweaked for target packages, and in any
case, the <pkg>_CONFIG_SCRIPTS variable is only used during staging
installation (to tweak the config scripts) and during target
installation (to remove them), and neither staging nor target
installation are relevant for host packages.
Therefore, let's remove this small piece of dead code.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This is needed to support a host build of gnupg2. It is not directly
selectable.
Signed-off-by: Ben Hutchings <ben.hutchings@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In commit ca9e08277c ("package/qt5: bump
packages to latest kde submodule versions"), the version of Qt5 was
bumped to 5.15.14. This means that the version of qt5webengine being
built is now qtwebengine-v5.15.14-lts.tar.bz2.
However, qt5webengine-chromium is packaging a submodule of
qtwebengine, so we must keep them in sync, and therefore it should
have been updated.
The current version of qt5webengine-chromium,
0ad2814370799a2161057d92231fe3ee00e2fe98, was matching with
5.15.11-lts, but for 5.15.14-lts, we need to use
18c9261dc5b8aa57a0bdd5b62ce6f648cca1ef5e, as can be seen at:
https://github.com/qt/qtwebengine/tree/v5.15.14-lts/src (thirdparty submodule)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When we use Buildroot's re2 package to build qt5webengine, there is an
issue at link time as the qt5webengine mistakenly uses the abseil-cpp
headers of its built-in copy rather than the ones of our abseil-cpp
library, built as a dependency of re2. However, we build our
abseil-cpp differently than the one bundled, and that causes some
conflicts.
Since teaching Chromium's build system to not use its own abseil-cpp
headers is an adventure we'd rather not engage into, we fallback to
use the bundled re2.
Therefore, we drop in Config.in all dependencies that were inherited
from Buildroot's re2 package. This warrants a few comments:
- BR2_PACKAGE_LIBABSEIL_CPP_ARCH_SUPPORTS should be dropped because
the list of architectures supported by our Buildroot abseil-cpp
package may be different than the set of architectures supported by
the bundled abseil-cpp in Chromium. In addition, the list of
architectures in BR2_PACKAGE_QT5WEBENGINE_ARCH_SUPPORTS is also a
very limited subset of BR2_PACKAGE_LIBABSEIL_CPP_ARCH_SUPPORTS, so
this change will not expose the build of qt5webengine to additional
architectures.
- We're dropping BR2_TOOLCHAIN_GCC_AT_LEAST_8 because it was inherited
from re2. The main qt5 package anyway depends on gcc >= 5.x. We will
figure out thanks to the autobuilders exactly which gcc version is
needed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Buildroot has moved to a modern ffmpeg, and the version of Chromium
part of qtwebengine is not ready to work with this modern version of
ffmpeg. While Chromium has been updated upstream to support a more
recent ffmpeg, it requires the backport of too many patches, so let's
use the bundled ffmpeg.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Backport a small subset of an upstream patch to fix:
../../3rdparty/chromium/gpu/command_buffer/service/program_manager.cc: In member function ‘std::string gpu::gles2::Program::ProcessLogInfo(const std::string&)’:
../../3rdparty/chromium/gpu/command_buffer/service/program_manager.cc:623:25: error: ‘using re2::StringPiece = class absl::string_view’ {aka ‘class absl::string_view’} has no member named ‘as_string’
623 | return output + input.as_string();
| ^~~~~~~~~
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fixes:
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/gen_ssl_error_assistant_proto.py", line 71, in <module>
sys.exit(main())
^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/gen_ssl_error_assistant_proto.py", line 68, in main
return SSLErrorAssistantProtoGenerator().Run()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/components/resources/protobufs/binary_proto_generator.py", line 200, in Run
self._ImportProtoModules(opts.path)
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/components/resources/protobufs/binary_proto_generator.py", line 107, in _ImportProtoModules
self.ImportProtoModule()
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/gen_ssl_error_assistant_proto.py", line 33, in ImportProtoModule
import ssl_error_assistant_pb2
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/pyproto/components/security_interstitials/content/ssl_error_assistant_pb2.py", line 9, in <module>
from google.protobuf import reflection as _reflection
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/pyproto/google/protobuf/reflection.py", line 51, in <module>
from google.protobuf import message_factory
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/pyproto/google/protobuf/message_factory.py", line 49, in <module>
from google.protobuf.internal import python_message as message_impl
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/pyproto/google/protobuf/internal/python_message.py", line 59, in <module>
from six.moves import range
ModuleNotFoundError: No module named 'six.moves'
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Further down the build:
FAILED: gen/components/resources/ssl/ssl_error_assistant/ssl_error_assistant.pb
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/host-bin/python ../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/gen_ssl_error_assistant_proto.py -w -i ../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/ssl_error_assistant.asciipb -d gen/components/resources/ssl/ssl_error_assistant -o ssl_error_assistant.pb -p pyproto -p pyproto/components/security_interstitials/content/
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/components/resources/ssl/ssl_error_assistant/gen_ssl_error_assistant_proto.py", line 23, in <module>
from binary_proto_generator import BinaryProtoGenerator
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/components/resources/protobufs/binary_proto_generator.py", line 12, in <module>
import imp
ModuleNotFoundError: No module named 'imp'
Backport an upstream patch fixing this.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
With the "-webengine-icu" config option qt5webengine uses icu headers
on the build host. Unfortunately, the build fails with:
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/host-bin/python "../../3rdparty/chromium/build/toolchain/gcc_link_wrapper.py" --output="v8_snapshot/gen-regexp-special-case" -- /usr/bin/g++ -pie -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-z,defs -Wl,--as-needed -m32 -pie -Wl,--disable-new-dtags -Wl,-O2 -Wl,--gc-sections -L/usr/lib -o "v8_snapshot/gen-regexp-special-case" -Wl,--start-group @"v8_snapshot/gen-regexp-special-case.rsp" -Wl,--end-group -ldl -lpthread -lrt -licui18n -licuuc -licudata
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicui18n.so when searching for -licui18n
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicui18n.a when searching for -licui18n
/usr/bin/ld: cannot find -licui18n
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicuuc.so when searching for -licuuc
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicuuc.a when searching for -licuuc
/usr/bin/ld: cannot find -licuuc
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicudata.so when searching for -licudata
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libicudata.a when searching for -licudata
/usr/bin/ld: cannot find -licudata
/usr/bin/ld: skipping incompatible /usr/lib/libgcc_s.so.1 when searching for libgcc_s.so.1
collect2: error: ld returned 1 exit status
So use the bundled icu instead, as it needs an icu with a bitness
matching the bitness of the target, and other complexities.
Signed-off-by: Kadir Yilmaz <kadir.c.yilmaz@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Further down the build, another issue:
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/tools/grit/grit.py", line 29, in <module>
sys.exit(grit.grit_runner.Main(sys.argv[1:]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/grit_runner.py", line 314, in Main
return toolobject.Run(options, args[1:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/tool/build.py", line 254, in Run
self.res = grd_reader.Parse(opts.input,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/grd_reader.py", line 231, in Parse
handler.root.AssignFirstIds(filename_or_stream, defines)
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/node/misc.py", line 592, in AssignFirstIds
src_root_dir, first_ids = _ReadFirstIdsFromFile(first_ids_filename,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/node/misc.py", line 56, in _ReadFirstIdsFromFile
first_ids_dict = eval(util.ReadFile(filename, 'utf-8'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/util.py", line 215, in ReadFile
with io.open(abs(filename), mode, encoding=encoding) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid mode: 'rU'
Let's backport an upstream Chromium commit fixing this.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Further down into the build, another issue pops up:
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/host-bin/python ../../3rdparty/chromium/tools/grit/grit.py update_resource_ids -o gen/tools/gritsettings/default_resource_ids --add-header --depfile gen/tools/gritsettings/default_resource_ids.d ../../3rdparty/chromium/tools/gritsettings/resource_ids.spec
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/tools/grit/grit.py", line 14, in <module>
import grit.grit_runner
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/grit_runner.py", line 19, in <module>
from grit import util
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/grit/grit/util.py", line 21, in <module>
from six.moves import html_entities as entities
ModuleNotFoundError: No module named 'six.moves'
Let's fix it with the partial backport of a much bigger upstream
commit.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
We have the qt5webengine-chromium-catapult package to override the
catapult code from qtwebengine-chromium with a newer version. It was
initially introduced because the catapult code in qtwebengine-chromium
was not Python 3.x compatible.
Turns out that the catapult version we're using exhibits new build
issues related to Python 3.12 (and perhaps others), so let's update it
again to fix:
FAILED: gen/content/browser/tracing/about_tracing.js gen/content/browser/tracing/about_tracing.html
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/host-bin/python ../../3rdparty/chromium/third_party/catapult/tracing/bin/generate_about_tracing_contents --outdir gen/content/browser/tracing
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/html_generation_controller.py:21: SyntaxWarning: invalid escape sequence '\('
if re.search('url\(.+\)', contents):
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/style_sheet.py:63: SyntaxWarning: invalid escape sequence '\('
return re.sub('url\((?P<quote>"|\'|)(?P<url>[^"\'()]*)(?P=quote)\)',
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/style_sheet.py:75: SyntaxWarning: invalid escape sequence '\('
'url\((?:["|\']?)([^"\'()]*)(?:["|\']?)\)',
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/js_utils.py:7: SyntaxWarning: invalid escape sequence '\/'
return js.replace('</script>', '<\/script>')
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/parse_html_deps.py:302: SyntaxWarning: invalid escape sequence '\/'
raise Exception('Escape script tags with <\/script>')
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/html_module.py", line 28, in Parse
parser_results = parse_html_deps.HTMLModuleParser().Parse(self.contents)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/parse_html_deps.py", line 304, in Parse
return HTMLModuleParserResults(html)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/parse_html_deps.py", line 170, in __init__
self._soup = bs4.BeautifulSoup(html, 'html5lib')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/third_party/beautifulsoup4-4.9.3/py3k/bs4/__init__.py", line 243, in __init__
raise FeatureNotFound(
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/third_party/catapult/tracing/bin/generate_about_tracing_contents", line 14, in <module>
sys.exit(generate_about_tracing_contents.Main(sys.argv[1:]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/tracing/tracing_build/generate_about_tracing_contents.py", line 32, in Main
load_sequence = vulcanizer.CalcLoadSequenceForModuleNames(names)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/project.py", line 146, in CalcLoadSequenceForModuleNames
modules = [self.loader.LoadModule(module_name=name,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/resource_loader.py", line 149, in LoadModule
m.Parse(excluded_scripts)
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/third_party/catapult/common/py_vulcanize/py_vulcanize/html_module.py", line 30, in Parse
raise Exception('While parsing %s: %s' % (self.name, str(ex)))
Exception: While parsing tracing.ui.extras.about_tracing.about_tracing: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Backport an upstream Chromium patch that fixes:
FAILED: gen/services/metrics/public/cpp/ukm_builders.cc gen/services/metrics/public/cpp/ukm_builders.h gen/services/metrics/public/cpp/ukm_decode.cc gen/services/metrics/public/cpp/ukm_decode.h
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/host-bin/python ../../3rdparty/chromium/tools/metrics/ukm/gen_builders.py --input ../../3rdparty/chromium/tools/metrics/ukm/ukm.xml --output gen/services/metrics/public/cpp
/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/metrics/ukm/../common/models.py:247: SyntaxWarning: invalid escape sequence '\w'
"""A complex node type that has attributes or other nodes as children.
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/tools/metrics/ukm/gen_builders.py", line 59, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/tools/metrics/ukm/gen_builders.py", line 27, in main
data = ReadFilteredData(args.input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/tools/metrics/ukm/gen_builders.py", line 48, in ReadFilteredData
data = ukm_model.UKM_XML_TYPE.Parse(ukm_file.read())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/metrics/ukm/../common/models.py", line 439, in Parse
return self._ParseMinidom(input_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/metrics/ukm/../common/models.py", line 418, in _ParseMinidom
return self.root_type.Unmarshall(root)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/metrics/ukm/../common/models.py", line 324, in Unmarshall
child.node_type.Unmarshall(n) for n in nodes]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/tools/metrics/ukm/../common/models.py", line 302, in Unmarshall
if not re.match(attr_re, attr_val):
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/__init__.py", line 167, in match
return _compile(pattern, flags).match(string)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/__init__.py", line 307, in _compile
p = _compiler.compile(pattern, flags)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/_compiler.py", line 745, in compile
p = _parser.parse(p, flags)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/_parser.py", line 979, in parse
p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/_parser.py", line 460, in _parse_sub
itemsappend(_parse(source, state, verbose, nested + 1,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/thomas/buildroot/buildroot/output/host/lib/python3.12/re/_parser.py", line 840, in _parse
raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 1
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Two Python scripts from the upstream Chromium code base uselessly
imported the "imp" module, causing the build to fail with Python
3.12. This commit backports an upstream patch that fixes this issue.
Traceback (most recent call last):
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/core/release/../../3rdparty/chromium/mojo/public/tools/mojom/mojom_parser.py", line 23, in <module>
from mojom.generate import translate
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/mojo/public/tools/mojom/mojom/generate/translate.py", line 17, in <module>
from mojom.generate import generator
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/mojo/public/tools/mojom/mojom/generate/generator.py", line 12, in <module>
from mojom import fileutil
File "/home/thomas/buildroot/buildroot/output/build/qt5webengine-5.15.14/src/3rdparty/chromium/mojo/public/tools/mojom/mojom/fileutil.py", line 6, in <module>
import imp
ModuleNotFoundError: No module named 'imp'
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When using the default value "." for of ArchData, qt6-applications that require QML2 imports or
plugins, will not run:
In my case the wayland-egl platform plugin is not found:
qt.qpa.plugin: Could not find the Qt platform plugin "wayland-egl" in ""
And I also get a lot of qml related warnings:
qrc:/qml/main.qml:3:1: module "QtQuick.Layouts" is not installed
Prior to this fix, we had to set the QML2_IMPORT_PATH and QT_PLUGIN_PATH or make sure qt.conf is
available.
After this fix, ArchData starts with "lib", which is necessary (according to my own experiments)
for qt6-applications to find plugins and qml2 imports. The root cause is still unknown, but I
think the current solution is sufficient.
Setting ArchData to "lib/qt6" is also a good first step towards a better qt6 filesystem layout.
I will continue improving the qt6 filesystem layout in a later patch.
Signed-off-by: Roy Kollen Svendsen <roykollensvendsen@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
