TestGitHash and TestGitRefs are failing on Gitlab-CI since 2025-08-18
(first runtime-test failures report for master) [1] due a new Gitlab-CI
security settings that prevents the any use of localhost (127.0.0.1) on
the Gitlab-CI runner [2]:
"To prevent exploitation of insecure internal web services, all webhook
and integration requests to the following local network addresses are
not allowed:
Private network addresses, including 127.0.0.1, ::1, 0.0.0.0,
10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, and IPv6 site-local
(ffc0::/10) addresses."
Gitlab suggest using a second container as a Services [3] since each
containers have access to one another and can communicate when running
the job. But this is really not practical to run git daemon since the
service is started before the Buildroot git tree is fetched and git
daemon needs git-remote directory content.
See [4]:
"If you have your service that tries to write to CI_PROJECT_DIR
immediately after it's started - it will not work and fail with an
error like '/builds/nknapp/gitlab-ci-test/': No such file or directory
error. Because that directory is most probably not there yet, as the
job step that creates it is yet to be executed!"
Also, using Gitlab-CI services requires to handle GitTestBase setUp
differently on Gitlab-CI than local use of run-tests.
So Gitlab-CI services is not an option.
We could move support/testing/tests/download/git-remote to our Gitlab
group (https://gitlab.com/buildroot.org) but since it's only intended
for testing the git download backend, we don't really want to do that.
So the only remaining choice to clone the repository from the Buildroot
git tree (local machine).
To do so, we have to do the following changes:
- Replace all "git://localhost:$(GITREMOTE_PORT_NUMBER)" by
"$(GITREMOTE_DIR)". GITREMOTE_DIR will contain the absolute path to
tests/download/git-remote directory.
- Force to use git download backend using "SITE_METHOD = git" otherwise
the download infra will fall back using wget download backend.
- Cloning a git repository with submodules using the "file://" protocol
needs to be allowed from git config. It's disabled by default:
"fatal: transport 'file' not allowed"
Use a local gitconfig file generated by GitTestBase before starting
TestGitHash and TestGitRefs with:
[protocol "file"]
allow = always
This is considered safe since we are using a git repository from the
Buildroot git tree.
- Running TestGitHash and TestGitRefs on Gitlab-CI requires additional
git config settings to disable ownership check of the git tree [5].
Thanks to Arnout for advices about GIT_CONFIG_GLOBAL.
Runtime tested:
https://gitlab.com/kubu93/buildroot/-/jobs/11428504450 (TestGitHash)
https://gitlab.com/kubu93/buildroot/-/jobs/11428490035 (TestGitRefs)
[1] https://lore.kernel.org/buildroot/68a3ee65.500a0220.397724.30db@mx.google.com/
[2] https://docs.gitlab.com/security/webhooks/#allow-requests-to-the-local-network-from-webhooks-and-integrations
[3] https://docs.gitlab.com/ci/services/
[4] https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25982#note_2139685797
[5] a016b693f7
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/11271124174 (TestGitHash)
https://gitlab.com/buildroot.org/buildroot/-/jobs/11271124173 (TestGitRefs)
Cc: Arnout Vandecappelle <arnout@rnout.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This reverts commit 559bb33ae7, which
was disabling BR2_BACKUP_SITE in our tests with the following
reasoning:
support/testing: do not use s.b.o
Currently, the runtime tests will use the sources.buildroot.org backup
mirror, which is the default setup.
However, in some cases we do not want to use the backup site, because we
want to ensure that the download actually works.
This is the case for vendored packages, like cargo or golang packages,
for whbich we want to check that gthe download still works when we
update the rust or go versions, or when our download helpers change.
So, disable the use of the backup site in all runtime tests, and drop
the no-longer needed special cases.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
However, not using the backup site is causing a LOT of random failures
in Gitlab CI, making the results barely usable. Since it's more
important to get real failures than random ones due to download
issues, let's re-enable BR2_BACKUP_SITE in our testing infrastructure.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently, the runtime tests will use the sources.buildroot.org backup
mirror, which is the default setup.
However, in some cases we do not want to use the backup site, because we
want to ensure that the download actually works.
This is the case for vendored packages, like cargo or golang packages,
for whbich we want to check that gthe download still works when we
update the rust or go versions, or when our download helpers change.
So, disable the use of the backup site in all runtime tests, and drop
the no-longer needed special cases.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Python 2.7 will not be maintained past 2020.
Many scripts on the tree are used during the build and should keep
Python 2 compatibility for a while.
This is not the case for the runtime test infra. It's meant to be run in
modern distros only, so it can safely switch to support Python 3 only.
An advantage of this approach is to have less scenarios to test in.
Otherwise every change to the test infra or runtime tests would need to
be tested against both versions of the interpreter, increasing the
effort of the developers, to ensure the compatibility to Python 2 was
not broken.
In order to accomplish the change to Python 3:
- change the shebang for run-tests;
- use Python 3 urllib as a drop-in replacement for Python 2 urllib2;
- when writing the downloaded binary files, explicitly open the output
file as binary;
- when subprocess is used to retrieve the text output from commands,
explicitly ask for text output. For this, use 'universal_newlines'
because 'text' was added only on Python 3.7;
- when pexpect is used to retrieve the text output from qemu or git,
explicitly ask for text output using 'encoding';
- the code using csv currently follows the example in the documentation
for the Python 2 module, change it to follow the example in the
documentation for the Python 3 module;
- fix the relative import for test_git.py to be Python 3 compliant.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: Romain Naour <romain.naour@smile.fr>
Tested-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This reverts commit 81771cfcdc.
The download of sha1 of a special ref currently works or not depending
on the git client version in use.
With git version 2.11.0 (present in the docker image) it does not work.
With git version 2.17.1 it works.
For the sake of reproducibility, remove this part of the TestGitRefs
test case until some code gets added to the download infra to handle
sha1 of a special ref for any git client version.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/158295269
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Add a special ref to the static repo and check on the git refs test case
the download of a git package:
- with the sha1 of a special ref as version;
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: change to use the sha1 of a special ref instead of the name]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a tag to the static repo and check on the git refs test case the
download of a git package:
- with the name of a tag as version;
- with the sha1 of a tag itself as version;
- with the partial sha1 of a tag itself as version;
- with the sha1 of a commit pointed by a tag as version;
- with the partial sha1 of a commit pointed by a tag as version;
- with the sha1 of a commit reachable only by a tag as version;
- with the partial sha1 of a commit reachable only by a tag as version.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add two submodules as static repos, add a branch to the main static repo
and check on the git refs test case the download of a git package:
- repo with submodule but without support in the package;
- repo with recursive submodules with support in the package.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
[Arnout: remove handling of inconsistent tarball hashes - that's an
actual bug that should be fixed]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a branch to the static repo and check on the git refs test case the
download of a git package:
- with a sha1 reachable by a branch name, but not pointed by it, as
version. This is the most common use case for git refs in the tree;
- with a partial sha1 of a commit reachable by a branch as version;
- with a sha1 of the commit head of a branch as version;
- with a partial sha1 of the commit head of a branch as version;
Enforce the download always occurs by removing the BR2_DL_DIR used for
the tarballs generated by the git download infra.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
All upcoming tests for git refs will rely on the return code of make to
determine whether a git ref can be downloaded or not and also to
determine whether the downloaded content is correct (all of this taking
advantage of the check-hash mechanism already in place for git
packages).
So to avoid false results i.e. in the case the check-hash mechanism
become broken in the master branch, add some sanity checks before the
actual test of download git refs.
Add the minimum test case for git refs containing only sanity checks.
Reuse the commit in the static repo.
Add a br2-external with two packages to check that:
- trying to download an invalid sha1 generates an error;
- downloading a valid sha1 that contains unexpected content generates
an error.
In order to ease the maintenance and review, each upcoming patch adding
checks to this test case will add at same time the commits to the static
repo, the equivalent packages to the br2-external and code to the test
case.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add one test case to ensure the hash is checked for git packages:
- correct hash;
- wrong hash;
- no hash file.
Add required infra:
- a GitRemote class, that can start a git server in the host machine to
emulate a remote git server under the control of the test;
- a new base class, called GitTestBase, that inherits from BRTest and
must be subclassed by all git test cases.
Its setUp() method takes care of configuring the build with a
br2-external, avoiding to hit http://sources.buildroot.net by using
an empty BR2_BACKUP_SITE. It also avoids downloading not
pre-installed dependencies (i.e. lzip) every time by calling 'make
dependencies' using the common dl directory, and it instantiates the
GitRemote object.
Besides the Python scripts, add some fixtures used during the tests:
- a br2-external (git-hash) with one package for each part of the test
case;
- a static git bare repo (repo.git) to be served using GitRemote class.
Neither the br2-external nor the check hash functionalities are the
subject of these tests per se, so for simplicity limit the check to the
error codes and don't look for the messages in the log.
Thanks to Arnout for the hint about how to add a bare repo to test.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Cc: Arnout Vandecappelle <arnout@mind.be>
Tested-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Arnout: split long line; reorder imports to satisfy flake8]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
