mirror of
https://gitlab.com/buildroot.org/buildroot.git
synced 2026-09-09 07:51:59 -09:00
d1f3ae32c2
The per-package-rsync stage can add a significant amount of time to
builds. They can also be annoying as the target-finalize and
host-finalize targets are the slowest and run on every `make all`, which
is used frequently for partial rebuilds.
The per-package-rsync is slow because it launches a new rsync for each
source tree, and each rsync must rescan the destination directory and
potentially overwrite files multiple times. We can instead merge all the
rsync calls down into one call, and rsync is smarter about scanning all
the source directories and only copying over the files it needs to.
We feed the source trees to rsync in reverse-order, as this preserves
the original behaviour. I.e. when using multiple rsyncs, the last source
tree would overwrite anything in the destination. Now when using a
single rsync, we put the last tree first as rsync will select the first
file it finds.
This only supports the 'copy' mode, which is used in the finalize step.
The 'hardlink' mode would require specifying each source tree with the
--link-dest flag, and rsync only support at most 20 such trees at a
time, so we'd need a clever loop, so the 'hardlink' mode is left out, at
least for now.
Below is a benchmark running the host-finalize target for a build with
200 packages.
Benchmark 1: before copy
Time (mean ± σ): 27.171 s ± 0.777 s [User: 6.170 s, System: 14.830 s]
Range (min … max): 26.343 s … 28.566 s 10 runs
Benchmark 2: after copy
Time (mean ± σ): 6.296 s ± 0.196 s [User: 2.874 s, System: 5.600 s]
Range (min … max): 6.094 s … 6.709 s 10 runs
Summary
after copy ran
4.32 ± 0.18 times faster than before copy
Cc: Herve Codina <herve.codina@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Herve Codina <herve.codina@bootlin.com>
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Tested-by: Yann E. MORIN <yann.morin@orange.com>
Reviewed-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation. The documentation can be found in docs/manual. You can generate a text document with 'make manual-text' and read output/docs/manual/manual.text. Online documentation can be found at http://buildroot.org/docs.html To build and use the buildroot stuff, do the following: 1) run 'make menuconfig' 2) select the target architecture and the packages you wish to compile 3) run 'make' 4) wait while it compiles 5) find the kernel, bootloader, root filesystem, etc. in output/images You do not need to be root to build or run buildroot. Have fun! Buildroot comes with a basic configuration for a number of boards. Run 'make list-defconfigs' to view the list of provided configurations. Please feed suggestions, bug reports, insults, and bribes back to the buildroot mailing list: buildroot@buildroot.org You can also find us on #buildroot on OFTC IRC. If you would like to contribute patches, please read https://buildroot.org/manual.html#submitting-patches