From 145e8e74068207a89088d716de4dac9264ed62ca Mon Sep 17 00:00:00 2001 From: "Yann E. MORIN" Date: Sun, 10 Mar 2024 00:44:29 +0100 Subject: [PATCH] support/download/check-hash: accept hash files without terminating \n Lots of people are using broken text editors that 1. do not naturally terminate text files with a final \n as is customary in UNIX text files, and 2. do not respect our .editorconfig settings, which explicitly require adding that final newline. See this nice summary of what a text file is (with references to applicable standards): https://stackoverflow.com/questions/12916352/shell-script-read-missing-last-line/12916758#12916758 So, it is not surprising that read does not read the last "line" of a file, when said "line" does not end with a newline, because it is thus not really a line. Even though we do mandate actual text files, let's be a little bit lax in this respect, because people may write packages, and their hash files, in a br2-external tree, and they may not have our .editorconfig in the directory heierarchy (e.g. if buildroot is a submodule of their br2-external tree, or whatever). mapfile does not suffer from this limitation, though, and correctly reads all lines from a file, even the final line-that-is-not-a-line. mapfile was introduced in bash 4.0, released on 2009-01-20, more than 15 years ago. Debian squeeze, released in 2011 already had bash 4.1. Those are really ancient. So, it means we can indeed expect bash version 4.0 or later; which means mapfile is available. "It should be fine!" Fixes: #15976 Reported-by: masonwardle@gmail.com Signed-off-by: Yann E. MORIN Signed-off-by: Arnout Vandecappelle (cherry picked from commit ac2e6b392791085bc29fa21901265a8eed4ae0ee) Signed-off-by: Peter Korsgaard --- support/download/check-hash | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/support/download/check-hash b/support/download/check-hash index 9db647885a..d18ec8b134 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -78,8 +78,10 @@ nb_checks=0 for h_file in "${h_files[@]}"; do [ -f "${h_file}" ] || continue : $((nb_h_files++)) - # shellcheck disable=SC2094 # we're really reading it only once - while read -r t h f; do + # mapfile reads all lines, even the last one if it is missing a \n + mapfile -t hash_lines <"${h_file}" + for hash_line in "${hash_lines[@]}"; do + read -r t h f <<<"${hash_line}" case "${t}" in ''|'#'*) # Skip comments and empty lines @@ -87,13 +89,12 @@ for h_file in "${h_files[@]}"; do ;; *) if [ "${f}" = "${base}" ]; then - # shellcheck disable=SC2094 # we're only printing the h_file filename check_one_hash "${t}" "${h}" "${file}" "${h_file}" : $((nb_checks++)) fi ;; esac - done <"${h_file}" + done done # shellcheck disable=SC2086 # nb_h_files is a non-empty int