You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A walk of sources or generates no longer leaves files out on an error.
It moves from directory to directory by handle, one name per lookup, so the
symlinks along a path no longer add up to the kernel's limit (a single link
whose own target spells out a long chain still can, and then ends its branch),
and a wildcard stops at the first directory it would revisit: a symlink loop
is cut there, the link kept as an entry. 4.8.x followed a loop until the
kernel refused, which past about 20 links happens at a varying depth on a busy
machine and changed the fingerprint from run to run. A name that is gone,
unreadable or not a directory (ENOENT, EACCES, ENOTDIR), or a link the kernel
will not resolve (ELOOP), ends a branch as in the shell; a directory removed
or made unreadable between its listing and its opening is skipped that way.
Any other error (too many open files, an I/O error) fails the task, naming the
path, where 4.8.x silently skipped the directory. --status reports such
errors too, instead of empty file lists. On Linux, a directory that can be
searched but not read is still crossed by a literal path, as by the shell.
Hashing still opens each file by its path, so a file behind more than about 20
nested links can fail to hash on a busy machine: an error, never an omission.
On Windows lookups still go through paths, and a loop is recognised by its
canonical path (type-checked, not run-tested).
Sources are hashed at compile time only when something reads {{.CHECKSUM}}. Compiling a task, the quick compile that checks its
platforms and required vars, and compiling every task a task: or from:
reference reaches no longer hash sources just to fill the variable. A task
reading it (in commands and their vars, task names, preconditions, the cache
block, deferred commands or requires, through a ref: var or Jinja debug()), and every task under --summary or templated output: group
markers, keeps the compile-time value as before. For the others, a task with generates computes the value once just before its build-once lock, after setup: the sources hash in the lock name (<task>:<cache checksum>), the
cache key and the cache entry's recorded sources is now taken there rather
than before setup. A task without generates takes no lock and has no cache
entry, and simply reuses its up-to-date reading. A cache entry is now restored
only when the run's cache key matches what its up-to-date check read, and
saved only when it matches what the run built from: a key taken before
preconditions, a lock wait or another writer changed the sources is skipped
(noted with -v), for these tasks and for those keeping the compile-time
value alike.
Fingerprinting walks each directory once and takes file types from the
listing, so hashing a regular file costs one stat instead of about four. A
source or generated file matched by the walk and removed before it is hashed
now fails the task, naming it, where it could be left out before, and so does
one whose type changed in between (a file replaced by a symlink, say). On
Windows that check opens reparse points without following them; it is
type-checked there but not run-tested, and reading files behind filter-driver
reparse points (deduplicated files, OneDrive placeholders) is unverified.
A cache restore cancelled before it is verified is undone, and so is one
that cannot durably mark itself kept: the task then runs normally, and an --import-cache fails. Recovering a restore cut short by a crash puts back
the originals it moved aside over entries it never verified, but only where an
entry is still exactly what the restore wrote: an entry changed since is kept,
with a warning naming the original, saved beside it as .task-restore-original-<name>-<random>. A restore cut short while committing
keeps its entries.
Fingerprints are read off the thread that schedules tasks. Tasks running
in parallel now fingerprint at the same time, and interrupts, output and lock
renewals are handled while a large tree is being hashed.
A run reads its sources and generates once after its commands. The
drift check, the saved fingerprint and the cache entry all use that one
reading, instead of reading the files up to three more times. The drift
warning now names the sources whose content changed, rather than those whose
size or modification time did.
Fingerprinting digests files in parallel, on at most as many threads as
there are CPUs available, shared by every task hashing at the same time. The
checksum does not depend on the number of threads.
Fingerprints use a new checksum format. Each file is recorded with its
path relative to the task directory, its kind (file or symlink) and its own
digest, in path order, and the task's rules by their fields, so adjacent
inputs no longer run together, and sources outside the task directory no
longer make the checksum depend on the checkout directory's name. Checksums
are 32 hex digits and no longer match the Go implementation: every task with sources rebuilds and misses the cache once, and one that renders {{.CHECKSUM}} in a field its identity covers (commands, their vars, task
names in deps/setup and their vars, preconditions, the cache block) also
gets a new checksum file name. A source that is neither a regular file nor a
symlink (a named pipe, a device) now fails the task instead of being read, and
fingerprinting errors name the file.