A crawl killed by the Windows test watchdog leaves nothing to diagnose it with#612
Merged
Conversation
A test the suite watchdog kills at 600s left nothing to diagnose it with. local-crawl.sh sends the crawl's output to its tmpdir and hts-log.txt lives there too, but the artifact only uploads tests/*.log, so the killed test's entry was a single truncated line. The hard kill also skips local-crawl.sh's cleanup trap, so the tmpdir survives on the runner and is then discarded. Dump those logs into the test's own .log on a timeout, and clear the tmpdir so a later timeout cannot re-report it. hts-log.txt is where "More than N seconds passed.. giving up" lands, which is the question #605 asks of a wedge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
dump_crawl_logs runs on the timeout path, where the caller is already handling a failure: rm -rf can fail on Windows while the killed httrack still holds a file, and under set -e that would abort the branch whose whole job is to salvage the evidence. Renumber the test to 60: #611 adds a 59 too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the suite watchdog from #595 kills a Windows test at 600s, the artifact it uploads carries almost nothing useful.
local-crawl.shredirects the crawl's output into its own tmpdir, andhts-log.txtlives there too, but the upload glob only takestests/*.log. I killed a real bigcrawl mid-flight to see what actually survives: the test's entry in the artifact is a single truncated line, "[running httrack ...] ..", and nothing else. The hard kill also skipslocal-crawl.sh's cleanup trap, so the tmpdir holdinghts-log.txtstays on the runner and is discarded with it.#605 asks one specific question of a wedge: does the crawl log say "More than 120 seconds passed.. giving up"?
hts-log.txtis the only place that line lands, so right now the answer is unavailable and a recurrence means another archaeology dig. This dumps the killed crawl's logs into the test's own .log on a timeout, where the existing upload picks them up, and clears the tmpdir so a later timeout cannot re-report it as its own.No engine change. In the same killed-bigcrawl probe the artifact goes from zero lines to the crawl log with its error lines. Test 59 covers the helper and fails when either the salvage or the clear is removed, with a vacuity control that an empty TMPDIR dumps nothing.
This is not a fix for #605 and does not close it; it makes the next occurrence diagnosable from the artifact alone.