"boulder new" takes long to scan huge tarballs

[livingsilver94]

Consider a Rust tarball. It's a huge archive that contains multiple projects (rustc, cargo, other Rust-related tools, llvm, vendored libraries...). This alone is source of a very long scan step, plus the stone.yml outcome is probably going to be garbage due to the fact multiple projects are vendored in the tarball, whilst we are only interested in rust dependencies and build steps.

Implementing heuristics to circumvent this situation is probably hard; a relatively easy workaround is to hit CTRL+C to abort the scan and let boulder finish without pre-filling stone.yml.

Example log (partial):

[22:03:44] INFO      Beginning download
[22:03:57] INFO      Downloaded: /tmp/boulderDrafterURI-t6AsPh
[22:03:57] INFO      Extracting: /tmp/boulderDrafterURI-t6AsPh
[22:03:57] INFO      Computing hash for /tmp/boulderDrafterURI-t6AsPh
[22:04:04] INFO      Scanning sources under /tmp/boulderDrafterExtraction.GpxvBk
[22:04:09] INFO      Analysing source trees
[22:04:44] WARNING   Unknown license for: rustc-1.63.0-src/src/tools/cargo/LICENSE-THIRD-PARTY
[22:05:36] WARNING   Unknown license for: rustc-1.63.0-src/src/llvm-project/lldb/third_party/Python/module/pexpect-4.6/LICENSE
[22:05:38] WARNING   Unknown license for: rustc-1.63.0-src/src/llvm-project/clang-tools-extra/clang-tidy/cert/LICENSE.TXT
[22:06:21] WARNING   Unknown license for: rustc-1.63.0-src/vendor/jemalloc-sys/jemalloc/COPYING
[22:09:35] WARNING   Unknown license for: rustc-1.63.0-src/src/tools/rust-analyzer/editors/code/LICENSE
[22:10:39] WARNING   Unknown license for: rustc-1.63.0-src/library/stdarch/crates/intrinsic-test/acle/LICENSE
[22:11:01] WARNING   Unknown license for: rustc-1.63.0-src/library/stdarch/crates/intrinsic-test/acle/mve_intrinsics/LICENSE
[22:12:00] WARNING   Unknown license for: rustc-1.63.0-src/library/stdarch/crates/intrinsic-test/acle/neon_intrinsics/LICENSE
[22:12:23] WARNING   Unknown license for: rustc-1.63.0-src/library/stdarch/crates/intrinsic-test/acle/main/LICENSE
[22:12:45] WARNING   Unknown license for: rustc-1.63.0-src/library/stdarch/crates/intrinsic-test/acle/morello/LICENSE
[22:13:04] WARNING   Unknown license for: rustc-1.63.0-src/src/doc/rustc-dev-guide/src/licenses.md

[livingsilver94]

The license scaninng algorithm isn't super efficient, however since we're in pre-alpha stage we're building tooling in debug mode. In truth, building in release mode improves scanning time by a factor of 10.