Kernel loading untangling

[danieldk]

Kernel dependencies are somewhat hard to implement because we have a bunch not nicely-orthogonal functions.

This is the first in a series of PR to clean up kernel loading and utils.py. I'm doing this over multiple PRs to keep it reviewable. In this PR:

• Remove unnecessary nearly-identical snapshot_download calls.
• Remove kernel validation based on lockfile hashes. This is superseded by the metadata digest + signing. Plus doing this on every load has quite some overhead.
• Remove offline argument from install_kernel_all_variants. This is basically a function only meant to be used online.
• Remove variant_locks argument from install_kernel (not needed anymore, since we removed the validation).
• Remove the revision argument from load_kernel. There is no real use case for it and it complicates the logic and potential refactoring.
• Move all bits related to locking to the new locking module.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[drbh]

lgtm!

[github-actions]

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 84.0% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	10	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	63	7	89%	46, 49, 52-53, 56-57, 100
src/kernels/backends.py	212	62	71%	40, 44, 48-51, 68, 90, 108, 117, 121, 125-127, 148, 157, 161, 165-167, 188, 199, 201, 208-211, 224, 228, 232-252, 260, 283-303
src/kernels/compat.py	8	1	88%	5
src/kernels/deps.py	54	4	93%	58-59, 95, 98
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	82	7	91%	83, 113, 185, 303, 309, 322, 340
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	74	8	89%	255, 281, 289-290, 296, 300, 316-318
src/kernels/layer/layer.py	211	16	92%	169, 212, 218, 231, 339, 419-420, 432, 441, 449, 460, 489, 493, 506, 559, 589
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/locking.py	108	70	35%	42-109, 113-136, 144, 148-155, 159-169, 173-180
src/kernels/status.py	49	2	96%	23, 81
src/kernels/utils.py	230	25	89%	63, 75-79, 85-86, 244, 248, 251, 311, 319, 357-358, 396, 414, 661, 667, 680-681, 702-714, 722, 724
src/kernels/variants.py	270	22	92%	63, 94, 115, 145, 254-255, 297-300, 302, 386-393, 399-405, 436-442, 454-460, 559-561
src/kernels/verify.py	88	1	99%	32
TOTAL	1785	286	84%

Updated by the Test kernels workflow on commit f588fb62f9aa0a1f1254130af4aa0a9e70f7200f.