Correct Tapas initialization

[Rocketknight1]

Some parameters in Tapas are initialized in __init__() and not reinitialized in _init_weights(), which means that if the model is created on the meta device, those parameters do not get a weight initialization. This causes a crash later if the uninitialized memory has some NaN values in it! This caused the test_all_tensors_are_parameter_or_buffer test to be flaky.

This PR leaves tensor creation in __init__() but moves initialization to _init_weights()

cc @vasqu

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: tapas

[Rocketknight1]

run-slow: tapas

[github-actions]

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/tapas"]
quantizations: []

[vasqu]

Thanks for the fix, looks good to me. Makes me wonder how we can detect these misses more easily 😓

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Rocketknight1]

@vasqu we could expand test_all_tensors_are_parameter_or_buffer to assert no NaNs in the model parameters. Because it just runs a forward pass and doesn't check outputs, most operations complete fine even with NaNs. tapas only reports the failure because those parameters are used in torch.distribution stuff which breaks on NaN values.

This wouldn't catch every failure (because the NaNs only appear sometimes) but I think (?) it should create a flaky failure for any uninitialized model parameter. Not sure how to make that flaky failure more reliable, though!

[github-actions]

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	041416bc	workflow commit (merge commit)
PR	b1574744	branch commit (from PR)
main	1cbb9c2e	base commit (on main)

✅ No failing test specific to this PR 🎉 👏 !

[Rocketknight1]

Merging this and will think about a follow-up PR to surface this class of bug more reliably and with less flakiness.