Update torch minimum version to 2.4

[cyyever]

What does this PR do?

PyTorch 2.10 has been released. Looking forward to lift the PT minimum version again.

[Rocketknight1]

I'm not sure we automatically bump the Torch version with new releases like we do with Python! cc @ydshieh @Cyrilvallez, is that a policy?

[Cyrilvallez]

Humm no indeed, we don't have something like "we support last 5 versions". Bumps are usually not correlated with a new version, but for other internal technical reasons

[ydshieh]

There is a (not-written) convention about : if a torch version is released 2 years ago, we will drop it.

[ydshieh]

But torch 2.2 is out Jan 30, 2024, not yet fully 2 years old.

[cyyever]

The PyTorch release time is not quite regular. I recall that 2.8.0 had taken half a year. IMO a two year policy is good..

[matthewdouglas]

@cyyever PyTorch does a release approximately every quarter now.

Given #41960 this may make sense even though 2.2 isn't at two years yet. There's overlap on this with #42135.

@Rocketknight1 wdyt?

[ydshieh]

OK for me

[Cyrilvallez]

Actually, there are other instances of that functions without protection. They should use our internal is_tracing instead for security, I'm gonna update them.
We can still bump torch though, I don't mind

[Rocketknight1]

cc @cyyever this got lost (my bad), and now it's time to bump to 2.4 instead of 2.3! Would you be okay with updating the version number here and then I can approve and merge it? If not I can make a separate PR

[cyyever]

@Rocketknight1 More updates

[Rocketknight1]

Okay, now we just need to fix our own CI 😅

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Rocketknight1]

CI is passing now, and thanks for the comprehensive cleanup of version checks that we don't need anymore!

[Rocketknight1]

well okay one test is failing, hold on 🫠

[cyyever]

@Rocketknight1 It is green

[Rocketknight1]

@cyyever there was another merge conflict today - if I can get it green again after that I'll merge!

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: albert, bert, clvp, deepseek_v3, depth_anything, diffllama, esm, falcon, gpt_neo, gptj, kyutai_speech_to_text, mimi, moshi, nemotron