Ignore Cargo.lock for subfolders

[hvaara]

No description provided.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[Narsil]

Shouldn't we remove tokenizers/Cargo.lock too ?

[hvaara]

I think it already is right? I only see Cargo.toml, not Cargo.lock, in that folder.

[Narsil]

My bad !

[kira-bruneau]

Hi! I was just wondering why these lock files were removed?

The NixOS build of tokenizers' python bindings uses the lock file to reproducibly fetch the rust dependencies: https://github.com/NixOS/nixpkgs/blob/eafd00c16a2113ca38017f63fd1b348384e422b4/pkgs/development/python-modules/tokenizers/default.nix#L72-L76.

Without it we'd have to manually generate a lock file our self at each version bump, and that may be different than what's actually used in pypi.org.

[Narsil]

Because Cargo.lock within the bindings folders are constantly changing, leading to very noisy PRs.

It's the same reason they weren't added in safetensors. As said over there, having reproducing builds is something which is nice.
But the pain of having a constantly changing file which has nothing to do with the code is also real.

In any case, it shouldn't be committed in the library folder (since it's a library) but could be kept in the bindings. Ideally though it should be only present in the release branches which is not the case right now.

[bobvanderlinden]

Hmm, I ran into the same problem. It's hard to reproduce tokenizers correctly without a Cargo.lock file. I created an issue for this: #1226