Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update legacy Repository usage in various example files #29085

Conversation

Hvanderwilk
Copy link
Contributor

@Hvanderwilk Hvanderwilk commented Feb 18, 2024

What does this PR do?

The usage in the example is marked for deprecation here https://huggingface.co/docs/huggingface_hub/guides/upload#legacy-upload-files-with-git-lfs. Use the new recommended methods.

I've further removed the epoch_/step_ gitignores, that seems a bit against the purpose of doing the intermediate push, for which some examples store models at an epoch.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating! Could you extend this to all the other examples which use Repository?

@Hvanderwilk
Copy link
Contributor Author

Thanks for updating! Could you extend this to all the other examples which use Repository?

@amyeroberts sure! Could you point out if I can find why the CI actions are failing? In the details it just shows as failed without any details. I ran the quality tools / tests locally and all seemed ok.

@amyeroberts
Copy link
Collaborator

@Hvanderwilk Great - thanks!

For the failing runs on CI there's two things you'll need to do to get things passing:

  • For the failing errors with pytest, rebasing onto main should resolve this
  • For the quality checks running make fixup and pushing the changes should resolve this

@Hvanderwilk Hvanderwilk force-pushed the sequence_classification_example_upload_hub branch from 1d9ae08 to 8366d95 Compare March 6, 2024 10:11
@Hvanderwilk Hvanderwilk force-pushed the sequence_classification_example_upload_hub branch from 8366d95 to da71021 Compare March 6, 2024 10:13
@Hvanderwilk Hvanderwilk changed the title [WIP] Update legacy Repository usage in `examples/pytorch/text-classification/run_glue_no_trainer.py Update legacy Repository usage in various example files Mar 7, 2024
Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the work on this and improving our examples!

Just a few small changes to make, then we're good to merge!

examples/research_projects/luke/run_luke_ner_no_trainer.py Outdated Show resolved Hide resolved
examples/pytorch/image-pretraining/run_mim_no_trainer.py Outdated Show resolved Hide resolved
examples/pytorch/language-modeling/run_clm_no_trainer.py Outdated Show resolved Hide resolved
examples/pytorch/question-answering/run_qa_no_trainer.py Outdated Show resolved Hide resolved
Comment on lines 317 to 321
with open(os.path.join(args.output_dir, ".gitignore"), "w+") as gitignore:
if "step_*" not in gitignore:
gitignore.write("step_*\n")
if "epoch_*" not in gitignore:
gitignore.write("epoch_*\n")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I've added it back and added it in an example which did not have it.

@Hvanderwilk Hvanderwilk force-pushed the sequence_classification_example_upload_hub branch from 9a2239f to e11fba4 Compare March 12, 2024 10:52
Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all this work improving our examples!

@amyeroberts amyeroberts merged commit b640486 into huggingface:main Mar 12, 2024
8 checks passed
@Hvanderwilk Hvanderwilk deleted the sequence_classification_example_upload_hub branch March 12, 2024 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants