Add push_to_hub to no_trainer examples #13659
Conversation
sgugger
requested review from
patrickvonplaten,
patil-suraj and
LysandreJik
|
this is great! |
| else: | ||
| repo_name = args.hub_model_id | ||
| repo = Repository(args.output_dir, clone_from=repo_name) | ||
| elif args.output_dir is not None: |
There was a problem hiding this comment.
Can this arg be None? Wouldn't this lead to a failure later?
There was a problem hiding this comment.
Also we assume in the if above that args.output_dir is not None already no in:
repo_name = get_full_repo_name(Path(args.output_dir).name, token=args.hub_token)=> maybe raise Error before the repo creation if args.output_dir is None?
There was a problem hiding this comment.
This is asserted here already :-)
| if accelerator.is_main_process: | ||
| if args.push_to_hub: | ||
| if args.hub_model_id is None: | ||
| repo_name = get_full_repo_name(Path(args.output_dir).name, token=args.hub_token) |
There was a problem hiding this comment.
Does the args.hub_token always have to be passed or is the locally saved fetched token taken in case args.hub_token is None?
Also is a nice error message thrown when the token is incorrect? Since push_to_hub now defaults to True - I think it'd be important to provide a nice error message and maybe also how to disable saving if wished
There was a problem hiding this comment.
Does the args.hub_token always have to be passed or is the locally saved fetched token taken in case args.hub_token is None?
Yes, it uses the local token if none is passed.
Note that the default to push_to_hub is not True, a user has to pass it. I'll add in the README that they should execute huggingface-cli login before running the script. Will also look at the error message and open an issue on huggingface_hub if it's not clear enough.
|
Also, since |
There was a problem hiding this comment.
Fantastic, thanks @sgugger! Good usage of the Repository object :)
|
@patil-suraj The requirements |
* Add push_to_hub to no_trainer examples * Quality * Document integration * Roll out to other examples
* Add push_to_hub to no_trainer examples * Quality * Document integration * Roll out to other examples
* Add push_to_hub to no_trainer examples * Quality * Document integration * Roll out to other examples
What does this PR do?
This PR adds the
--push_to_hubflag to theno_trainerexamples in the Transformers repository. For now, it only deals withrun_glue_no_trainer.pybut once everyone has commented, the final version will be duplicated on the other examples.