Push to hub mixins that do not leverage git

[LysandreJik]

This draft PR proposes a refactor of both mixins, ModelHubMixin and push_to_hub_keras, so that they do not leverage git under the hood. It is the continuation of #321.

This PR was done to present how it would behave. It was not done with backward-compatibility in mind, but instead by thinking of the best API, therefore with lean signatures. I recommend reading the files themselves rather than the diff as they are drastically simpler:

If we choose to go this way, a standard backward compatibility approach would be implemented.

PyTorch	Keras
Relevant code: ModelHubMixin	Relevant code: Keras
def push_to_hub( self, repo_id: str, *, commit_message: Optional[str] = "Add model", private: Optional[bool] = None, api_endpoint: Optional[str] = None, token: Optional[str] = None, branch: Optional[str] = None, config: Optional[dict] = None, ) -> str:	def push_to_hub_keras( model, repo_id: str, *, log_dir: Optional[str] = None, commit_message: Optional[str] = "Add model", private: Optional[bool] = None, api_endpoint: Optional[str] = None, token: Optional[str] = True, branch: Optional[str] = None, config: Optional[dict] = None, include_optimizer: Optional[bool] = False, task_name: Optional[str] = None, plot_model: Optional[bool] = True, **model_save_kwargs, ):
import os.path from typing import List import torch from huggingface_hub import ModelHubMixin from torch import nn class MyModel(nn.Module, ModelHubMixin): def __init__(self): super().__init__() self.ln_1 = nn.Linear(10, 100) self.ln_2 = nn.Linear(100, 10) def __call__(self, *args, **kwargs): return self.ln_1(self.ln_2(*args, **kwargs)) def _save_pretrained(self, save_directory) -> List[str]: path = os.path.join(save_directory, 'pytorch_model.bin') torch.save(self, path) return [path] MyModel().push_to_hub( 'Jikiwa/my-new-model', commit_message="Adding a config.", config={'parameter': 2} )	import tensorflow as tf from huggingface_hub import push_to_hub_keras class MyModel(tf.keras.Model): def __init__(self, **kwargs): super().__init__() self.l1 = tf.keras.layers.Dense(2, activation="relu") dummy_batch_size = input_dim = 2 self.dummy_inputs = tf.ones([dummy_batch_size, input_dim]) def call(self, x): return self.l1(x) model = MyModel() model(tf.ones([2, 2])) push_to_hub_keras( model, 'Jikiwa/my-new-keras-model', commit_message="Committing model number 2!" )

cc @osanseviero @NielsRogge @nateraw @merveenoyan

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[adrinjalali]

Merging #849 should reduce the diff here.

[merveenoyan]

I loved the refactor with HF API! I can open a PR for tests if you want @LysandreJik

[LysandreJik]

Thank you for the proposal @merveenoyan, I'll likely proceed to a backward compatibility implementation + test update next week. I'll let you know if I can use assistance on any of it!

[nateraw]

I'm excited to move towards this type of logic 😄 - here's a few comments while its still WIP.

[LysandreJik]

This is on hold as another, more robust mechanism to upload files is in the making on the backend side.

[fcakyon]

This is on hold as another, more robust mechanism to upload files is in the making on the backend side.

🤔 waiting for the new upload mechanism to integrate push to hub functionality to my packages!

[LysandreJik]

Glad to hear it @fcakyon, we will let you know.

[LysandreJik]

This looks good! Only left a few comments regarding documentation & error messages.

Very impressed by the tests! Super cool to use mocking, I believe this is the first occurrence of such tests in the codebase.

[codecov]

Codecov Report

Merging #847 (44d5941) into main (880408a) will increase coverage by 0.65%.
The diff coverage is 97.82%.

@@            Coverage Diff             @@
##             main     #847      +/-   ##
==========================================
+ Coverage   80.69%   81.34%   +0.65%     
==========================================
  Files          29       29              
  Lines        3316     3378      +62     
==========================================
+ Hits         2676     2748      +72     
+ Misses        640      630      -10     

Impacted Files	Coverage Δ
src/huggingface_hub/utils/endpoint_helpers.py	97.84% <ø> (ø)
src/huggingface_hub/hf_api.py	87.03% <91.66%> (+0.41%)	⬆️
src/huggingface_hub/hub_mixin.py	89.81% <100.00%> (+3.60%)	⬆️
src/huggingface_hub/keras_mixin.py	90.47% <100.00%> (-0.62%)	⬇️
src/huggingface_hub/utils/_deprecation.py	100.00% <100.00%> (+22.22%)	⬆️
src/huggingface_hub/file_download.py	85.35% <0.00%> (+0.27%)	⬆️
src/huggingface_hub/repository.py	80.55% <0.00%> (+0.34%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[LysandreJik]

Big effort! Let's merge this, thanks for the help and all the work improving & wrapping it up @Wauplin!

[Wauplin]

Many thanks for the help and support to finally make it @LysandreJik ! 😀

[fcakyon]

Great PR 💯