Automatically obtain organization in keras push_to_hub

[osanseviero]

If I do this, my model gets pushed to osanseviero/test-osan6

push_to_hub_keras(model, "keras-io/test-osan6")

It would be great if we can automatically detect the organization. We can likely do it similar to https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/keras_mixin.py#L145-L146.

[osanseviero]

Full example

!pip install huggingface_hub
!huggingface-cli login

!apt-get install git-lfs
!git-lfs install

import tensorflow as tf
import huggingface_hub

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(2, activation="relu"))
model.compile(optimizer="adam", loss="mse")
model.build((None, 2))

from huggingface_hub import push_to_hub_keras

# Goal: this should be pushed to keras-io org
push_to_hub_keras(model, "keras-io/example")

[osanseviero]

This might not be possible at all as discussed in Discord with @cakiki since the argument can be either the name (just name, no namespace) of a remote model repo in the Hub or a local path.

[osanseviero]

As discussed internally, maybe we can have a deprecation that will simplify the usage to something more intuitive for users

Actual

def push_to_hub_keras(
    model,
    repo_path_or_name: Optional[str] = None,
    repo_url: Optional[str] = None,
    commit_message: Optional[str] = "Add model",
    organization: Optional[str] = None,
    private: Optional[bool] = None,
    api_endpoint: Optional[str] = None,
    use_auth_token: Optional[Union[bool, str]] = True,
    git_user: Optional[str] = None,
    git_email: Optional[str] = None,
    config: Optional[dict] = None,
    include_optimizer: Optional[bool] = False,
    **model_save_kwargs,
):

Maybe this would be a good alternative

def push_to_hub_keras(
    model,
    repo_id: str,
    commit_message: Optional[str] = "Add model",
    private: Optional[bool] = None,
    use_auth_token: Optional[Union[bool, str]] = True,
    config: Optional[dict] = None,
    include_optimizer: Optional[bool] = False,
    **model_save_kwargs,
):

WDYT @merveenoyan @LysandreJik @nateraw ?

[LysandreJik]

I think that clarifies things significantly. An unfortunate behavior linked to this is that it hides the fact that there's a clone/pull/push happening behind the scenes. Except it's not really behind the scenes, as IIRC it takes place in the current working directory.

I think ideally this would work with http requests that push files (#321), but these files can only be of 20GB.

Maybe 20GB is more than enough for what we're looking for, however.

[Wauplin]

Related to #846 and solved in #847 . I'm closing this issue as well as git-based method is not the way to go anymore.

[osanseviero]

Yes let's close this