feat(hugginface): allow specifying huggingface endpoint

[kszucs]

Rationale for this change

Huggingface Private Hub may use custom endpoint, it is also useful for enabling open source projects to use https://hub-ci.huggingface.co for testing.

What changes are included in this PR?

Add support for configuring huggingface endpoint. If not provided, it tries to read from HF_ENDPOINT environment variable.

It also fixes a bug when passing revision which must be encoded, like refs/convert/parquet should be encoded as refs%2Fconvert%2Fparquet.

Are there any user-facing changes?

There is a new builder method called .endpoint().

[kszucs]

The produced URLs haven't been tested so I would need to mock the http client the assert on the created URLs, do you have an already established approach for that?

[Xuanwo]

The produced URLs haven't been tested so I would need to mock the http client the assert on the created URLs, do you have an already established approach for that?

I think this change is simple enough to merge directly. Let's go.

[Xuanwo]

Thank you for working on this!

[Xuanwo]

it is also useful for enabling open source projects to use https://hub-ci.huggingface.co for testing.

We can set up another integration test for https://hub-ci.huggingface.co at https://github.com/apache/opendal/tree/main/.github/services/huggingface/huggingface.

Is this endpoint available for registration? I can help set up this test infra.

[kszucs]

Thanks @Xuanwo for reviewing and merging!

it is also useful for enabling open source projects to use https://hub-ci.huggingface.co for testing.

We can set up another integration test for https://hub-ci.huggingface.co at https://github.com/apache/opendal/tree/main/.github/services/huggingface/huggingface.

Is this endpoint available for registration? I can help set up this test infra.

Yes. @lhoestq can provide more context.