Add utility to send telemetry

[Wauplin]

Related to internal discussion related to Gradio telemetry. Idea is to centralize telemetry across HF libraries in a single helper. Gradio could be a first step (cc @abidlabs @freddyaboulton @dawoodkhan82) , but it can also replace implementation from transformers and diffusers (cc @sgugger @patrickvonplaten, even though this is not a priority).

Telemetry is sent to https://huggingface.co/api/telemetry/whatever/path using a HEAD request. Headers contain a user_agent with library name/version + some stuff (hf_hub version, python version, pytorch/tf/...,). Additional data can be also provided in the user_agent, either with a string or with a dictionary. Entries are de-duplicated from the user_agent before being sent to the Hub.

Since telemetry is a bit sensible, it will be at the discretion of each library to decide when/where to put telemetry. There is always an opt-out option for the users to disable it globally with HF_HUB_DISABLE_TELEMETRY (or HF_HUB_OFFLINE which has other side effects). DISABLE_TELEMETRY is also supported as legacy for transformers. (We can also add helpers like disable_telemetry/enable_telemetry to programmatically enable/disable it we you think it's useful).

Example:

>>> from huggingface_hub.utils import send_telemetry

# Send telemetry without library information
>>> send_telemetry("ping")

# Send telemetry to subtopic with library information
>>> send_telemetry("gradio/local_link", library_name="gradio", library_version="3.22.1")

# Send telemetry with additional data
>>> send_telemetry(
...     topic="examples",
...     library_name="transformers",
...     library_version="4.26.0",
...     user_agent={"pipeline": "text_classification", "framework": "flax"},
... )

TODO:

• send_telemetry
• unit tests
• docs in package reference section
• docs in environment variables sections
• (optional) run telemetry in a separate thread to avoid blocking the main thread in case of slow connections

cc @julien-c @LysandreJik who brought up the topic :)

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[codecov]

Codecov Report

Base: 49.56% // Head: 83.99% // Increases project coverage by +34.42% 🎉

Coverage data is based on head (57d23ee) compared to base (a7f7702).
Patch coverage: 96.15% of modified lines in pull request are covered.

❗ Current head 57d23ee differs from pull request most recent head 8e62a5f. Consider uploading reports for the commit 8e62a5f to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1344       +/-   ##
===========================================
+ Coverage   49.56%   83.99%   +34.42%     
===========================================
  Files          47       48        +1     
  Lines        4788     4791        +3     
===========================================
+ Hits         2373     4024     +1651     
+ Misses       2415      767     -1648     

Impacted Files	Coverage Δ
src/huggingface_hub/utils/_runtime.py	56.45% <0.00%> (+4.41%)	⬆️
src/huggingface_hub/constants.py	94.23% <100.00%> (+0.11%)	⬆️
src/huggingface_hub/utils/__init__.py	100.00% <100.00%> (ø)
src/huggingface_hub/utils/_headers.py	100.00% <100.00%> (+17.30%)	⬆️
src/huggingface_hub/utils/_telemetry.py	100.00% <100.00%> (ø)
src/huggingface_hub/__init__.py	75.75% <0.00%> (ø)
src/huggingface_hub/utils/_fixes.py	85.71% <0.00%> (+2.85%)	⬆️
src/huggingface_hub/utils/_subprocess.py	100.00% <0.00%> (+4.54%)	⬆️
src/huggingface_hub/utils/_chunk_utils.py	100.00% <0.00%> (+7.14%)	⬆️
... and 29 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[julien-c]

really nice implem!

EDIT: raised a question on Slack

[sgugger]

Looks very nice, thanks!

[dawoodkhan82]

@Wauplin any update on when we can expect this to be merged?

[Wauplin]

@dawoodkhan82 It's tested and reviewed so let's merge it now :)

But I guess your question is more, "when will it be released"? For this question I'm not sure yet. Except if it's urgent I don't think I'll make a new release of huggingface_hub in the next 2 weeks. Would that be ok for you? Open to discussion of course :)

[dawoodkhan82]

@Wauplin Not urgent, but would be nice if it was released sooner. It would help with fully migrating away from segment sooner (which bills us monthly).

[julien-c]

@dawoodkhan82 but are you going to pin the latest huggingface_hub in gradio then? (just curious)

[dawoodkhan82]

@julien-c Yeah that's the plan. I think we should also collect analytics using both segment + huggingface_hub for the first 2 weeks or so to make sure the transition is smooth + there's no discrepancies. Shouldn't be an issue right? @abidlabs

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.