Provide the ETag header

[severo]

• set and manage the ETag header to save bandwidth when the client (browser) revalidates. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching and https://gist.github.com/timheap/1f4d9284e4f4d4f545439577c0ca6300

      # TODO: use key for ETag? It will need to be serialized
      # key = get_rows_json.__cache_key__(
      #     dataset=dataset, config=config, split=split, num_rows=num_rows, token=request.user.token
      # )
      # print(f"key={key} in cache: {cache.__contains__(key)}")

• ETag: add an ETag header in the response (hash of the response)
• ETag: if the request contains the If-None-Match, parse its ETag (beware the "weak" ETags), compare to the cache, and return an empty 304 response if the cache is fresh (with or without changing the TTL), or 200 with content if it has changed

[severo]

Note that the /assets are served by nginx and have an eTag, eg https://datasets-server.us.dev.moon.huggingface.tech/assets/mnist/--/mnist/train/0/image/image.jpg

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[severo]

The etag could be a hash of the response + the git version of the dataset repo + the datasets library version (or, better, the worker version). Related to #545

[severo]

Moving to the internal tracker