add HuggingFace streaming support in data input pipeline

[aireenmei]

No description provided.

[aireenmei]

Perf testing

• No v5p-128 capacity. Tested on v4-128 with per_device_batch_size=1 (bigger batch size cause OOM).
• Since with HF streaming, we apply caption encoding and vae on the fly. The baseline is the original "make_pokemon_iterator" but with these operations on the fly. On v4-128 with small batch_size, HF streaming a larger dataset (I use BleachNick/UltraEdit_500k as example here, other datasets of better resolution need custom preprocessing for cleaning/reformat caption, so I choose BleachNick/UltraEdit_500k as the quickstart example in the doc) has comparable step time vs baseline.
• baseline config: dataset_type=tf cache_latents_text_encoder_outputs=False dataset_name=diffusers/pokemon-gpt4-captions train_split=split
• baseline cloud logging: https://cloudlogging.app.goo.gl/gqvkgydmkak7UfJW6
• HF streaming config: dataset_type=hf dataset_name=BleachNick/UltraEdit_500k image_column=source_image caption_column=source_caption train_split=FreeForm
• HF streaming cloud logging: https://cloudlogging.app.goo.gl/xjihHuCPAAbQmu3s6