Reduce memory usage and load time by skipping costly fingerprint generation

[Irozuku]

Summary

This PR addresses excessive RAM usage and delays when uploading datasets by optimizing the initialization of DashAIDataset.

Problem

DashAIDataset inherits from HuggingFace’s Dataset class, whose constructor performs an expensive fingerprint generation step (computing a unique identifier for the dataset).
This step significantly increases both initialization time and memory usage, even though the fingerprint is not required for our custom dataset saving/loading workflow.

Fix

The solution is to manually generate a lightweight fingerprint and pass it to the super() call in DashAIDataset.
This bypasses the costly default computation, reducing memory usage and improving dataset loading speed.

Notes

The fingerprint is only used for faster reloads when using HuggingFace’s save_dataset / load_dataset APIs, which we do not use.
Therefore, this change does not affect functionality or reproducibility within our system.

[Copilot]

Pull Request Overview

This PR optimizes DashAIDataset initialization by manually generating lightweight fingerprints to avoid expensive computation performed by HuggingFace's Dataset constructor. This reduces memory usage and improves load time during dataset uploads.

• Manual fingerprint generation using table metadata and UUID to bypass costly default computation
• Addition of uuid import to support fingerprint creation
• Optimization targets the initialization bottleneck without affecting functionality

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The fingerprint generation combines a deterministic hash with a random UUID, which could create inconsistency. If the goal is to bypass expensive computation while maintaining some determinism for caching, consider using only the hash of table metadata without the random UUID component.

Suggested change

	fingerprint = (
	f"manual-{hash((table.num_rows, str(table.schema)))}-{str(uuid.uuid4())}"
	)
	fingerprint = f"manual-{hash((table.num_rows, str(table.schema)))}"