Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,19 @@ brew install act

```bash
act -j run --verbose
```

Use the verbose flag, at least the first time, since the image downloads take so long

## Run the HuggingFace update script locally via Python

```bash
export HUGGINGFACE_LOGIN_TOKEN=<xxx>
export VITE_ENV=DEV
cd src/scripts
python updateHfDataset.py
```

### Information About Technologies

- Frontend
Expand Down
11 changes: 11 additions & 0 deletions src/scripts/updateHfDataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,17 @@ def hfUpload(year=None):

login(TOKEN)
api = HfApi()

# If the current year's repo doesn't exist, create it
# the exist_ok=True flag means we'll ignore any errors if it already exists
url = api.create_repo(
repo_id=repo_id,
token=TOKEN,
private=False,
repo_type="dataset",
exist_ok=True,
)

api.upload_file(
path_or_fileobj=local_filename,
path_in_repo=dest_filename,
Expand Down