-
Notifications
You must be signed in to change notification settings - Fork 0
Using Box for Large File Storage
Until June 2026, large files for projects were stored in repos. in order to store the files this way the lab used a service called git LFS. However the university enterprise github account only allows for 250GB of lfs storage among all repos, and our lab was using most of that, and so we are migrating our large files to Box.
Download Box drive here (hyperlink)
Make sure that your box drive is in this folder:
~/Library/CloudStorage/Box-Box
You will also need access to the lab box folder gnlab
Inside this folder there is a replica of the structure of your repo directory. However, the only files contained in this box directory are large files.
In template_repo there is a directory called box_relocation. Inside that repo there are 3 files. box_relocation_files.csv is a csv with both the box path of all large files stored in box as well as the local paths where links that point to the files should be. symlink_executor_bash.sh is a bash script which will create a symlink at a local file location pointing to the box location. run_box_relocations.py is a python script which runs the bash script, creating a symlink for every file which has been relocated to Box.
In your terminal, run python3 ~/repo/template_repo/box_relocation/run_box_relocations.py
This should create all necessary symlinks for all repos you have cloned onto your local device.
In order to add a new file to Box, and replace the file in the GitHub repo, do the following:
Navigate to the directory in which te large file is in.
Run the following command: git rm --cached FILE_NAME.x
Then run echo "FILE_NAME.x" >> .gitignore \n git add .gitignore
Replacing FILE_NAME with the name of your file, and x with the type of your file (likely .csv).
Then add the location of the local file, as well as the path to the same file on Box to box_relocation_files.csv. After you have added the file, re-run run_box_relocations.py (expect the row which you added to take longer because the bash script compares the local file to the Box file to ensure that they are identical).
Running the python script should delete the large data file.
Lastly, commit your changes.
Starting at gnlab
- Goals, Norms, Rhythms, and Professional Dev't
- Analyst, not a coder and common challenges
- For UChicago Undergraduates
Advice for doing research
Code and computation
- IT setup guide
- Code style guide
- RCC guide
- IT platform guide
- Logging: Best Practices
- Consumption model guide
- Simulation techniques
- Bootstrap
- Using Box for Large File Storage
Github and git
- Task and Code Management Guide
- Pull Requests and Code Reviews
- git hooks guide
- Shortcuts for git (and other) commands
Exhibits
Producing papers
- Writing style guide
- Replication kit guide
- Paper production guide
- Citation guide
- AEA Submission guide
- Overleaf guide
Professional development and career rhythms
- Professional Development and PhD Application guide
- Offboarding
- Exit interview questions
- Questions you might get asked at an interview
- Questions to ask at a PhD admissions visit
- Recommendation letters (non‐PhD)
Miscellaneous
Legacy