LOLO Hyak

Here's some information on Lolo storage:

The Archive filesystem is intended for long term storage of data that changes infrequently, if ever. It is a tape-based system with a small disk buffer to improve write performance. It should NEVER be used for ANY sort of interactive access. It is equally important that you NEVER store thousands of SMALL FILES. Instead, collect directories of small files into tar files BEFORE transferring them to the Archive. The Archive quota limits you to 10.000 files/TB - an average file size of 100MB. This means if you store even a few tiny files, the remaining files must be very large - typically several GB in size.

The filesystem will perform best when used with large (> 1MB) files. For this and other reasons, we strongly encourage you to use tar to combine collections of small files into tar files before transferring them to the Archive filesystem for safekeeping. Be sure to read the section on Quotas below: limits are in place on the capacity each group can use and on the number of files.

Files written to the archive are cached on disk before being migrated to tape. Once migrated, a backup is made on tapes stored in a separate data center. Filesystem metadata is always maintained on disk, allowing a consistent view of the filesystem regardless of where the data blocks reside (i.e. 'ls -l' will list your files the same way whether they're in disk or tape). Copy data to or from the archive using the same commands as you for any other filesystem. For the Archive to function well, however, there are some important guidelines:

The Archive should NEVER be used for ANY sort of interactive access

This means it's OK to recall the files associated with a project a few months or years after copying them to the Archive. It is NOT OK to recall the same files from the Archive on a weekly basis. When interacting with the Archive filesystem it is very important to avoid unnecessarily recalling files from tape. In most cases, opening an archived file requires recalling it from tape. This triggers a sequence of events:

The tape library fetches the appropriate tape and loads it into a tape drive (< 2 minutes)

The tape drive seeks to the location of your file (< 2 minutes)

The tape drive begins streaming your file to the disk cache (> 100MBs for large files) As you can see, this is a heavyweight operation and should be avoided when possible. For example, recalling a file simply for appending is almost never a good thing. Instead, we encourage everyone to make "whole file" copies (use the "W" or "-whole-file" flag with rsync). This will replace the earlier version of the file with the new version without triggering a recall from tape. For example, to update your archive of a very important directory from your laptop:

   $ rsync -av --whole-file ImportantDir lolo.uw.edu:/archive/hyak/GROUPDIR/
Where GROUPDIR is the name of your Hyak group.

Anytime you use rsync with the Archive filesystem it is very important to use the "--whole-file" argument to prevent rsync from recalling previous versions of each file from tape for comparison. You can rerun this command anytime you want to update the archive copy. It will not delete any files from the archive and will replace any previously archived versions of files with the updated versions.

Files deleted from the Archive persist on tape for some time. This is accomplished by two means. First, it is an (nearly) unavoidable consequence of working with tape. When you issue a delete command, the filesystem metadata is updated to reflect the new state of the file (deleted), but the blocks occupied by that file on tape are not immediately reclaimed. Periodically, an automatic reclamation process transfers data from tapes with deleted file "holes" to new tapes, returning the tapes with the deleted file blocks to the empty tape pool for reuse. Second, we back up the migration tape copy to a second tape copy. The backup retention policy is to preserve all files for at least one month beyond their date of deletion. This means you can be confident that when you replace a file in-place (as with the rsync example, above), the previous version of the file will be recoverable if something goes wrong.

Be aware of the quota on the number of files you can store in the Archive: 10,000 files per TB. This works out to an average file size of 100MB If you are using the Archive to store directories containing many small files, consider using tar to combine these files into one big file before transferring them to the Archive. Not only will this help manage your quota, but large files ensure much better tape drive performance, resulting in quicker recall of your data when you need it.

Stergachis Lab has disk space on lolo at /archive/stergachis-lab where stergachis-lab is our group name.

• How can I find out what files I have already got on lolo?

Below abc is your userid.

ssh abc@lolo.uw.edu
cd /archive/stergachis-lab/backups/
ls

• How can I copy files from hyak to lolo and from lolo to hyak?

Below abc is your userid.

Files on /archive/stergachis-lab should be 100MB or larger. You can use tar to create a single tar file from a directory and then use sftp or scp to copy the tar file to lolo.

Suppose the files that you want to transfer are in the directory mydir. Use below command to tar the files:

tar -cvf mydir.tar mydir

1. Method 1 sftp Use below commands from the hyak login node to sftp a tar file from hyak to lolo (abc is your userid).

sftp abc@lolo.uw.edu
cd /archive/stergachis-lab/[target-directory]
put /gscratch/stergachislab/[data-folder]/mydir.tar 
quit

If you want to get a file from lolo to hyak then use "get mydir.tar" instead of "put mydir.tar" in above steps.

2. Method 2 scp

Use below commands from the hyak login node to scp a tar file from hyak to lolo.

scp /gscratch/stergachislab/[data-directory]/mydir.tar abc@lolo.uw.edu:/archive/stergachis-lab/[target-directory]  

If you want to get a file from lolo to hyak then use below commands from the hyak login node as shown below:

scp abc@lolo.uw.edu:/archive/stergachis-lab/[data-directory]/mydir.tar /gscratch/stergachislab/[target-directory]/

• Can I delete files from tape to make more space? Yes

* How do I know how much space on tape a directory is taking up?
You must use the apparent size:

du -sh --apparent-size