Jump to secure copy (scp) <scp-jump>
Jump to rsync <rsync-jump>
Jump to secure file transfer protocol (sftp) <sftp-jump>
Jump to Globus <globus-jump>
scp is a command line interface (CLI) application that provides a secure way to copy files and directories between machines over an unsecured network. Use scp
for small to modest transfers to avoid impacting usability of a system's login node.
scp
requires a source and a destination; these are specified with a file path if it is on your local machine or as <username>@<hostname>:<file_name>
if it is on a remote machine.
scp <options> <username>@<source_hostname>:<source_file_path> <username>@<destination_hostname>:<destination_file_path>
- scp <options> <https://man.openbsd.org/scp.1>
- Delta node hostnames
- Hydro node hostname
- ICC DTN node hostname
- Nightingale node hostnames
scp <options> <source_file_name> <username>@<hostname>:<destination_path>
## ICC example:
## testuser1 transfers a file ("local_file") from their
## local machine to the their home directory on the Campus Cluster
[testuser1_machine] ~ % scp local_file testuser1@cc-xfer.campuscluster.illinois.edu:~/
scp <options> <username>@<hostname>:<source_file_path> <destination_path>
## ICC example:
## testuser1 transfers a file ("remote_file") from their
## home directory on the Campus Cluster to their local machine
[testuser1_machine] ~ % scp testuser1@cc-xfer.campuscluster.illinois.edu:~/remote_file ./
rsync is a CLI utility that syncs files and directories. Use rsync
for small to modest transfers to avoid impacting usability of a system's login node.
rsync
requires a source and a destination; these are specified with a file path if it is on your local machine or as <username>@<hostname>:<file_name>
if it is on a remote machine.
rsync <options> <username>@<source_hostname>:<source_file_path> <username>@<destination_hostname>:<destination_file_path>
- rsync <options> <https://download.samba.org/pub/rsync/rsync.1#OPTION_SUMMARY>
- Delta node hostnames
- Hydro node hostname
- ICC DTN node hostname
- Nightingale node hostnames
rsync <options> <source_file_name> <username>@<hostname>:<destination_path>
## ICC example:
## testuser 1 transfers the "images" directory from their
## local machine to a projects directory on the Campus Cluster
[testuser1_machine] ~ % ls
images
[testuser1_machine] ~ % rsync -avP images testuser1@cc-xfer.campuscluster.illinois.edu:/projects/$teams_directory/
rsync <options> <username>@<hostname>:<source_file_path> <destination_path>
You can transfer data using sftp
via the command line or one of many common transfer utilities. Two transfer utility options, WinSCP and Cyberduck, are described below; both are free to download and install.
- Download and install WinSCP.
Open WinSCP and log into the associated NCSA system node.
- File protocol: SFTP
- Host name:
- Port number: 22
- User name: Your username for the associated NCSA system.
- Password: Your password for the associated NCSA system.
ICC example:
- Once you're logged in, WinSCP works like a drag and drop interface for moving files between your local machine and remote machine connection.
- Download and install Cyberduck.
Open Cyberduck and click the Open Connection button in the upper left corner.
Connect to the associated NCSA system node.
- Select SFTP in the drop-down menu.
- Server:
- Port: 22
- Username: Your username for the associated NCSA system.
- Password: Your password for the associated NCSA system.
ICC example:
- Once connected, you should see a listing of your home directory, and you can navigate the file system via the GUI. Download and upload files, as needed.
Globus is a web-based file transfer system that works in the background to move files between computer systems with Globus endpoints. Globus is a good tool to use to transfer many files or large files between directories (within the same system or between a systems).
Note
If you are new to Globus, the Globus log in and transfer files tutorial includes step-by-step instructions for transferring files that you can follow along with using their built-in demonstration collections.
The NCSA systems listed have Globus endpoints configured by the system administrators; the collection names for these endpoints are at the links below. To transfer data to/from a system that does not have a Globus endpoint, see globus_connect_personal
.
If you have issues using Globus, review the resources on the Globus Contact Us page.
Install Globus Connect Personal to transfer files between a Globus endpoint and a system that does not have an existing Globus endpoint (a personal laptop, for example). On Nightingale, the protected data requirements still apply.
- When you select a Globus endpoint for the first time, you may see an Authentication/Consent Required prompt (Nightingale example shown below).
- Reference the Globus link an identity tutorial for instructions on how to proactively link an identity to your Globus account. Consider linking your UIUC, NCSA, and/or ACCESS identity, as applicable, depending on the compute resource(s) you're using and how you created your Globus account.
- After you have navigated to an endpoint using the Collection search, entering a forward slash ( / ) into the Path field displays the top-level directories you have access to at that endpoint. From there you can navigate to the location you want to transfer to/from. (You can also enter the direct file path into the Path field.)