Merge pull request #274 from LRFreeborn/dtn23_updates

Dtn23 updates

docs/additional-resources/registrycilogon-instructions.md

@@ -0,0 +1,77 @@
+## Uploading an SSH Key to CILogon Registry  
+
+CU Boulder Research Computing uses [CILogon](https://cilogon.org) to manage key-based login (CU Anschutz users, only) and passwordless data transfers with `scp` and `rsync` (CU Boulder and AMC, only). Access to the [RMACC CILogon Registry](https://registry.cilogon.org/registry/) is available by invitation only.
+
+### *Step 1: Enroll in the RMACC CILogon Registry*
+
+Request an invitation to the RMACC CILogon Registry by emailing <rc-help@colorado.edu> and explaining your intended use case (e.g., automated data transfers). After your case has been assigned, you will receive an email from <registry@cilogon.org> inviting you to enroll in the RMACC Registry. 
+<br>
+
+![](./registry-images/email_invitation.png)
+
+Follow the invitation URL and click 'Accept'. __The invitation link becomes invalid once you click 'Accept', so complete all of Step 1 before closing your browser tab.__ 
+
+![](./registry-images/accept_invitation.png)
+
+You will be automatically routed to the login page. If you access CURC resources with a CU Boulder account, select 'University of Colorado at Boulder' from the Identity Provider dropdown menu and click 'Log On'. 
+If you access CURC resources with an ACCESS account, select 'ACCESS CI (XSEDE)' from the dropdown menu.
+<br>
+![](./registry-images/cu-boulder-dropdown.png)
+
+
+Enter your CU Boulder or ACCESS username and password and click 'Login'. Accept the Duo push from your device.
+
+> **_IMPORTANT:_** You can move on to Step 2, but please make sure 10 minutes have elapsed between completing Step 1 and beginning Step 3.
+
+### *Step 2: Generate an ssh key*
+
+You are strongly encouraged to set a passphrase for your key pair. You will be prompted to enter the passphrase each time you log in. 
+
+SSH Key Generation for Windows Users: 
+- Using command prompt, PowerShell, or Windows Terminal: https://www.howtogeek.com/762863/how-to-generate-ssh-keys-in-windows-10-and-windows-11/
+- Using PuTTY: https://devops.ionos.com/tutorials/use-ssh-keys-with-putty-on-windows/
+
+SSH Key Generation for Mac Users: 
+https://docs.tritondatacenter.com/public-cloud/getting-started/ssh-keys/generating-an-ssh-key-manually/manually-generating-your-ssh-key-in-mac-os-x
+
+
+### *Step 3: Upload your ssh key to [registry.cilogon.org](https://registry.cilogon.org/registry/)*
+
+After logging into [registry.cilogon.org](https://registry.cilogon.org/registry/) with your ACCESS 
+credentials, click on the dropdown menu in the upper right hand corner by your name. 
+Select 'MY PROFILE (RMACC)'.
+
+Select 'Authenticators' from the menu on the right. 
+<br>
+
+![](./registry-images/menu_options.png)
+<br>
+
+Click 'Manage' in the SSHKeyAuthenticator row. 
+![](./registry-images/manage_sshkeyauthenticator.png)
+
+Select 'Add SSH Key'.
+![](./registry-images/add_sshkey.png)
+
+Click 'Choose File'. If you store your SSH keys in a hidden directory (e.g. `~/.ssh`), it may be difficult to locate your public key using a Finder/File Explorer window. As a workaround, you can copy your public key to an easily discoverable location using the Terminal App/Windows Command Prompt: <br>`cp ~/.ssh/id_rsa.pub ~/Desktop`
+<br>
+
+Locate your __public__ key (`<keyname>.pub`) on your local drive, then click 'UPLOAD'.
+![](./registry-images/upload_sshkey.png)
+
+<br>
+
+A green message box will notify you that your SSH key was successfully added to your account.
+<br>
+![](./registry-images/sshkeyadded.png)
+
+### Deleting or Replacing SSH Keys in RMACC CILogon Registry
+
+Log in at [registry.cilogon.org](https://registry.cilogon.org/registry/) and navigate to the 'Authenticators' section of your profile (see Step 3, above). Click 'Delete', then repeat the process for adding a new key. It will take a few minutes before you can log in with your new key.
+
+### Troubleshooting
+
+* If you receive an error message indicating that you are not in the COmanage registry (see screenshot below), please email <rc-help@colorado.edu>. Please include a screenshot of the error message and the date/time of your last login attempt.
+
+![](./registry-images/notregistered_error.png)
+<br>

docs/compute/data-transfer.md

@@ -3,9 +3,7 @@
 Research Computing supports several methods of file transfer. File
 transfers from a local system can be done through a web-based
 application called _Globus_ or through command-line tools such as
-secure copy (_scp_), secure ftp (_sftp_) and _rsync_. Some
-command-line tools may be unavailable on Windows, though alternative
-applications exist. (e.g., WinSCP, FileZilla)
+secure copy (_scp_), secure ftp (_sftp_) and _rsync_. 
 
 Data transfers using SSH protocols can be done via CURC login nodes or
 through the [CURC data transfer nodes](https://curc.readthedocs.io/en/latest/compute/node-types.html) (DTN). Transfers via the login
@@ -93,7 +91,7 @@ RC Users also have the option of connecting to RC via _Filezilla_. _Filezilla_ i
       - Username: your-rc-username
       - Password: your-rc-password
     - _Data Transfer Node_
-      - Host: sftp://dtn.rc.int.colorado.edu
+      - Host: sftp://dtn23.rc.colorado.edu
       - Username: your-rc-username
       - Password: your-rc-password
 
@@ -120,14 +118,14 @@ would like to send the file to.
 # Copying files from a local workstation to Research Computing
 
 scp <path-to-file> <username>@login.rc.colorado.edu:<target-path>    # using a login node
-scp <path-to-file> <username>@dtn.rc.int.colorado.edu:<target-path>  # using DTN while on campus network
+scp <path-to-file> <username>@dtn23.rc.colorado.edu:<target-path>    # using DTN while on campus network
 ```
 
 ```bash
 # Copying files from Research Computing to a local workstation
 
 scp <username>@login.rc.colorado.edu:<path-to-file> <target-path>    # using a login node
-scp <username>@dtn.rc.int.colorado.edu:<path-to-file> <target-path>  # using DTN while on campus network
+scp <username>@dtn23.rc.colorado.edu:<path-to-file> <target-path>    # using DTN while on campus network
 ``` 
 
 Windows users can access scp through PowerShell or using a GUI
@@ -157,14 +155,14 @@ would like to send the file to.
 # Synchronizing from a local workstation to Research Computing
 
 rsync -r <path-to-directory> <username>@login.rc.colorado.edu:<target-path>    # using a login node
-rsync -r <path-to-directory> <username>@dtn.rc.int.colorado.edu:<target-path>  # using DTN while on campus network
+rsync -r <path-to-directory> <username>@dtn23.rc.colorado.edu:<target-path>    # using DTN while on campus network
 ```
 
 ```bash
 # Synchronizing from Research Computing to a local workstation
 
 rsync -r <username>@login.rc.colorado.edu:<path-to-directory> <target-path>    # using a login node
-rsync -r <username>@dtn.rc.int.colorado.edu:<path-to-directory> <target-path>  # using DTN while on campus network
+rsync -r <username>@dtn23.rc.colorado.edu:<path-to-directory> <target-path>    # using DTN while on campus network
 ```
 
 rsync is not available on Windows by default, but [may be installed
@@ -186,7 +184,7 @@ between a local system and Research Computing resources.
 
 ```bash
 sftp <username>@login.rc.colorado.edu    # using a login node
-sftp <username>@dtn.rc.int.colorado.edu  # using DTN while on campus network
+sftp <username>@dtn23.rc.colorado.edu    # using DTN while on campus network
 ```
 
 We can then use various commands to traverse and manipulate both local
@@ -214,24 +212,19 @@ resources](#more-reading) or consult the sftp manual page.
 
 ### Automated scp and rsync
 
+Key-based transfers over the DTNs are only available to CU Boulder, CU Anschutz, and CU Denver users. We do not permit key-based transfers over the login nodes.
+
+*You must be on a CU (UCB/AMC/UCD) campus network or logged into the campus VPN to perform passwordless data transfers to CURC.*
+
 The `scp` and `rsync` commands both allow a user to transfer files
 without needing to reenter a password. All that is required is a few
-simple set up procedures to prepare your local machine.
+simple setup procedures to prepare your local machine.
 
 *These instructions only apply to local macOS and Linux
  systems. Automating file transfers from Windows is outside of the
  scope of this document.*
 
-*You must be on a campus network or [logged into the campus
- VPN](https://oit.colorado.edu/services/network-internet-services/vpn)
- to perform passwordless data transfers to CURC.*
-
-
-#### Generate a local ssh key pair
-
-You only need to perform this step once per local system.
-
-From a local terminal run:
+1. Generate a local ssh key pair. You will only need to perform this once per local system. From a local terminal run:
 
 ```bash
 ssh-keygen -t ed25519
@@ -243,41 +236,18 @@ is public, and can be shared with others (including Research
 Computing). `id_ed25519` is private and **should never leave the
 system that it was generated on.**
 
+2. Follow the instructions [here](https://curc.readthedocs.io/en/latest/additional-resources/registrycilogon-instructions.html) for requesting access to the CILogon Registry and uploading an ssh key.
 
-#### Copy the public key to CURC
-
-You only need to perform this step once per local system.
+3. Use `rsync` or `scp` to transfer files using an ssh key. 
 
-From a local terminal run:
+With an ssh key pair generated and uploaded to <https://registry.cilogon.org>,
+you are ready to transfer files over the DTNs using `rsync` or `scp`. 
 
 ```bash
-cat ~/.ssh/id_ed25519.pub | ssh <your-username>@login.rc.colorado.edu -T "cat >> ~/.ssh/authorized_keys"
-```
-
-Substitute your Research Computing username for `<your-username>`.
+rsync -av ./myfile.txt dtn23.rc.colorado.edu:/projects/ralphie/myfile.txt    # using rsync
 
-You will be required to enter your password and accept a Duo push in
-order to transfer the key.
-
-*If you have trouble running the command above, you can also just
- login to a CURC login node, open `~/.ssh/authorized_keys` and paste
- the text from `~/.ssh/id_ed25519.pub` that resides on your local
- machine.*
-
-
-#### Use `rsync` or `scp` to transfer files using an ssh key
-
-With an ssh key pair generated and configured in Research Computing,
-you are ready to transfer files. Key-based transfers are only
-supported via DTN. Make sure you are within the CU network and
-transfer with scp or rsync.
-
-```bash
-rsync -av ./myfile.txt dtn.rc.int.colorado.edu:/projects/ralphie/myfile.txt  # using rsync
-
-scp -v ./myfile.txt dtn.rc.int.colorado.edu:/pl/active/crdds/myfile.txt      # using scp
+scp -v ./myfile23.txt dtn23.rc.colorado.edu:/pl/active/crdds/myfile.txt      # using scp
 ```
-
 ---
 ### Rclone