Setup of Globus for File Transfers using the CHPC Data Transfer Nodes
- Create a Globus account at: https://www.globus.org/ or you can use the "alternate login" and use your institutes identity provider & authenticate appropriately.
- Log in to your newly created Globus account.
- Under "Quick Links" select "Transfer Files." You will be directed to a page with a look similar to the WinSCP interface.
- Log in to an endpoint on each side of the interface: one side being the source and the other the destination.
- Next to the "Endpoint" box, click the "..." botton for a list of all possible endpoints. CHPC currently offers several public endpoints:
uofuchpc#dtn01 (single server attached at 40gbps)
uofuchpc#dtn02 (clustered endpoint consisting of 6 servers each attached at 10gbps)
uofuchpc#dtn04 (single server connected at 10gbps)
- In most cases, users must authenticate to activate an endpoint. There are a few different possible authentication methods, but the most commonly used are either the organization authentication method or a federated authentication. The endpoints offered by CHPC use either uNID and campus password directly for authentication, or the InCommon federated authentication, which merely redirects users through the InCommon webpage to the U's CAS page to enter username and password.
- Once authenticated, choose the desired files and/or directories and click the appropriate directional arrow button to start the transfer.
- As the transfer is proceeding, click the "Activity" button and select your transfer and monitor progress. When the transfer completes, you will receive an email notifying you it is done and giving you statistics of the transfer.
Use of Globus to access CHPC Archive Storage
As an alternative to using rclone, globus is a feature-rich way to interact with the archive storage. We have created a clustered Globus endpoint specifically for PANDO. Note that the globus ceph plugin is a beta product still being developed by the globus team and therefore if you experience any issues or have any comments about the tool, please assist us and the globus team by providing feedback.
Steps to use:
- Navigate to the Globus web page.
- Log in using “existing organizational login”
- Pick “University of Utah” from the list.
- Authenticate using your campus ID and password.
- The name of the Endpoint is “University of Utah - CHPC PANDO Archive"
Globus will automatically query the archive and obtain your access and secret keys and you will be brought to the top level bucket of your portion of the archive storage.
Use of Globus with the Command Line Interface (CLI)
- Globus's Command Line Interface is maintained as a python package. It requires python 2.7+ or 3.3+ to function and relies on pip for installation. Installation instructins for Globus CLI can be found here: https://docs.globus.org/cli/installation/
- A quick guide to using Globus CLI can be found here: https://docs.globus.org/cli/#getting_started
- Examples of Command Line Interface usage can be found here: https://docs.globus.org/cli/examples/
- A reference to all Globus CLI commands can be found here: https://docs.globus.org/cli/reference/