¶ Globus Connect Overview
The Globus Connect Server provides secure access to users' files in the HPC environment at ODU and allows transfers of research data seamlessly between institutions and users. You can:
- Transfer files from your computer (e.g. laptop/workstation) to ODU HPC and vice versa.
- Transfer files between two HPC sites (or any two sites which have Globus Endpoint).
- Share data with external parties (ODU and/or non-ODU). Please see the Data Sharing with Globus documentation.
all from a convenient web interface. The Globus Connect Software provides both a web interface and a CLI interface for transferring files. For
Why using Globus for data transfer? Globus provides a more reliable way to transfer data into/out from HPC, since it does not depend merely on scp, rsync, or an SFTP client's connectivity. In the case of interruptions, Globus can retry the failed data transfer. Furthermore, Globus can in principle use much higher data transfer rates than what classic scp and friends can allow (provided your remote computer has equally high bandwidth).
¶ Globus Web Portal
The web interface for file transfers can be used by visiting app.globus.org. This will take you to the Globus Web App login form where you will need to select your identifying organization:
For an ODU faculty member or student that would be "Old Dominion University". External collaborators with ODU affiliate account can also use ODU as the organization. (Those without ODU credentials need to select their own institution or third-party organization such as "ACCESS CI (formerly XSEDE)". Access to ODU HPC data will not be possible unless explicitly shared by an ODU member. More on that on the Data Sharing with Globus page.)
After selecting the appropriate organization, click the "Continue" button.
This will momentarily take you to the MIDAS login page where you will authenticate with your MIDAS ID and password.
As the logo shows, this process will authenticate you to CILogon. After authenticating you should see the Globus web portal and the File Manager tab should be active.
¶ Accessing Data on HPC
The Collection used by ODU HPC is called "ODU RCC" and provides access to your home directory and the /RC share, respecting the access permissions of your MIDAS account. To access this collection, simply type "ODU RCC" (no quotes) on the Collection search bar in the web interface. Upon opening this collection, you will be dropped in your HPC home directory, but the long-term storage paths that begin with /RC can also be accessed. (More details in the next section)
¶ Accessible Storage
The "ODU RCC" Globus endpoint provides access to the following storage areas (the paths below are to used in the Globus File Manager, see the illustration above):
-
Your own HPC home directory, in
/isilon/home/$USER, where$USERstands for your user ID on HPC. (This corresponds to the/home/$USERpath in HPC itself.) -
The research long-term storage, in
/isilon/RC/homeand/isilon/RC/group. Any subfolders under/RCthat you can access in the HPC environment will also be accessible from Globus. This includes your long-term user directory (/RC/home/$USER), as well as shared group directories under/RC/group(if you have access to the directories).
Due to technical limitations, at this moment the scratch partitions cannot be accessed from Globus.
Non-ODU users who do not sign up with ODU credentials will not be able to access ODU HPC files in this way. For them, please contact your collaborator to set up a shared folder, as described in the Data Sharing with Globus page.
¶ Whose Files Can I See?
With Globus, you can only see the contents of your home directory, and not anyone else's directories. This is by design for security reasons. If you need to collaborate on files via the web interface, there are several ways to do this:
-
Your group have a
/RC/groupworkspace (directory tree) that has been set up by RCS administrator, and your group members have been added to this workspace. -
You can use Globus data sharing facility. Please also see https://forum.hpc.odu.edu/tag/data-sharing for an idea.
¶ File Transfers
File transfers can be initiated using Globus' web-based File Manager app. Transfers in the File Manager are done between two collections, which are abstractions representing data sources which can be copied to or from.
For a step-by-step tutorial for Globus data transfers, please visit: https://docs.globus.org/how-to/get-started/#request_a_file_transfer.
If you need to transfer files between a personal machine and a collection, there is a client application (Globus Connect Personal) that can be used for reliable data transfer. Please read the details here: https://docs.globus.org/how-to/get-started/#move_beyond_getting_started.
We strongly encourage you to contact us (the RCS team) to obtain assistance in this matter.