LinkSCEEM provides mechanisms for moving data between its resources as well as your local machine and LinkSCEEM resources. A number of clients are available at the HPC sites, most of which are based on GridFTP, to assist when moving data between LinkSCEEM resources. You may choose an appropriate client depending on the location and size of your transfer. Transferring data from your local client machine can be done using common Unix commands and some sites have clients to assist in moving your data to LinkSCEEM more efficiently. While Unix commands can also be used to move data between LinkSCEEM resources, it is highly recommended that GridFTP transfer methods be used for optimum performance.
In this lesson, you will learn how to transfer data:
- between your local machine and LinkSCEEM resources
- between LinkSCEEM resources
In this section we describe two methods for transfering files between your local workstation and your LinkSCEEM Resources. The first method uses Unix command lines while the second uses the WinSCP Graphical User Interface (GUI).
- Using Secure Copy (SCP)
To transfer data from a local client to LinkSCEEM resources you can use Secure Copy (SCP), the file transfer mechanism provided by the SSH protocol to send files securely over the network. SCP uses the same encrypted channel as SSH and public key encryption to authenticate and encrypt communications. Please note that SCP on most LinkSCEEM sites requires an SSH2 client. If you are trying to copy files from your local workstation using SSH1, you may receive an error message.
While SCP is easy to use, it provides very poor performance when transferring multiple files or large files. In addition, scp does not support file transfers above 2 GB so data loss may result. As a rule-of-thumb, when transferring file larger than 1 GB, GridFTP-based programs should be used for most LinkSCEEM file transfers.
Within home and scratch directories, the Unix commands mv and cp work as usual for moving and copying files.
The syntax of scp is:
Directories may be copied with a recursive option (-r). As with most Unix utilities, more information about SCP may be found from its man page (type man scp at the prompt).
For scp, the source and destination host will challenge for passphrases as necessary.
- Using WinSCP
GSI-SSHTerm has a built-in SSH File Transfer Protocol (SFTP) program you can use to move files between your local system and most LinkSCEEM resources. GSI-SSHTerm is just SCP at the core (ssh/scp that uses gsi authentication) with the added benefit of a GUI. This example uses GSI-SSHTerm, to move local files to the TeraGrid's Abe system at NCSA.
To begin an SFTP session with GSI-SSHTerm, you must first connect to a resource as you can see in SSO via the TeraGrid User Portal.
- In your GSI-SSHTerm session, select the Tools menu, and then select SFTP Session (Figure 27). (Alternatively, you can click the SFTP Session icon button on the icon bar.) A new window opens that shows the contents of your home directory on the remote resource (Figure 28).
- From the File menu, select Upload Files. The “Select files to upload” box, which contains the contents of your desktop, opens.
- Navigate to the file that you want to transfer and select it.
- Click the Upload button. Check the Remote Site window to verify that the file has been moved.
- To verify permissions associated with a file, in the Remote Site window click a file to select it and then click the Properties tab. The Properties window opens.
- Verify the file's permissions or make any changes that you desire. To implement any changes that you made, you must click the OK button when you are done. If you made no changes, you may click the OK button or the Cancel button to close the window.
- From the File menu in the GSI-SSHTerm window, select Exit. The Close Session box opens.
- Click the Yes button in the Close Session box to end your session.