Data Transfer for CHESS Users
Data collected at most CHESS beamlines are written to a centralized data acquisition and storage system known as CHESS-DAQ
. This system consists of a dedicated high-speed network that connects the experimental stations to hundreds of terabytes of enterprise-class redundant disk arrays, as well as an offsite magnetic tape library for long-term archival.
Remote Data Access
CHESS users are encouraged to download their data directly from the CHESS-DAQ
to their own computers. This data transfer can be done while onsite at CHESS (using one's laptop) or back at one's home institution.
For both access methods below, CHESS users will need to obtain a CLASSE account. Please contact the CHESS Users' Office or a staff scientist for assistance.
Method 1 (preferred): Globus
) is our recommended data transfer tool because it is optimized for large data volumes, and it works on any platform. To use it, users simply install Globus Connect Personal (free software) on their computers and then log in to a CLASSE Globus endpoint with their CLASSE password.
- CHESS users are strongly advised to test Globus data transfers before arriving onsite, using the CHESS Test endpoint.
- CHESS users may access their data during and after their beamtime using the CHESS Raw and CHESS Aux endpoints.
- Please report any problems by opening a CLASSE-IT Service Request.
Method 2: Remote Login
CHESS users with a CLASSE account can log into CLASSE's general-purpose Linux login node,
- Linux and Mac: please see RemoteLinux for instructions on using SSH, SFTP, and SCP.
- Windows: we recommend the
scp client WinSCP, for which we have instructions at SFTP-SCP.
After logging into
, CHESS users can find their files in these locations:
- Data for the current run:
- Data from previous runs:
- If a particular dataset no longer appears on disk, it can be retrieved from archival tape storage by submitting a ServiceRequest.
- Sample files for testing are available at
/nfs/chess/raw/test-download/, which contains two directories:
small contains three small text files
large contains one binary file with a size of 2 GB, which may be used to measure transfer speed.
Onsite Data Access
Although it is not recommended as a general solution, CHESS users may also transfer their data to a portable USB device while onsite. Please see DataTransferRecs
for important considerations when selecting a USB device. The computers located at each beamline may be used for this purpose. Please consult with the staff scientists for instructions.
If the beamline computers are not available, there are public terminals (Windows and Linux) in Wilson 315 (instructions here
As a last resort, users may connect their systems to the CHESS Public subnet and access the DAQ directly. Please see DAQClientConfiguration
If all else fails, we can ship a hard drive containing the data that was collected to the user's home institution. The user is responsible for providing the hard drive and bearing the shipping costs. Please contact the CHESS Users' Office to make arrangements.
Getting Additional Help
If you have any computing questions:
For help with central Cornell services, contact IT@Cornell