Using Remote Computing Resources
OverviewTeaching: 30 min
Exercises: 0 minQuestions
How do I access remote computing resources?
How do I transfer files back and forth between my own computer and the remote computing resource?
How do I submit jobs on a remote computing resource?Objectives
Use ssh to connect to remote computing resources.
Use scp or an scp client to transfer files between computers.
Submit and monitor jobs on remote computing resources that use different job managers.
Connecting to other computers
You will often need to connect to other computers to run calculations or use resources. The most common way you connect to remote computers is using something called the Secure Shell Protocol or ssh.
- Using ssh If you are just learning about ssh, reading the sections Using the Linux client, Specifying a Different User name, and Executing Remote Commands on this page is sufficient.
To move or copy files between computers, you use secure copy or scp.
- Using scp If you are just learning about scp, the section SCP examples on this page will cover most things you need to do.
Running jobs on remote computing resources
You can imagine that if 100 people accessed a computer at one time and all tried to submit calculations, the computer would quickly become overwhelmed. Consequently, most of the time a remote computing system will have a queuing system or workload manager. This type of system has a queue where your job waits in line until the computing resources it needs are available. To submit a job to a computing resource that uses a queue, you need to copy your input file to the remote computer and you also need to create some type of submission script that tells the computer how to run your calculation and what computing resources it needs. The format of submission scripts and how they are submitted to the queue depends on what type of queue management software the computer uses.
Portable Batch System (PBS)
The ssh protocol is used to access remote computing resources.
The scp protocol is used to transfer files between computers.
Most remote computing resources will have a job manager to manage the submission and running of calculations. Using a job manager requires creating a submission script for your job in addition to the input file for the actual program.