Blog

marinetics blog


Bioinformatics Solutions in Ecology and Evolution

09 Feb 2015

File Transfer

Tagged: Commandline






Related Posts

Got some sequencing data? Many powerful tools to analyse them are based on the command line and this is part of a series of short but essential posts that help you getting started. I assume that you are working on a UNIX-based operating system (‘Mac’ or ‘Linux’ computer).

Before you can work on a remote server with your own data, you first need to know how to transfer them. One of the best platform-independent GUI programs that allows you to up- and download files is FileZilla (Download and Documentation: https://filezilla-project.org/). In the following lines I want to introduce the two command line tools rsync and scp, that allow you to transfer and synchronize files.

rsync

rsync stands for “remote sync”. This powerful tool has plenty of options. To see them all, type 

rsync --help

These options alter the way that files are transferred from a source (SRC) location to a destination (DEST). Here is the most basic syntax to transfer files with rsync 

rsync [OPTIONS] SRC DEST

Thereby, SRC and DEST can either be files or folders. For example, to transfer the file ‘file.txt’ from your local home folder to a remote server, you can type: 

rsync --progress /home/user/directory/file.txt user@127.0.0.1://home/user/

Here, you need to change /home/user/directory/ to your own filepath and file.txt to your own filename. In ‘user@127.0.0.1’, user represents your username on the remote server and 127.0.0.1 the IP address of the remote server. The --progress option will indicate the progress of the file transfer - which is useful when transferring big files.

If you want to transfer files from the remote server to your local computer, just swap the source and destination path specifications: 

rsync --progress  user@127.0.0.1://home/user/file.txt /home/user/directory/

If you want to transfer all files that are located in your local folder /home/user/directory/, you can use the following command 

rsync -avz --progress /home/user/directory/ user@127.0.0.1://home/user/

Here,

  • -az will transfer the files in ‘archive mode’ (which combines several options, including recursing into directories)
  • -z will compress the files durig the transfer

Note the trailing slash after the source directory: /home/user/directory/. If you do not use this trailing slash, like /home/user/directory, then rsync will create a folder with the name directory at the destination and copy all files from the source folder into it.

scp

scp stands for “secure copy”. The syntax of this tool is very similar to rsync. For example, to transfer a single file from a remote host to your local computer, you can type 

scp user@127.0.0.1://home/user/file.txt /home/user/

To transfer all files within your local directory user/directory to the remote server, use the -r tag: 

scp -r /home/user/directory user@127.0.0.1://home/user

This will copy the entire directory to the /home/user location on the remote server.