File Transfer
Published:
Got some sequencing data? Many powerful tools to analyse them are based on the command line and this is part of a series of short but essential posts that help you getting started. I assume that you are working on a UNIX-based operating system (‘Mac’ or ‘Linux’ computer).
Before you can work on a remote server with your own data, you first
need to know how to transfer them. One of the best platform-independent
GUI programs that allows you to up- and download files is FileZilla
(Download and Documentation: https://filezilla-project.org/). In the
following lines I want to introduce the two command line tools rsync
and scp, that allow you to transfer and synchronize files.
rsync
rsync stands for “remote sync”. This powerful tool has plenty of
options. To see them all, type
``` {.bash org-language=”sh”} rsync –help
These options alter the way that files are transferred from a *source*
(SRC) location to a *destination* (DEST). Here is the most basic syntax
to transfer files with `rsync`
``` {.bash org-language="sh"}
rsync [OPTIONS] SRC DEST
Thereby, SRC and DEST can either be files or folders. For example, to transfer the file ‘file.txt’ from your local home folder to a remote server, you can type:
``` {.bash org-language=”sh”} rsync –progress /home/user/directory/file.txt user@127.0.0.1://home/user/
Here, you need to change `/home/user/directory/` to your own filepath
and `file.txt` to your own filename. In '`user@127.0.0.1`', `user`
represents your username on the remote server and `127.0.0.1` the IP
address of the remote server. The `--progress` option will indicate the
progress of the file transfer - which is useful when transferring big
files.
If you want to transfer files from the remote server to your local
computer, just swap the source and destination path specifications:
``` {.bash org-language="sh"}
rsync --progress user@127.0.0.1://home/user/file.txt /home/user/directory/
If you want to transfer all files that are located in your local folder
/home/user/directory/, you can use the following command
``` {.bash org-language=”sh”} rsync -avz –progress /home/user/directory/ user@127.0.0.1://home/user/
Here,
- `-az` will transfer the files in 'archive mode' (which combines
several options, including recursing into directories)
- `-z` will compress the files durig the transfer
Note the trailing slash after the source directory:
`/home/user/directory/`. If you do not use this trailing slash, like
`/home/user/directory`, then `rsync` will create a folder with the name
`directory` at the destination and copy all files from the source folder
into it.
**scp**
-------
`scp` stands for "secure copy". The syntax of this tool is very similar
to `rsync`. For example, to transfer a single file from a remote host to
your local computer, you can type
``` {.bash org-language="sh"}
scp user@127.0.0.1://home/user/file.txt /home/user/
To transfer all files within your local directory user/directory to
the remote server, use the -r tag:
{.bash org-language="sh"}
scp -r /home/user/directory user@127.0.0.1://home/user
This will copy the entire directory to the /home/user location on
the remote server.