rsync - How to transfer or synchronise files between servers over the Internet

rsync is a software tool for efficiently transferring data over a network or synchronising files between two locations. Rsync compares what is already the same between locations (to avoid needing to transfer data unnecessarily) and uses compression and other techniques to minimise the amount of resources needed to transfer files between network locations.

Usually Rsync is installed as standard on Linux. If it is not you can install it using:

sudo apt install rsync

sudo yum install rsync

Below are a couple of practical examples that are common ways of using rsync.

For the following examples you need to log into a machine via SSH. If you are unfamiliar with this process you can see our guide on how to log in via SSH. It is also worth being reasonably familiar with the Linux command line too.

In this example we have logged into a machine via SSH (this is the machine on the left) and we are ‘pushing’ the folder ‘htdocs’ complete with all the files and folders contained within it to another server (the one on the right in the animation above).

First we confirm a directory where we are going to transfer the files and folders to actually exists. On the right (in this case receiving server) we check the directory we are in with the command:

pwd

This seems fine to use as a receiving directory for this demonstration, you will want to change this to a directory on your own (receiving) machine based on your own needs.

Next we find the folder we are transferring and confirm its path with the command:

realpath htdocs

So we now know the full (absolute) path of the folder we are transferring is /home/ub1/htdocs and the full (absolute) path of the folder we are transferring to is /home/ub1. We will take a note of these for use in our rsync command.

rsync -vhzaP /home/ub1/htdocs [email protected]:/home/ub2

• rsync - We are using rsync so we use the rsync command to initiate it.
• -vhzaP - These are the options we are using for the rsync transfer. Let's break down what they mean:
  • v - for verbose (gives us some extra information as the command is performed.)
  • h - for human readable means that sizes are in GB, MB or kB rather than everything in bytes.
  • z - Enables compression, will reduce the amount of data that needs to be transferred over the network.
  • a - Enables archive mode, means things like permissions and last modified dates are preserved.
  • P - (note capitalised) Enables showing the progress of each file transferred, also allows the partial transferring of files. This is useful if the command is interrupted as some of the data will have been transferred already.
• /home/ub1/htdocs - This is the filepath of the folder to be transferred. (replace with your own folder path to be transferred)
• ub2 - This is the username of the recipient server (replace with your own username for your own recipient server)
• 10.0.2.5 - This is the IP address of the recipient server, (replace with your own server's IP address, or you can use a domain or subdomain that points to the server.)
• /home/ub2 - This is the path of the folder that is going to receive the transferred folder (htdocs).

In this example we have logged into a machine via SSH (this is the machine on the right) and we are ‘pulling’ the folder ‘htdocs’ complete with all the files and folders contained within it from another server (the one on the left in the animation below).

The command is very similar to the Push command previously, really the only difference is that the locations are reversed in order.

rsync -vhzaP [email protected]:/home/ub1/htdocs /home/ub2