Get Mozilla Firefox
Rsync is a powerfull command line tool to copy/syncronize files and folders to local or remote hosts.
It's perfectly suited for copying files and backing up a file server for example. However, it is not suitable for copying files that are in use such as databases or virtual machines because in this case you will end up with corrupted (unusable) file copies.
Generally rsync is not preinstalled so the first thing to do is to install it.
user@host:~$ sudo apt install rsync
[root@host ~]# pacman -S rsync
Let's start simple, we will use rsync to copy a file from our /home/std/ folder to /tmp/.
user@host:~$ rsync -av /home/std/myfile_source /tmp/myfile_dest
We can also use rsync to recursively copy a folder. Here we will copy our entire /home/std folder to /tmp/.
user@host:~$ rsync -av /home/std/ /tmp/folder_dest
More interestingly, we can use rsync to copy files or folders over the network. For this to work, rsync must be installed on both parties and an ssh server must be installed on the remote host.
user@host:~$ rsync -av -e 'ssh -p 22' /home/std user@192.168.1.100:/data/
user@host:~$ rsync -av -e 'ssh -p 22' user@192.168.1.100:/data/std /home/
As explained earlier rsync is particulary suitable for backing up file servers. We will see how to make backup to usb drive, to a remote host and also how to do incrementals in order to save disk space.
Here we will make a full backup of our GNU/Linux operating system. We will exclude special or useless folders (/dev, /mnt, /proc, /sys and /tmp). The destination folder on the usb disk will be automaticaly named with the current date in back-YYYY-MM-dd format.
root@host:~# rsync -av --delete --delete-excluded --exclude=/tmp --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/mnt / /mnt/usb/back-$(date "+%Y-%m-%d")
One of the most beautiful things about rsync is that it is able to do incremental backups. To do this, it uses hardlinks. For each file to be backed up, rsync compares to the last backup (the current folder on the USB drive which is a symbolic link to the last backup folder) : if the file is the same, it creates a hardlink and does not consume disk space, otherwise (if the file is new or is different) it creates a new file.
Of course the current symbolic link must be created manually and recreated after each backup in order to make it point to the last backup.
As we can see on the diagram, the first backup will take the most space (10GB here), then the others backups will only take the difference (1GB or 2GB in this example).
One last thing to know, in case we want to recover space on our USB disk we will have to delete the backups from the oldest to the most recent (in order to avoid breaking the hardlinks).
It is the detailed application of what has been seen just above.
user@host:~$ rsync -av --delete --delete-excluded --exclude=/tmp --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/mnt --exclude=/media --link-dest=/mnt/USB/current / /mnt/usb/back-$(date "+%Y-%m-%d")
user@host:~$ rm -f /mnt/usb/current
user@host:~$ ln -s /mnt/usb/back-$(date "+%Y-%m-%d") /mnt/usb/current
user@host:~$ du -sh /mnt/usb/back-*
10G back-2022-08-11
1G back-2022-08-12
2G back-2022-08-13
1G back-2022-08-14
Same as above except that this time the destination is a GNU/Linux server so the backup will be done over the network.
user@host:~$ rsync -av --delete --delete-excluded --exclude=/tmp --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/mnt --exclude=/media --link-dest=/backup/current / user@192.168.1.100:/backup/back-$(date "+%Y-%m-%d")
user@host:~$ ssh -p 22 user@192.168.1.100 "rm /backup/current"
user@host:~$ ssh -p 22 user@192.168.1.100 "ln -s /backup/back-$(date "+%Y-%m-%d") /backup/current"
I put here everything I found useful.
user@host:~$ rsync --progress -av /home/std/folder_source/ remoteuser@192.168.1.100:/tmp/folder_dest
user@host:~$ rsync --stats -av /home/std/folder_source/ remoteuser@192.168.1.100:/tmp/folder_dest
user@host:~$ rsync --progress -av --bwlimit=100k /home/std/folder_source/ remoteuser@192.168.1.100:/tmp/folder_dest
Note : needs local transfer or ssh passwordless to work
user@host:~$ rsync --progress -av remoteuser@192.168.1.100:/ --no-relative --files-from=<(ssh remoteuser@192.168.1.100 find /data/* -mtime -3 -type f) /tmp/
user@host:~$ rsync --progress /home/std/ /tmp/folder_dest
root@host:~# visudo
user ALL=NOPASSWD:/usr/bin/rsync
user@host:~$ rsync -av -e 'ssh -p 22' --rsync-path='sudo rsync' /home/std user@192.168.1.100:/data/
Contact :
