CLI – rsync

The rsync command is a utility common to Linux, Unix, BSD and macOS. Versions of rsync are now available on Windows systems as well.

This command is used to synchronize files and directories. This can be done between locally attached storage, or between two different network connected systems. Design of the rsync algorithm minimizes the network usage, while still maintaining file synchronization.

The rsync daemon can be run on a machine to allow other remote machines to copy file to or from it. The rsync command requires arguments indicating the source and destination locations.

There are many different command line options available to rsync. The general format is: rsync options source destination

One of the nice features of rsync is that it only copies files that have changed since the last time they were transferred. Check the reference list at the bottom of this article for suggestions of the best set for your application. Some of the simplest are:

  • rsync -avh /home/usr/dir/ /media/disk/backup/
    – this copies everything in the directory /home/usr/dir/ to /media/disk/backup/
  • rsync -avh –delete /home/user/dir/ /media/disk/backup – does the same as above except that files deleted from /home/user/dir/ will also be deleted from /media/disk/backup
  • rsync –progress -avh /home/usr/dir/ /media/disk/backup/ – does the same as the first example, but show how much of the copy is remaining

When I run rsync -h on my macOS 10.11.6 system I get the following list of options:

rsync is a file transfer program capable of efficient remote update
 via a fast differencing algorithm.

Usage: rsync [OPTION]... SRC [SRC]... DEST
 or   rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
 or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
 or   rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
 or   rsync [OPTION]... [USER@]HOST:SRC [DEST]
 or   rsync [OPTION]... [USER@]HOST::SRC [DEST]
 or   rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]

The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect to an rsync daemon, and require SRC or DEST to start with a module name.

Options

-v, --verbose               increase verbosity
 -q, --quiet                 suppress non-error messages
 --no-motd               suppress daemon-mode MOTD (see manpage caveat)
 -c, --checksum              skip based on checksum, not mod-time & size
 -a, --archive               archive mode; same as -rlptgoD (no -H)
 --no-OPTION             turn off an implied OPTION (e.g. --no-D)
 -r, --recursive             recurse into directories
 -R, --relative              use relative path names
 --no-implied-dirs       don't send implied dirs with --relative
 -b, --backup                make backups (see --suffix & --backup-dir)
 --backup-dir=DIR        make backups into hierarchy based in DIR
 --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)
 -u, --update                skip files that are newer on the receiver
 --inplace               update destination files in-place (SEE MAN PAGE)
 --append                append data onto shorter files
 -d, --dirs                  transfer directories without recursing
 -l, --links                 copy symlinks as symlinks
 -L, --copy-links            transform symlink into referent file/dir
 --copy-unsafe-links     only "unsafe" symlinks are transformed
 --safe-links            ignore symlinks that point outside the source tree
 -k, --copy-dirlinks         transform symlink to a dir into referent dir
 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
 -H, --hard-links            preserve hard links
 -p, --perms                 preserve permissions
 --executability         preserve the file's executability
 --chmod=CHMOD           affect file and/or directory permissions
 -o, --owner                 preserve owner (super-user only)
 -g, --group                 preserve group
 --devices               preserve device files (super-user only)
 --specials              preserve special files
 -D                          same as --devices --specials
 -t, --times                 preserve times
 -O, --omit-dir-times        omit directories when preserving times
 --super                 receiver attempts super-user activities
 -S, --sparse                handle sparse files efficiently
 -n, --dry-run               show what would have been transferred
 -W, --whole-file            copy files whole (without rsync algorithm)
 -x, --one-file-system       don't cross filesystem boundaries
 -B, --block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND           specify the remote shell to use
 --rsync-path=PROGRAM    specify the rsync to run on the remote machine
 --existing              skip creating new files on receiver
 --ignore-existing       skip updating files that already exist on receiver
 --remove-source-files   sender removes synchronized files (non-dirs)
 --del                   an alias for --delete-during
 --delete                delete extraneous files from destination dirs
 --delete-before         receiver deletes before transfer (default)
 --delete-during         receiver deletes during transfer, not before
 --delete-after          receiver deletes after transfer, not before
 --delete-excluded       also delete excluded files from destination dirs
 --ignore-errors         delete even if there are I/O errors
 --force                 force deletion of directories even if not empty
 --max-delete=NUM        don't delete more than NUM files
 --max-size=SIZE         don't transfer any file larger than SIZE
 --min-size=SIZE         don't transfer any file smaller than SIZ
 --partial               keep partially transferred files
 --partial-dir=DIR       put a partially transferred file into DIR
 --delay-updates         put all updated files into place at transfer's end
 -m, --prune-empty-dirs      prune empty directory chains from the file-list
 --numeric-ids           don't map uid/gid values by user/group name
 --timeout=TIME          set I/O timeout in seconds
 -I, --ignore-times          don't skip files that match in size and mod-time
 --size-only             skip files that match in size
 --modify-window=NUM     compare mod-times with reduced accuracy
 -T, --temp-dir=DIR          create temporary files in directory DIR
 -y, --fuzzy                 find similar file for basis if no dest file
 --compare-dest=DIR      also compare destination files relative to DIR
 --copy-dest=DIR         ... and include copies of unchanged files
 --link-dest=DIR         hardlink to files in DIR when unchanged
 -z, --compress              compress file data during the transfe
 --compress-level=NUM    explicitly set compression level
 -C, --cvs-exclude           auto-ignore files the same way CVS does
 -f, --filter=RULE           add a file-filtering RULE
 -F                          same as --filter='dir-merge /.rsync-filter'
 repeated: --filter='- .rsync-filter'
 --exclude=PATTERN       exclude files matching PATTERN
 --exclude-from=FILE     read exclude patterns from FILE
 --include=PATTERN       don't exclude files matching PATTERN
 --include-from=FILE     read include patterns from FILE
 --files-from=FILE       read list of source-file names from FILE
 -0, --from0                 all *-from/filter files are delimited by 0s
 --address=ADDRESS       bind address for outgoing socket to daemon
 --port=PORT             specify double-colon alternate port number
 --sockopts=OPTIONS      specify custom TCP options
 --blocking-io           use blocking I/O for the remote shell
 --stats                 give some file-transfer stats
 -8, --8-bit-output          leave high-bit chars unescaped in output
 -h, --human-readable        output numbers in a human-readable format
 --progress              show progress during transfer
 -P                          same as --partial --progress
 -i, --itemize-changes       output a change-summary for all updates
 --out-format=FORMAT     output updates using the specified FORMAT
 --log-file=FILE         log what we're doing to the specified FILE
 --log-file-format=FMT   log updates using the specified FMT
 --password-file=FILE    read password from FILE
 --list-only             list the files instead of copying them
 --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
 --write-batch=FILE      write a batched update to FILE
 --only-write-batch=FILE like --write-batch but w/o updating destination
 --read-batch=FILE       read a batched update from FILE
 --protocol=NUM          force an older protocol version to be used
 -E, --extended-attributes   copy extended attributes
 --cache                 disable fcntl(F_NOCACHE)
 -4, --ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
 --version               print version number
 (-h) --help                  show this help (-h works with no other options)

References:

  1. How to Backup Linux? 15 rsync Command Examples
  2. Do-It-Yourself Backup System Using Rsync
  3. Back up like an expert with rsync

See my other Command Line articles


 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s