A Guide to Syncing Local and Remote Directories with Rsync

Rsync (Remote Sync) is a powerful tool for synchronizing files and directories between two locations, either on the same machine or between a local and a remote system. It is efficient because it only transfers the changed parts of files rather than the entire file, which saves time and bandwidth. In this guide, you’ll learn how to use rsync to sync directories locally and remotely, including the basic syntax, important options, and practical examples.

Prerequisites

A Unix-based system (Linux, macOS) or a Windows system with a Unix-like shell (like Cygwin or Windows Subsystem for Linux).
Rsync installed on both the local and remote machines (most Unix-based systems have rsync pre-installed).
SSH access to the remote machine (if syncing to/from a remote location).

Installing Rsync

If rsync is not already installed on your system, you can install it using the package manager for your distribution.

On Debian/Ubuntu:

  sudo apt update
  sudo apt install rsync

On Red Hat/CentOS:

  sudo yum install rsync

On macOS (using Homebrew):

  brew install rsync

Basic Rsync Syntax

The basic syntax for an rsync command is:

rsync [options] source destination

source: The file or directory you want to sync.
destination: The location where you want to copy the files or directories.
[options]: Rsync flags that customize the behavior of the command.

Rsync Options and Flags

Some of the most commonly used options are:

-r : Recursively copy all directories and subdirectories.
-a : Archive mode, which preserves symbolic links, file permissions, user & group ownership, and timestamps. It also includes the recursive flag.
-v : Verbose mode, to display detailed information about the progress.
-z : Compress file data during the transfer to save bandwidth.
-P : Shows progress during the transfer and allows resuming interrupted transfers.
-e ssh : Use SSH for remote transfers.
--delete : Delete files from the destination that are not in the source.

Syncing Directories Locally

If you want to synchronize directories between different locations on your local machine, the command is straightforward.

Example: Sync two local directories

rsync -av /path/to/source_directory/ /path/to/destination_directory/

In this example:

-a preserves file attributes and timestamps.
-v gives verbose output.
The trailing slashes / ensure that the content of the source directory is copied to the destination, rather than the directory itself.

Syncing Directories Between a Local and Remote Machine

When syncing with a remote machine, rsync typically uses SSH to securely transfer files.

Example: Sync from a local machine to a remote server

rsync -avz -e ssh /path/to/source_directory/ username@remote_host:/path/to/destination_directory/

This command will sync the content of source_directory from the local machine to destination_directory on the remote machine.

-z compresses the data during transfer.
-e ssh specifies the use of SSH for the connection.
username@remote_host is the SSH login for the remote machine.

Example: Sync from a remote server to a local machine

rsync -avz -e ssh username@remote_host:/path/to/source_directory/ /path/to/local_destination_directory/

This command pulls files from a remote machine to your local system.

Excluding Files and Directories

You can exclude specific files or directories from being synced by using the --exclude option.

Example: Exclude files from sync

rsync -avz --exclude '*.log' --exclude 'temp/' /path/to/source_directory/ /path/to/destination_directory/

In this case, files with the .log extension and the temp/ directory will be excluded from the sync.

Syncing Only Changed Files

By default, rsync only transfers files that have changed since the last sync, which makes it extremely efficient. It compares the timestamps and file sizes to detect changes.

Example: Sync only updated files

rsync -av --ignore-existing /path/to/source_directory/ /path/to/destination_directory/

The --ignore-existing option ensures that only new or updated files are transferred, leaving existing files untouched.

Deleting Files on the Destination

If files have been deleted from the source and you want to mirror that change on the destination, use the --delete option.

Example: Mirror directories by deleting files that are no longer present in the source

rsync -av --delete /path/to/source_directory/ /path/to/destination_directory/

This will ensure that the destination directory is an exact copy of the source, removing any extra files that no longer exist in the source directory.

Dry Run Mode

Before actually syncing, it’s a good idea to perform a dry run to see what changes will be made without applying them.

Example: Perform a dry run

rsync -av --dry-run /path/to/source_directory/ /path/to/destination_directory/

The --dry-run option simulates the command and shows what would be transferred, but doesn’t make any changes.

Scheduling Rsync with Cron

If you want to automate the sync process, you can schedule it to run periodically using cron.

Example: Sync files every day at midnight

Open your crontab file:

   crontab -e

Add the following line to schedule the sync:

   0 0 * * * rsync -avz -e ssh /path/to/source_directory/ username@remote_host:/path/to/destination_directory/

This cron job will run the rsync command every day at midnight.

Practical Use Cases for Rsync

Backup Local Data to a Remote Server: Use rsync to automatically back up important data from your local machine to a remote server.
Sync Large Data Sets: Rsync is ideal for transferring large datasets between machines, as it minimizes the amount of data transferred by only sending changes.
Website Deployment: Developers use rsync to deploy updated website files to remote web servers efficiently.

Conclusion

Rsync is a versatile and efficient tool for synchronizing files and directories between local and remote systems. With its wide array of options, it can be customized for various use cases like backups, mirroring directories, or syncing large datasets. By leveraging SSH for secure transfers, excluding unnecessary files, and using flags like --delete to mirror directories, you can keep your data synchronized efficiently with minimal bandwidth usage.

Start with simple directory syncing, and as you become more comfortable, explore advanced options like scheduled backups with cron or the use of SSH for secure transfers. Rsync is a must-have in any system administrator’s toolkit for file synchronization and backup automation.

Author
Recent Posts

Eric Sandler

Chief Writer at WiFi Planet

Eric Sandler is a veteran US tech writer with over 25 years of experience covering the latest advancements in the world of technology. From groundbreaking innovations to in-depth reviews, Eric has built a reputation for delivering insightful, reliable content. His work spans across a variety of tech topics, including smartphones, gadgets, AI, and more, making him a trusted voice in the industry.