Copying directory trees with rsync

You can use cp -a to copy directory trees, but rsync can do the same and give you more flexibility. rsync supports a syntax for filter rules which specify which files and directories should and should not be copied.

Examples

Copy only the directory structure without copying any files:

$ rsync -a -f"+ */" -f"- *" source/ destination/

The two -f arguments mean, respectively, "copy all directories" and then "do not copy anything else".

Copy only directories and Python files:

$ rsync -a -f"+ */" -f"+ *.py" -f"- *" source/ destination/

This is really handy for replicating the general directory structure but only copying a subset of the files.

Copy everything but exclude .git directories:

$ rsync -a -f"- .git/" -f"+ *" source/ destination/

Conclusion

Of course, rsync also works great for copying files between machines, and it knows better than to transfer files that already exist on the destination. I use something similar to the above to do backups, copying my homedir but excluding stuff like caches that are not even worth copying.

13 comments:

  1. Very good examples. Thank you.

    ReplyDelete
  2. First, thanks :)
    Second, how can I have it copy all directories under /var/www but exclude all files under /var/www ; recursively...

    ReplyDelete
  3. PERFECT. Been looking this for a while.

    ReplyDelete
  4. Thanks for the tip! Just used this today.

    ReplyDelete
  5. Shai, you replace "source/" and "destination/" with:

    /var/www/ /some/directory/

    /var/www/ somemachine:/some/directory/

    ReplyDelete
  6. thnx for tip very helpful

    ReplyDelete
  7. Excellent. Rsyncing the directory structure only was exactly what I needed!

    ReplyDelete
  8. Thanks for the tip Phil!

    For those who have an older version of rsync (one that doesn't support -f), you can do the same thing with --include and --exclude.

    So Phil's example becomes:

    $ rsync -a --include=*/ --exclude=* source/ destination/

    -klam

    ReplyDelete
  9. I have a question. I tried to use the syntax above under rsync 3.0.9 with the following results (source and dest names changed to protect the not-so-innocent):

    rsync -v -f"+ */" -f"- *" /source desthost:/dest

    skipping directory source


    rsync -v -f"+ */" -f"- *" /source/ desthost:/dest/

    skipping directory .


    Am I missing something, or did the behavior change in 3.0.9?

    ReplyDelete
  10. Found the trick. I had to use -r to recurse the tree.

    rsync -v -a -r -f"+ */" -f"- *"

    ReplyDelete
  11. -v is verbose output
    -r will recurse through the directories
    -a is "archive" mode which includes -r
    FROM "rsync --help":
    -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)

    So rsync -rv would have worked as well.

    ReplyDelete