Copy sparse file across network using dd or rsync

Submitted by Jeff on Tue, 03/07/2018 - 13:14

Simple, tried and trusted:

# time rsync --sparse --rsync-path="ionice -c 3 nice rsync" -havW --progress source.qcow2 je.ff.co.za:target.qcow2

Note: The 'time' command is obviously optional. The ionice is not, if you don't want rsync consuming all resources on the target server. (Although, I'm not entirely sure that it's actually necessary on the command line of the source server, but one would presume that it might also require it.)

It must, however, be added to all target servers' /root/.bashrc, as an alias:

alias rsync="/usr/bin/nice -n 19 /usr/bin/ionice -c3 -n7 rsync"

Update (2023-06-23)

Gist:

/etc/systemd/system/rsync.service:ExecStart=/usr/bin/nice -n 19 /usr/bin/ionice -c3 /usr/bin/rsync --daemon --no-detach
 

Some experimental options:

# dd if=image.qcow2 | gzip -1 - | ssh user@remote dd of=image.qcow2.gz

dd if=whatever | gzip | ssh dd | gunzip >whatever

netcat if security is not an issue, would far faster than ssh's encryption.

and then there is ssh -C which adds its own compression to the link

 

 

Category