So: you've got two or more clients and/or servers. They contain files that you want to have automatically synced when possible, because that would save a lot of time. Well, I got the solution for you: with a little bit of thinking in an innovative way I have found the solution that might bring you onto the right path as well. Rsync is a great solution, but having to run rsync manually would take a lot of unnecessary time away, right? And that is where inotify is for: real-time monitoring of your filesystem so that your files can be synced between multiple machines with the power of rsync!
Setting up a test scenario
I needed to get myself a nice development environment at first so I started off 3 virtual servers which all run Ubuntu 16.04, my personal favourite. All these 3 machines needed to be setup with the following software packages:
- OpenSSH server
Also noteworthy is that these machines are absolutely not connected through a private network. All rsync traffic is supposed to be worked out over SFTP.
For who is this for?
I could see some potential for workflow improvement on these situations:
- A development environment, where constant file transfers are taking up a lot and/or too much time
- Load balanced file storage clusters/servers
- Backup/failover servers with the need for constant replication
Working on it
First things first: we need to get all the dependencies installed on the 3 servers with this one-line command:
apt update && apt -y install openssh-server rsync inotify-tools
After that, let's create a specific folder that we want to sync. Let's call it SyncFiles:
And for secure file transfer, we want a public-private key link for the transfer link that rsync uses, this is how to configure it:
ssh-keygen -t rsa -f ~/rsync-key -N '' # Paste the output in your destination servers' ~/.ssh/authorized_keys file: cat ~/rsync-key.pub # Removing public key for security purposes.. rm ~/rsync-key.pub # Remember to execute this script on all servers separately! Then, copy the output of the script in all of your servers' authorized_keys files.
Now that you have got all the pre-configuration work done, it's about time to write a script that goes through an infinite loop with inotifywait in it:
#!/bin/bash # Supposed to run on rsync-host01, change rsync-host02 to rsync-host01 to make a script that is meant to run on rsync-host02. while true; do inotifywait -r -e modify,attrib,close_write,move,create,delete /opt/syncfiles rsync -avz -e "ssh -i /root/rsync-key -o StrictHostKeyChecking=no“ /opt/syncfiles/ root@rsync-host02:/opt/syncfiles/ done
I saved this script in the /opt directory as file-sync.sh.
To finish things off, lets create the systemd service file that can stop, start, and reset the script on demand or on specific events like a system bootup.
Create a file called sync.service in the directory /etc/systemd/system/ and put the following contents in it:
[Unit] Description = SyncService After = network.target [Service] PIDFile = /run/syncservice/syncservice.pid User = root Group = root WorkingDirectory = /opt ExecStartPre = /bin/mkdir /run/syncservice ExecStartPre = /bin/chown -R root:root /run/syncservice ExecStart = /bin/bash /opt/file-sync.sh ExecReload = /bin/kill -s HUP $MAINPID ExecStop = /bin/kill -s TERM $MAINPID ExecStopPost = /bin/rm -rf /run/syncservice PrivateTmp = true [Install] WantedBy = multi-user.target
Chmod this service file and reload the systemd daemon:
chmod 755 /etc/systemd/system/sync.service systemctl daemon-reload
You are all set! You can now use these commands to manage your self-made directory sync daemon:
# Start your service systemctl start sync.service # Obtain your services' status systemctl status sync.service # Stop your service systemctl stop sync.service # Restart your service systemctl restart sync.service