Install and configure Lsyncd (CentOS)

Symptoms

The customer wants to sync directory or files between to computers.

Explanation

A quick start for setup on Lsyncd …

When configured, lsyncd is a replication service that will run on the “master” server to ensure the same data exists on all “slave” servers. The lsyncd process monitors the web content directory and replicates changes every few seconds to clone nodes.

Lsyncd is especially useful for synchronizing data from a secure area to a less secure area. To accomplish this, Lsyncd uses Rsync to automate the replication of files from one master server to one or more subordinate servers. Lsyncd transfers only files that have changed.

Resolution

Install and configure Lsyncd

Complete the following steps to install and configure Lsyncd.

Prerequisites

• Two or more Linux servers
• Ability to run the installation and configuration commands while logged in as the root user or from a super-user account
• Public and private IP addresses for the master server
• Public and private IP addresses for the subordinate servers

Install Lsyncd on Master Server

1. On the master server, run the following command:

ssh-keygen -t rsa

Two new files are created in ~/.ssh/ id_rsa and id_rsa.pub directories.

1. Use SFTP to access the master server, copy the id_rsa.pub file to the /root/.ssh/ folder of each of your subordinate servers, and change the name of each copied file to master.pub.
2. On your subordinate servers, run the following command:

cat ~/.ssh/master.pub >> ~/.ssh/authorized_keys

1. From the master server, SSH to log in to each subordinate server and choose Yes to accept the generated key.

Now you can log in to the subordinate servers without a password.

Install Package Dependencies on the Master Server

Run the following command for CentOS to install all needed dependencies:

        # yum -y install lua lua-devel pkgconfig gcc asciidoc 

Install Lsyncd on Destination Server

Important: Be sure to manually check the destination directory on the subordinate server. If you have NFS or CloudFuse mount your Cloud Files, these could be deleted by the Lsyncd process.

To install the version of Lsyncd required for this procedure, you need to compile the code from source. If you are unfamiliar with this process, you can copy and paste the following commands:

cd /var/tmpwget http://lsyncd.googlecode.com/files/lsyncd-2.1.5.tar.gztar xzvf lsyncd-2.1.5.tar.gzcd lsyncd-2.1.5./configure && make && make install

When the make install command finishes running, the Lsyncd service is installed on the master server. The compressed Lsyncd package does not come with startup scripts, so you must install your own.

Setup the startup scripts

Perform the following steps for CentOS.

1. Check for the existence of the following file at /etc/init.d/lsyncd.
2. Configure Lsyncd to start at boot:

       # chkconfig lsyncd on

Configure log rotation

 After Lsyncd has an assigned log area, it will create a log event if an error occurs during replication. Log files can become large and unruly when not properly monitored, causing a hard disk to fill up. To ensure this does not happen, add Lsyncd to logrotate.

Start by creating a directory to store the log files.

   # mkdir /var/log/lsyncd

Create a file at /etc/logrotate.d/lsyncd with the following shell script:

cat << EOF > /etc/logrotate.d/lsyncd               /var/log/lsyncd/*log {              missingok              notifempty              sharedscripts              postrotate              if [ -f /var/lock/lsyncd ]; then              /sbin/service lsyncd restart > /dev/null 2>/dev/null || true             fi              endscript       }       EOF

With this script enabled, the logs will be rotated instead of accumulating until they are manually deleted.

Create the configuration file

 On the master server, create a file at /etc/lsyncd.lua and insert the following data with a text editor.

Note: Replace the target parameter with the private IP address of the subordinate server. You can find the IP address in the Rackspace Cloud Control Panel.

settings {                    logfile = "/var/log/lsyncd/lsyncd.log",                         statusFile = "/var/log/lsyncd/lsyncd-status.log",                         statusInterval = 20                 }                 sync {                         default.rsync,                         source="/var/www/",                   target="subordinatePrivateIpAddress:/var/www/",                         rsync = {                        compress = true,                        acls = true,                        verbose = true,                        rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no" }                 }

Take special note of the block of code beginning with sync. If you require more web servers, this block of code will must be replicated with the IP address of each additional subordinate server.

With configuration complete, start the lsyncd service.

   # service lsyncd start

Test the configuration

 Before continuing, test the configuration.

1. Create a file in the /var/www/ directory on the master server. You can do this with the touch command.

         # touch /var/www/dummy_file

If everything is working correctly, the file is replicated to the /var/www/ directory on the subordinate server.

1. To verify that the file was replicated, use SSH to log in to the subordinate server and run the ls command.

         # ls /var/www

Notes:

References:

1. Article: Install and configure lsyncd