www.enhydra.org
 

 
FRONT PAGE
Table of Contents  
       
  Syntax

    Enhydra application files

 
    Customizing configuration files

 
    Copying files

 
    Security

 
    Availability

 

Using rsync to Deploy Enhydra Web Sites

Bill Karwin, Lutris Technologies Engineer

You're developing a web site in a multi-server environment, with separate servers for development, staging, and production. After spending all morning trying to debug your Javascript rollover code, you find that you forgot to copy the latest images from the development server to the staging server. There must be a better solution. You need a tool to help keep all your deployed files synchronized across multiple servers.

Rsync is a command-line tool to help transfer collections of files incrementally from one host to another. Rsync is an open source software package (GNU Public License) available at http://rsync.samba.org/.

Below are some of the capabilities of rsync:

  • File collections:
    Group together files, links, devices, and directories recursively.
    Exclude specified files, or CVS-related files.
    Preserve file ownership and permissions.

  • Copying collections:
    Copy collections over a network to another host.
    Don't transfer files that are identical on both hosts.
    Compress data during transfer.

  • Security:
    Support anonymous or authenticated connections.
    Encrypt data during transfer using ssh.
    Don't require root privileges to perform a transfer.

There are other ways to use rsync, for instance to copy files from one location to another on the same machine, or to get a list of files on a remote machine. I will describe ways that I have used rsync in web site projects.

Syntax:

Below  is an example of using rsync to copy an Apache htdocs directory from a staging server to a production server.

rsync -avuzCb /usr/apache/htdocs webadmin@www.myfirm.com:/usr/apache/htdocs

You would execute this command within a shell login on your staging server. I'll separate it so I can describe the elements.
 

rsync

The command rsync, assuming you have installed it. 

 -a

The option for "archive mode," which specifies recursive copying and preservation of all file attributes. 

 -v

The option for verbose messages. 

 -u

The option for updating files, without overwriting newer files. 

 -z

The option for data compression using zlib. 

 -C

The option for ignoring CVS-related files. 

 -b

The option for backing up old files that would be overwritten. 

 /usr/apache/htdocs

The source directory for the files. 

 webadmin@www.myfirm.com:/usr/apache/htdocs

The destination location for the files.
The name "webadmin" is a valid login ID on the remote host.
The name "www.myfirm.com" is an example of a production server hostname.
The path of the destination directory is the same as the source path in this example, but it doesn't have to be. 

The example above uses the syntax "login@host:/path", which indicates that the rsync client connects to the remote host using standard rsh or ssh. Rsync uses rsh by default, but you can set the environment variable RSYNC_RSH to "ssh", for example, to use that mechanism for secure connections.

You can also configure a specialized rsync service on the remote host if you need anonymous access, more control over authentication, or an option to restrict rsync activity to predefined collections of files. That's a topic for another article.
 

Enhydra Application Files

Using rsync to deploy Enhydra applications is a little different. You have to shut down and restart a running instance of the application if you replace its .jar file. You also need to restart the application to take advantage of changes in configuration files.

  1. Deploy application binaries and configuration files using rsync.

  2. rsync -avuzCb ./output/ webadmin@www.myfirm.com:/usr/apache/apps/

  3. Open the Enhydra Multiserver Admin console interface for the deployment server.

  4. http://www.myfirm.com:8001/

  5. Shut down the currently running instance of the application.

  6. Start the application; this should use the new version of your application and its config files.

  7. Run regression tests and test the new application changes. (This goes without saying!)

You should use the -b option to make a backup of files on the destination server that are overwritten during the transfer of new files.  Backup files are given a suffix character "~". This allows you to revert to the previous version of your application (manually, by renaming the backup files and restarting the application) after deploying an update.
 

Customizing Configuration Files

Your application configuration file might have host-specific entries, such as a JDBC URL or a pathname to an error log file. If entries in the configuration files differ between environments, you have to maintain different edited files on each host. Usually, making these edits is a manual task and keeping the files under source control is complex.

To solve this, I keep several config files under source control, named with host-specific suffixes. For example, I might name these files myApp_dev.conf, myApp_stage.conf, and myApp_www.conf. On each server, I create a symbolic link from myApp_<host>.conf to myApp.conf:

ln -s myApp_stage.conf myApp.conf

The link myApp.conf therefore points to a different file on each server, even though all three config files reside in the source control database and on all three servers. When I deploy application files, I exclude myApp.conf from the copying, to avoid overwriting the myApp.conf link on each server. I use the --exclude=myApp.conf option of rsync.

rsync -avuzCb --exclude myApp.conf ./output/ webadmin@www.myfirm.com:/usr/apache/apps/

You might also find it convenient to create a Makefile rule to deploy the finished application to another server after building it. For example:

dist: jar
@echo "Copying finished $(JARNAME) application to deployment server..."
rsync -avuzCb --exclude myApp.conf \
$(OUTPUT)/ webadmin@www.myfirm.com:/usr/apache/apps/$(JARNAME)/

Copying Files

Rsync is efficient at transferring files. First, it filters files by comparing them with the contents of the destination locations. If the destination copy of the file is newer or identical to the local file, then rsync doesn't copy that file. Also, rsync compresses file data before transferring it across the network. These steps reduces overall transfer time when deploying.  There are options to override this behavior, but usually it's desirable.  If you want to watch the progress and get some statistics on the transfer, use the options -v --progress --stats.

Security

In secure environments, rsh and NFS should be disabled outside a firewall. Ssh is a secure method of connecting to a host. Rsync can use the ssh protocol to establish an encrypted connection for copying files. Set the RSYNC_RSH environment variable to "ssh". You can set the environment variables USER and RSYNC_PASSWORD to automate authentication, but it's not recommended to put a plaintext password in your environment.

Availability

Rsync is open source (GPL), and you can download the source from http://rsync.samba.org/rsync/download.html. Volunteers have contributed binaries for selected versions of Linux i386/PowerPC, Solaris SPARC/x86, BSDi, AIX, HP-UX, SCO, and SunOS. If you use another platform, you have to build rsync using a C compiler. Rsync works best in most UNIX/Linux environments, and it reportedly can work in Microsoft Windows 9x/NT environments too.
 

I now view rsync as a valuable tool to help automate copying collections of files from development or staging environments to production servers. I can relieve myself of many repetitive details of deployment, reduce the risk of mistakes, and perform site updates more quickly.

SEE ALSO: By coincedence, this month in Webtechniques (November 2000) Bryant Durrell contributes an article about using rsync for web content distribution. Great minds think alike. http://www.webtechniques.com/archives/2000/11/durrell/

 

 

TOP OF PAGE

Lutris Technologies    Legal Notices    Privacy Policy