I have been trying for years to find a way to keep a central source of my important files that could be accessed from anywhere.  For a long time I kept these files on my Fileserver, which is an old G4/533 Mac that is constantly online.  This was fine for text files, as I could ssh into the fileserver and adjust access the text files through Putty on a PC or terminal on another Mac.  The issue is with other types of files such as images and PDFs.

I decided to try Dropbox to scratch this itch, but I still had some issues:

1. Dropbox will keep all the files in one designated folder in sync between the Dropbox server and any other computers you hook up using the Dropbox application.  However, there is no way to only sync up certain folders within, say, your Documents folder.  One requirement I have is to keep my Documents folder on my Mac Book Pro the way it is, but only sync up certain folders from my Documents.
2. I tried variations of Aliases and Symbolic links, but this was not acceptable.  I usually use list view in the finder, and I make heavy use of the expanding triangles there, so Aliases and Links were not working for me.
3. Dropbox only lets you store 2GB on their server, so I had no issue with having my data in two places locally to solve my problem.
4. I would like to have my Documents folder be the system of record – meaning any changes to Documents would be reflected on Dropbox, and any changes to Dropbox would be overwritten by what was on my documents.  The only issue here is the ability to transfer from another computer to my Mac view Dropbox, which I also would like to do.

I think that’s it for my requirements.  I went through a few different designs, but here is what I ultimately came up with:

1. Create a script on the mac that would use rsync to copy my select Document folders to my Dropbox folder.
2. Set up my system so this rsync script would run at a set interval.
3. Create a transfer folder in the Dropbox that was not included in the sync for moving files from the cloud to my Mac.

So I will go through each of these steps in a little detail just to show how this went down.  The first step was to set up the rsync script.  For those who don’t know, rsync is a Unix utility that will keep a source location in sync with a destimation location by copying over anything that is at the destination.  There are a ton of options with rsync – if you want to learn more, the man page is excellent.

What I want to do here is set 5 or 6 of the folders in my Documents folder to sync up and leave the rest unsynced.  Here is a diagram of what I’m trying to do:

Documents Folder 		Dropbox Folder
Articles	--rsync---->	Articles
Beer-Recipes	--rsync---->	Beer-Recipes
Business
Ebooks		--rsync---->	Ebooks
Finance
Lilypond	--rsync---->	Lilypond
Manuals-Receipts
Notes		--rsync---->	Notes
Parallels
Photos		no sync		Photos
Public		no sync		Public
Unix		--rsync---->	Unix
 				Transfer

So, as you can see, there are several folders I will sync with my Dropbox, some folders I will not sync and that are not in my Dropbox (I don’t need these folders available away from my Mac), and two folder that are not synced, but exist in both folders.  These two – Photos and Public – serve different purposes in the two folders.  Public on my Mac are files that are available to other users on my network, whereas Public on Dropbox is open to the entire internet.

There is also a folder on Dropbox that is not in my Documents folder – Transfer.  This is folder I will use to load things to my Mac from other computers I set up with Dropbox.  If any of the synced folders are changed by other computers, though (like if I accidentally delete all of my notes, for example), they will automatically be copied from my Mac to Dropbox next time I log in ot my Mac.

Here is the script I wrote to accomplish this, nice and simple.  I am sure there are other things I could do to imporve this.  Feel free to comment or drop me an email if you have suggestions:

#!/bin/bash
# Sync of Document directories to my dropbox - Nathan Bibb 2009-01-22
rsync -a --delete ~/Documents/Articles ~/Dropbox/
rsync -a --delete ~/Documents/Beer-Recipes ~/Dropbox/
rsync -a --delete ~/Documents/Ebooks ~/Dropbox/
rsync -a --delete ~/Documents/Lilypond ~/Dropbox/
rsync -a --delete ~/Documents/Games ~/Dropbox/
rsync -a --delete ~/Documents/Music-Notes ~/Dropbox/
rsync -a --delete ~/Documents/Networking ~/Dropbox/
rsync -a --delete ~/Documents/Notes ~/Dropbox/
rsync -a --delete ~/Documents/Science ~/Dropbox/
rsync -a --delete ~/Documents/Unix ~/Dropbox/

So I saved this text file in my Library/Scripts folder as “dropboxrsync”.  Now I just have to set it up to run every 30 seconds or so.

This is where I ran into a problem.  I was planning to use the unix command cron, but I ran into some problems, then found out the as of Mac OS 10.4 cron was depreciated in favor of a program called launchd.  I had to do a lo of research to figure out how this whole thing works, but I finally got it going.  Here’s what I did:

1. Create a directory in the Library folder of my home directory called “LaunchAgents”
2. Create a “plist” XML file and save it in this directory.  I used the name “com.nathanbibb.dropboxrsync.plist”.  Here is the XML file I created:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC -//Apple Computer//DTD PLIST 1.0//EN http://www.apple.com
/DTDs/PropertyList-1.0.dtd >
<plist version="1.0">
<dict>
        <key>Label</key>
        <string>com.nathanbibb.dropboxrsync</string>
        <key>LowPriorityIO</key>
        <true/>
        <key>Program</key>
        <string>/Users/nathanbibb/Library/Scripts/dropboxrsync</string>
        <key>ProgramArguments</key>
        <array>
                <string>dropboxrsync</string>
        </array>
 <key>StartInterval</key>
        <integer>30</integer>
</dict>
</plist>

This file tells the launchd program to run my “dropboxrsync” script (Program and ProgramArguments both do this – I am not sure why you need both of these, but apparently you do) every 30 seconds (StartInterval does this), keep this a low priority task in case the processor is doing something importnat (LowPriorityIO does this), and track this activity in launchd as “com.nathanbibb.dropboxrsync” (This is the label).  The format of this label is using the Apple convention of “reverse DNS”.

So now I have an rsync script that runs every 30 minutes keeping my Dropbox folder in line with my chosen folders in my Docments folder.  The only real downside here is that I have to keep the files in two locations on my local hard drive.  If I ad access to a public rsync server, I suppose I could set up the same rsync script to sync my files with a different server rather than using Dropbox, but I would need command line access to my web server, I think, which I don’t have on my current server.  So we’ll see how long this whole crazy process lasts.

Here are some of the really helpful sites I found that got me where I needed to go:

http://developer.apple.com/MacOsX/launchd.html

http://www.macresearch.org/tutorial_backups_with_launchd

And some pertinent man pages (these are accessible from the Mac Terminal.app interface):
man rsync
man launchd
man launchd.plist
man launchctl
man plist

Cheers!

I’ve heard a bit about Pandora.com, but I haven’t tried it out until today, spurred on by Richard Friedman’s post about his interesting results from a Kyle Gann station there.  I’m a pretty big fan of Gann’s music, but I was only two songs in to this station before I could no longer resist creating a Sun City Girls station.  Maybe it’s the alure of the forbidden – SCG is banned in my house by my wife.  Too bad there’s no Caroliner station in their database.  But then, Caroliner never really could be digitized…

I am really excited about the new version of Lilypond that was released last month.  On Leopard you still have to go through some hoops, but I’ve gotten pretty comfortable with that – I’ll try to post my process shortly.

The most exciting thing for me, though, is not the updated code.  It’s the massively updated documentation, which I’ve been reading through the last few days.  It answers a lot of questions I’ve had and opens up some interesting compostitional ideas for me.

Now it’s time to go back to my well worn copy of Gardner Read and brush up on my notation skills…