Uncategorized

Having recently switched to a laptop for my primary machine, I spent a bunch of time reworking my storage strategy for all my photos and other files. Below is my attempt to document where I landed. Maybe it will help someone else.

First a description of my requirements:

  • I need a relatively large amount of space for photos. Currently my entire Lightroom library is about 500G, but grows at a rate of 2-8 gigs every week.
  • I have a bunch of other files that I need reliable storage for. All my scanned documents. Videos of my kids. Old VM’s with data in them. This amounts to around 300G
  • I have just under 1TB of a bunch of other junk. Movies, music, etc. which I don’t really need backups of, but might as well if I have the space. 
  • I want/need all of 1) a bootable backup, 2) local “onsite” backup in my house that can keep up with the rate that my photo library grows, and 3) an offsite backup.

On my old desktop, I had one 128G SSD for OS and Apps, and one 2TB internal drive for everything else. I had a 2TB external USB drive for local backup, and I ran Crashplan in background constantly to keep everything synced in the cloud.

Photos

The first piece I needed to figure out was what to do with my photos. I splurged on the laptop and got 512G of built-in SSD. But that’s not enough to fit my library. So I ended up having to split my photos between my main drive and an external.

For an external drive, I wanted one that wasn’t chained to my desk. I found the mac edition of the 2.0TB Western digital my passport drive on sale for $125 so I added that to my collection.

Lightroom is actually quite capable at managing split libraries. All I had to do was migrate my photos directory and catalog over to my new drive, open Lightroom on the new machine, and tell it about the directory. Then I set the import settings in light room to a location on my SSD. With this setup, I can do new imports/edits/exports without the external drive attached. I’ll use more and more internal SSD space until I accumulate enough photos, at which point, I’ll take the oldest ones and move them to the external using Lightroom’s builtin folder management UI.

Lightroom seems to be super-smart about what to do when the external drive is not plugged in. If I previously looked a file on the external and generated a thumbnail for it, that will still appear in the Library view. All the metadata is there as well.

Backup

Backup was a bit trickier. There are two main problems:

  1. A bunch of my photos are on the external drive. I want those photos backed up both locally and in the cloud, but my drive is not connected to the laptop most of the time.
  2. Cloud backup is slow. In addition to the external not being connected to the laptop most of the time, I put the laptop to sleep quite often, which seems to interrupt Crashplan’s backup, causing it to never finish.

To get this all working, I ended up having to enlist my second machine (an old 2009 mac mini hooked up to the TV).

First, I moved everything that was on the 2TB internal drive on my desktop to the external laptop drive (which also had my photos).

Second, I moved the 2TB external desktop drive (which I was using to backup my desktop) over to the mac mini, reformatted, and used it to store time machine backups. My laptop now backs up regularly over wi-fi to this drive, including the external drive when it’s plugged in. (Wi-fi N seems to be fast enough to keep up with things after an initial sync that took a full night).

Third, I had to not rely on Crashplan running on my laptop. So I took the 2TB internal drive that I had in my desktop, put it in a case, and attached it as another external drive on the mac mini. I then use Carbon Copy Cloner (which is basically just a fancy front-end around rsync, but solves a bunch of usability problems) to copy over a subset of data from both my laptop’s SSD and external drives to the added space on the mac mini.

Because Carbon Copy Cloner is rsync-based, the copies it makes appear as normal files on the mac mini. Now I can just run crashplan on the mini to push things to the cloud in the background. CCC also has some niceties like running scheduled backups, and being smart about when data sources (like external drives) are not available. It’s a little work to fight with the UI, but once it gets set up, it seems to work reliably.

To summarize:

  • I use Lightroom and split a large library between internal laptop drive and an external drive. I accumulate new photos on the internal drive until it fills up and move them over to the external.
  • I use time machine for local backup of both the SSD and external drive. 
  • I use Carbon Copy Cloner to copy files from both internal and external drives over to a machine which is always connected and can run cloud backup constantly.

A little convoluted, yes. Ultimately, I’d love if Apple could just do a bunch of this for me. Perhaps something along the line of Time Machine write-through to the cloud. I’d happily pay Apple more than I’m paying crashplan to have that just work. I’m not holding my breath though.