# Backup Strategy



## WhiskeyTango (Jul 10, 2012)

I have no idea where to put this, so if I'm in the wrong place, I'm counting on the mods to move me...  (apologies in advance)

I'm considering a backup strategy involving a 2 drive RAID 1 array.  I'm thinking about buying (3) 2Tb drives, (2) of which will be in the array at any point in time, while the 3rd is in a safe deposit box offsite.  On a periodic basis (weekly/bi-weekly/monthly, etc.), I'd pull (1) of the drives in the array and swap it for the one in the safe deposit box, then rebuild the mirror.

Thoughts?


----------



## Dao (Jul 11, 2012)

I do not think it is a good idea for breaking the array and rebuild it.  You may be better off get a USB 3.0 dock and 2x internal drives (or 2x USB 3.0 external drives) and use them as backup.  Rotate them and store one as off site backup.  You can sync the files over with a script of 3rd party software.  In Windows, you can use syncback.

Here is what I have.

- RAID 5 setup (more storage)
- A external USB 3.0 drive for backup
- Another internal drive on the USB 3.0 dock for another set of backup that I keep in my work place.  Bring home once in awhile for backup.


----------



## nmoody (Jul 11, 2012)

I have a main desktop which gets backed up by a Windows Home Server 2011 to a 2TB raid5. That Array also gets backed up to another 2TB raid1. This is a bit costly but does have some nice redundancy.

Stuff that I find supper important gets put in my dropbox which is an offsite backup.

I hear a lot of people on this forum also use Carbonite for an off site backup. This one works well and is reasonably priced.


----------



## BuS_RiDeR (Jul 11, 2012)

RAID 5 is probably the best way to go... But it requires either an internal RAID controller capable of RAID 5 or an external enclosure with a built in RAID 5 controller; as well as a minimum of 3 (or more) drives (all the same).

See... THIS.


----------



## WhiskeyTango (Jul 11, 2012)

I'd considered RAID 5, but I'm discounting it a bit for a couple of reasons.  The primary one is simplicity.  I like the idea that, in the event of a failure, all of the data is on all of the drives, and I can simply plug one of the mirrored elements directly into any computer to retrieve it.  With a RAID 5 array, I'd have to rebuild the array first.

I'm open to the non-RAID solution.  It's actually where I started...  On Windows, I'd used Microsoft's SyncToy as well as Karen's Replicator, to sync files between volumes.  I'll look into a Mac solution.  I have Delta Walker, which may actually have that functionality...

As for a dock, I've looked at Thermaltake BlacX and NewerTech Voyager.  I like that the NewerTech is Firewire, but it seems to get lower reviews.  Anything else I should be looking at?


----------



## BuS_RiDeR (Jul 11, 2012)

One nice thing about RAID 5...  you have 3+ drives...  One fails, you simply replace it...  you lose nothing. Its a very good strategy... But I understand your point...


----------



## KmH (Jul 11, 2012)

Get the inexpensive book - The DAM Book: Digital Asset Management for Photographers


----------



## WhiskeyTango (Jul 11, 2012)

KmH said:


> Get the inexpensive book - The DAM Book: Digital Asset Management for Photographers



Apparently I'm going to have to breakdown and order a hardcopy of that book...  I've been putting it off for a couple of months in hopes that there will eventually be a Kindle, or other e-book version, but alas...

In the meantime, I've read everything I can get my hands on, which means there's lots of competing opinions.  I'd love to hear yours.


----------



## Garbz (Jul 12, 2012)

BuS_RiDeR said:


> One nice thing about RAID 5...  you have 3+ drives...  One fails, you simply replace it...  you lose nothing. Its a very good strategy... But I understand your point...



This breaks down for two reasons. 
1. RAID is for high availability not for data security. A RAID system has MANY common mode failure problems that make it not even remotely a replacement for a proper backup solution. Simple things such as user error (deleting files), house fire, lightning strikes, etc all take out your entire "redundant" array. 
2. RAID5 has not been viable since drives started exceeding around 300GB. Statistically speaking you have half the chance of recovering from a RAID5 hardware failure than a RAID1 hardware failure with data intact. The only reason RAID5 exists is for scalability (i.e. it's much cheaper to make a 5TB RAID5 array than a 5TB RAID1 array. But conversely because of the size of dives it's cheaper to make a 4TB RAID1 array than a 4TB RAID5 array. 
On the failure modes, harddisks are likely to die due to common causes. They reach their Mean Time To Failure at the same time, they experience the same heat, the same load, and as such when a RAID member dies the remaining drives in the array are often on the edge of their life. Now replacing the failed harddisk results in almost a full day of continuous THRASHING of the disks. Very many RAID5 arrays fail during a rebuild which is exactly why most places with any kind of valuable data have moved to RAID6 (two parity drives).


Now as to what you should do: A backup needs to be separate from all common mode causes of failure. I.e. not connected to the computer, not stored in your home, etc.  Your safety deposit box is a good idea. But rather than swapping the drives and horrendously thrashing them you should do a mirror backup. There's free programs like Synchronicty which can do this for you. So when you bring your backup drive in at the end of the week for a refresh rather than copying some many hundreds of GB over again (which happens on a RAID rebuild) it'll scan the drives, check the last edited dates on files and copy only the new changes over. It's less stress on your drives and the backup time is cut from a day to probably half an hour. 

As for my earlier comments on a RAID failure the backup drive plays an important role there too. The first thing you do when you suffer a RAID failure is resist all temptation to rebuild the array. Bring in your backup drive and do a quick backup as described above. THEN use another drive to rebuild the array. That way if your RAID array falls over during the rebuild you still have current copies of all your data.


----------



## 480sparky (Jul 12, 2012)

If all your stuff is sitting on your desk (internal, external, and raid drives), what happens when your house burns down, or you have a tornado, or a flood?

If your work isn't at _three physically separate locations_, it doesn't exist.

Also, keep in mind safe-deposit boxes are not fire and flood proof.  And fire-proof safes will not prevent a hard drive from melting into a puddle of plastic with electronics imbedded in it.


----------



## Gaerek (Jul 12, 2012)

If you're looking for simplicity...

Drobo

It's a little spendy, but the benefits outweigh the drawbacks if you're looking at keeping your photos safe. Drop a drive in and forget it. They don't even have to be matched (like most RAID implementations do). If a drive fails, simply remove it and replace it with another. I'm planning to purchase a couple of these in the future, but at the moment, a bit out of budget.



480sparky said:


> If all your stuff is sitting on your desk  (internal, external, and raid drives), what happens when your house  burns down, or you have a tornado, or a flood?
> 
> If your work isn't at _three physically separate locations_, it doesn't exist.
> 
> Also, keep in mind safe-deposit boxes are not fire and flood proof.  And  fire-proof safes will not prevent a hard drive from melting into a  puddle of plastic with electronics imbedded in it.



And I'd like to second this. Having a single onsite backup of your primary data is a good idea, but make sure you have AT LEAST one backup moved off-site. There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.


----------



## WhiskeyTango (Jul 12, 2012)

The problem I have with Drobo is that it's a proprietary interface.  I'm less worried about individual drive failures in a Drobo system, but if the Drobo itself fails I can't recover until I've got a new unit...

The offsite storage of a drive is one piece of what I have in mind.

My primary, i.e. "working" drive is a 2Gb Raid1 array.  It is currently backed up to a single 2Gb external drive.  I haven't yet implemented offsite redundancy, and thus the question to this group.  I have considered a (3) site strategy, but am/was discounting it due to what I perceive to be minimal risk of catastrophic loss at (2) sites.  A 3rd drive and a firesafe is pretty cheap, though, so perhaps I need to reconsider.

Also worth noting: I'm a hobbyist who occasionally takes paid work.  I'd like "occasionally" to become more frequent, but I'm not trying to "force" that and photography is not/will not be putting food on my table or clothes on my kids anytime soon.  I can afford to scale up my backup strategy over time.  I simply want to start on the right road.

Garbz, I love the idea of doing a sync between the two backup volumes rather than a full copy over.  My only concern with that is that it puts both volumes, along with the primary drive, all in the same location at the same time.  Since all of my volumes are the same size, I wonder if I'd be safer to swap the volumes at the offsite location and then sync the stale backup drive to the current working drive.  Seems like I'd accomplish the same thing without ever putting all of my backups under one roof...


----------



## Gaerek (Jul 12, 2012)

People have given you all sorts of ideas and you always go back to your original plan. It seems like you have already made up your mind as to what you want to do. Continually breaking a RAID 1 is such a bad idea, not to mention extremely inconvenient. The less handling you have to do the better. You should only remove a drive after a failure. If your solution requires you to do it more than that, you're doing it wrong and you'll run into more issues down the line. People continually forget the fact that hard drives are 50 year old technology, and although the technology has gotten better and more robust with age, hard drives are still very fragile. A grain of dust inside, and you can potentially ruin the whole drive. Thus, the less handling the better.

You're going to come up with compromises with whatever it is you decide to do. There is no perfect, end all solution. Your concerns about the Drobo are certainly valid, but your solution is far more prone to failure than the actual Drobo failing. Anyway, whatever you decide, I wish the best of luck.

EDIT:

When it comes to backups, here's what you can get.

1. Cheap
2. Simple
3. Robust

Now, choose two. If you want it simple and robust, it's going to cost you. If you want it cheap, but simple, your solution will be prone to failure. If you want cheap and robust, your solution will be complicated and inconvenient. What's your data worth? I don't make money off my photography, but I consider my photos very valuable. If I have to choose (and this is unrealistic, but you get my point) between saving my camera and lens (several thousand $'s) or saving the 16GB memory card full of photos, I'll take the memory card, each and every time. This is why I won't skimp on a backup solution.


----------



## WhiskeyTango (Jul 12, 2012)

Gaerek said:


> People have given you all sorts of ideas and you always go back to your original plan.



Whoa, hang on.  Not at all.  My "original plan" was to ditch the 2Gb external in favor of a 2nd array, swapping an array element weekly.

I've definitely taken the advice given to heart.  Now my plan is to skip the 2nd array and simply swap out a set of 2Gb external drives.

I'm still VERY open to refinement, criticism, etc.  I don't ask questions on subjects that I'm closed minded on.  I apologize if I gave another impression.

Your point on spending for backup is also taken to heart.  I've got damn near 20k invested in this "hobby" now.  I don't mind spending money for a robust backup solution, I simply want to spend wisely and not foolishly.


----------



## 480sparky (Jul 12, 2012)

Gaerek said:


> ........There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.



I have a simple method.  My work is backed up on two external drives (these are _in addition_ to my 4-drive backup routine).  One is kept at a neighbors house.  This prevents me from losing everything in case my house burns down, a thief cleans me out, something like that.  But it's not fool-proof when it comes to floods, tornadoes, large local disasters like that. 

So a second copy is kept at a relative's house 872 miles away.  Now, unless we have something like this happen, I think I'm fairly safe.  And if we do, then I doubt I'd care about my stuff any more.


----------



## WhiskeyTango (Jul 12, 2012)

480sparky said:


> Gaerek said:
> 
> 
> > ........There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.
> ...



Now that's a strategy I love.

Out of curiosity, how often do you refresh the backup 872 miles away?  My in-laws have a place about 4 hours from me that I get to every couple of months, hell in hunting season I'm there every weekend .  I could definitely keep a 3rd backup there...


----------



## 480sparky (Jul 12, 2012)

WhiskeyTango said:


> Now that's a strategy I love.
> 
> Out of curiosity, how often do you refresh the backup 872 miles away?  My in-laws have a place about 4 hours from me that I get to every couple of months, hell in hunting season I'm there every weekend .  I could definitely keep a 3rd backup there...


----------



## Gaerek (Jul 12, 2012)

WhiskeyTango said:


> Gaerek said:
> 
> 
> > People have given you all sorts of ideas and you always go back to your original plan.
> ...



Noted. 

I may have misunderstood. I have ~10 years experience in IT dealing with backups and different things like that. In an enterprise environment, backups are so vitally important (and usually required by law to meet certain specifications, etc) that money is no object. But then again, when your potential loss is in the millions of $'s for losing data, you can afford expensive solutions.

Consumers, on the other hand, want a cheap solution. But they also want it easy to set up and use, as well as be robust. This is impossible. My recommendation is to spend as much on a solution as you can reasonably afford. If your budget is $100, well, you're looking at basically a single external hard drive and that's about it. I'm not sure what your budget is, or if you've even thought about it.

I seem to be pushing Drobo, but they really are an excellent solution. If you're worried about the actual unit failing, they have some really amazing warranty offerings. Basically, if the unit fails, they'll ship another one to you express the day you call, so you'll be back up and running with only a loss of a couple days. They are certainly spendy, but again, what's your data worth? Anyway, they are one of many options out there, but for someone who wants an easy, robust solution, there aren't many other options out there.


----------



## Gaerek (Jul 12, 2012)

480sparky said:


> Gaerek said:
> 
> 
> > ........There are several ways of doing this. Carbonite and Mozy both offer good solutions. There's other ways of accomplishing this as well, but those are the easiest.
> ...



How do you back it up? Are you able to remotely back up? Or do you have to send disc/drives? I used to send a DVD a month to my in-laws in Arizona, but I just use a commercial off-site solution now.


----------



## Garbz (Jul 13, 2012)

WhiskeyTango said:


> Garbz, I love the idea of doing a sync between the two backup volumes rather than a full copy over.  My only concern with that is that it puts both volumes, along with the primary drive, all in the same location at the same time.  Since all of my volumes are the same size, I wonder if I'd be safer to swap the volumes at the offsite location and then sync the stale backup drive to the current working drive.  Seems like I'd accomplish the same thing without ever putting all of my backups under one roof...



There's a risk reward case here. A few simple things can make this not a problem for you. Statistically speaking you're not likely to deal with a full house fire or flood or things like that. Unless you live in central Africa you're also not likely to suffer continuous sustained lightning strikes. Most data loss results from user error and failure of mechanical parts. For the user error, don't use the computer while doing the backup. 

At this point unless you're backing up nuclear launch codes you should be relatively safe. You get to a point as with everything else where to reduce your risk you start having to have considerable expense. You could do the backup remotely but at that point it's either out of your control, or the system is online for sustained periods which may open it to it's own failures. This is something I do at home. It's not a remote backup but it is another computer, but since the other computer also always runs the last hardware failure I had was actually on that machine. I still have a separate backup though. 


You could use dual disks for backups and alternate them every other week. So one day bring in one disk and do a full mirror, then the next week bring in the other. That way if you really suffer a bad catastrophe, your backups are only 1 week old.


----------



## 480sparky (Jul 13, 2012)

Gaerek said:


> ............How do you back it up? Are you able to remotely back up? Or do you have to send disc/drives? I used to send a DVD a month to my in-laws in Arizona, but I just use a commercial off-site solution now.




All my backing up / file copying is done on my desk.  I physically move the drives from their storage location to my desk.


----------



## unpopular (Jul 13, 2012)

If you are a professional, you should consider placing your backup on a server which is located outside of your main area, such as in a detached garage or storage shed, or (and perhaps better) use an online backup service. The reason for this is because if there is ever a fire you do not loose everything.

In the very least, archive old full hard drives in a safe location or fireproof safe.


----------



## 480sparky (Jul 13, 2012)

unpopular said:


> If you are a professional, you should consider placing your backup on a server which is located outside of your main area, such as in a detached garage or storage shed, or (and perhaps better) use an online backup service. The reason for this is because if there is ever a fire you do not loose everything.
> 
> In the very least, archive old full hard drives in a safe location or fireproof safe.




"Fireproof" safes will not always protect hard drives in a fire.  Yes, they may prevent _combustion_, but they *cannot* prevent heat damage.


----------



## unpopular (Jul 13, 2012)

I was thinking that after I posted. It's very much not the best idea.

I had a boss that lost everything in a fire. I suggested that he have a local server and a RAID mirror in a storage shed he had in back. You'd think he would have listened, but he refused and insisted on keeping everything in disorganized mess of DVD's. Some people just don't learn I guess.


----------



## TCampbell (Jul 13, 2012)

WhiskeyTango said:


> I have no idea where to put this, so if I'm in the wrong place, I'm counting on the mods to move me...  (apologies in advance)
> 
> I'm considering a backup strategy involving a 2 drive RAID 1 array.  I'm thinking about buying (3) 2Tb drives, (2) of which will be in the array at any point in time, while the 3rd is in a safe deposit box offsite.  On a periodic basis (weekly/bi-weekly/monthly, etc.), I'd pull (1) of the drives in the array and swap it for the one in the safe deposit box, then rebuild the mirror.
> 
> Thoughts?




That's not how RAID 1 works.  RAID 1 is "simple mirroring".  Meaning that it behaves as though you really own a single drive and anything written to disk 1 is also written to disk 2.  You cannot just unplug one of the drives to throw put in storage and pop another drive in... you'll break the mirroring and then the array will have to spend hours rebuilding the integrity of the RAID 1 filesystem.

RAID 1 is simple and usually inexpensive.  But since you buy 2 disks but only get to use the storage space of 1 disk, it has 50% overhead.  A RAID 5 system (which is more expensive) gives you N-1 effective storage space for N disks.  E.g. if you have 4 disks, then you get to use the capacity of 3 of them.  You're data really is spread across all 4, but it's spread in a way that if any single disk fails the array can continue to operate in "degraded" mode.  You get to replace the failed disk and the array will then rebuild the integrity of the filesystem (and you can continue to use it in degraded mode while it does this -- it will take hours and hours to complete.)



I use a different approach.  I have 3 different backups.   I have a Mac

1)  The Mac uses "time machine" (this is the built-in backup system on the mac).  It backs up the machine using an incremental backup once per hour.  You can restore any file to any point in time.  This is important (and has saved my bacon more than once.)

2)  I use a disk cloning backup called "Super Duper".  It essentially just creates a clone of your hard drive and then maintains the clone each time it runs.  It looks for any files that have changed since the last backup and applies those changes.  Effectively you end up with what appears to be a full disk backup but it creates this in the amount of time necessary to do an incremental backup.  The upside is that if my computer crashes, I can actually boot and run off this cloned drive.  The downside is it's not an incremental backup.

3)  I also create FileVault backups (this is a built-in feature of Aperture) and these are external disks that I detach and store.  It only backs up my images.  It is also NOT an incremental backup.

Why incremental? 

A while ago I had a drive fail... but not all at once.  It was a slow gradual failure.  Basically the drive started to lose it's accuracy in aligning with specific tracks.  The access arm that positions the head over a specific track was getting sloppy.  This meant that as it would "write" data, it was bleeding some data onto adjacent tracks and damaging whatever files happened to be stored there... but I didn't know it was doing this.  One day I went to open an image that I hadn't touch in about a year and discovered it was damaged and wouldn't render correctly.  I tried to restore the file from backup, but since the drive was failing "gradually", several backups had actually occurred AFTER the damage was done.  That means that even my BACKUP images were damaged.

If not for the fact that I used Time Machine as an incremental backup solution, all would have been lost.  I told Time Machine to go back a week at a time and re-inspected the images... it turns out I had to go back about 1 month before I could find a backup where none of the images were damaged.  I was REALLY glad I kept multiple backups which included incremental backups with a long history.


----------



## bratkinson (Jul 16, 2012)

Being a former mainframe IT consultant, I can readily affirm the necessity of not only HAVING a regular backup schedule, but also having restore capability immediately at hand, if needed.  One regional bank I worked at kept a continuous log of all online activity/updating, 7-day cycle of 8 hour major-file backups on site, and kept duplicate 24 hour backups, one on site, the other, about 20 miles away in a secure facility.  Losing track of someones' or some companys' money is NOT an option.

For the professional photographer, a regular backup schedule and methodology is also necessary.  Hard drives fail.  Some more than others.  And as noted above, they frequently fail s-l-o-w-l-y.  So a simple backup A and backup B methodology would likely both have the same damaged files.  But then, one must ask, why is 'forever' storage necessary?  It's not likely you'll ever need to retrieve photos shot 10 years ago and print more pictures.  To that end, I'd probably move any shoots that are more than a year old to duplicate DVDs stored in separate locations.

For the bulk of us non-professional photogs, I suspect anything more than a weekly backup of what your 'current' work is overkill.  Although hard drives are now less than a dollar per gigabyte (vs $250 for 30 Megabyte drive 25 yrs ago!) and RAID may seem an easy solution, from what I've read, RAID is not-so-easy for a computer novice to deal with, especially in the event of a failure.  

Through all my years in computers, one rule has always remained...KISS (Keep It Simple, Stupid).  And that's how I still do it.  My post processing methodology is still evolving, but here's what I do at present:

1. As 2nd or 3rd level protection from accidentally wiping out a picture or 5 while doing post processing, I do not delete/erase the memory card until ALL processing and delivery is completed on that shoot.  Memory cards are cheap insurance.

2. I also shoot in RAW + JPG, and transfer both sets to to physically different drives my computer. 

 3. I then work on only one set of RAW in Lightroom, and output as JPG to a different folder.  When all my LR work is done, I copy that output folder to another, and use the copy as input for Photoshop editing.  Some things are easier in PS than LR, so I use both, as needed.  Photoshop outputs replace the files they came from, as I still have the Lightroom output 'originals' in another folder.

4.  When all is complete, I make a copy of the 'finished product' to another folder, on another drive.  

5.  I print, make CDs or DVDs, or PowerPoint presentations from there.

6.  When everything is done, I move the RAW, LR output folder, and finished products to a long-term 1TB drive, deleting everything else on my 'in process' drives.  If it's a shoot I really have to keep 'forever', I may burn a DVD or two of the images.  

7.  The 1TB drive gets backed up to clone 1TB drive when convenient...hopefully, monthly, and keep it in my file cabinet at my office. 

8.  To facilitate making drive clones, I have each of my drives in 'hot swap' bays.  I simply turn off the computer, slide in the backup target drive, make a backup, and remove it.  SATA-3 transfers are so much faster than USB 2 or even 3 to an external drive.

In the event of incoming major weather problems, fire, riot, or anything else, I can slide all the drives out of all my computers in less than 30 seconds and be gone!  And, oh yes, ALL original purchased software CDs as well as downloaded software (purchased and free-ware) are copied on my long-term hard drive...as is a WORD document with -ALL- serial numbers, etc.  If I have to start over from nothing more than my HDs, I can.


----------

