# How do you backup 20TB of photos offline?



## ph0enix (Jan 31, 2013)

I got a call from a friend who is a pro photographer this morning.  He's always struggling with backups of his photos that he's been accumulating for years/decades (around 20TB at this point).  He was recently able to consolidate about 60 external hard drives that he'd been using for backups into a few RAID-5 units but his data still isn't really safe.  Even if he keeps spare drives for the RAID gizmos around and replaces bad drives as soon as they fail, a faulty controller, simultaneous, multiple drive failure or a natural disaster could take all of his life's work away in a heartbeat. 

I understand storing files on multiple computers and in multiple locations. It's not that difficult when you have a few hundred GB or even a couple of TB worth of data but 20TB (and growing) gets difficult.  The problem with storing hard drives on shelves is that they may not turn on when you need them 7 or 18 years from now.  Also the USB, Firewire, eSATA, Thunderbolt or whatever other interface that's commonly used now will not be around forever and getting access to a computer that an outdated hard drive can be connected to is going to be a challenge in the future.
How do you do it without putting your estate up as collateral for a photo storage solution?

Edit: meant "off site", not "offline" in the subject.


----------



## gsgary (Jan 31, 2013)

Shoot film


----------



## KmH (Jan 31, 2013)

Many are discovering that digital storage has some issues compared to older technology.

Your friend is about 16TB late in considering his/her digital image archival needs. 

I converted all my digital archives from magnetic and optical media (hard drives/CD/DVD) to SSD.

I expect my digital archives will have to be redone periodically as technology changes.


----------



## Village Idiot (Jan 31, 2013)

Redundancy. That means a lot of money for that much data. You can go with an internet back up service, but like your example of connectors going obsolete, there's no guarantees with online back up services (look at the photo company something or other rail-road that folded and left major corporate agencies in the dust when it came time to retrieving files). 

The best option would to use NAS at home and have a way to store backups off site as well You could set up a similar NAS system, but when you're getting into using that many drives, it gets expensive. Maybe he could find some used enclosure or something. HDDs will either be cheap for a lot of smaller capacity ones or expensive for high capacity. It would take 10 2TB drives just to reach 20TB and that's not giving him any room to grow. You'd need even more. Other than getting a cheap computer case that could handle 10 drives and making it into a server or disk farm he's going to have to look at enclosures and they're in the thousands of dollars. They make 4GB HDDs and you'd only need 5 of them, but you're looking at $400 each, so that's $2,000 just in disks to handle what he already has and that's not with an off site solution.

Biggest suggestion, cull everything he absolutely doesn't need and then try again.


----------



## ghache (Jan 31, 2013)

2 copy of shelf hard drive is the way to. once the technology is near EOL, move the data to new technology. 

dlt tapes is also an option but required alot more technical understanding and expertise than regular backup on hard drives.

in the meantime, SATA drives are still here for years. there is no way SSDs are going to be the only standards soon. to many people still use sata hard drives, this technology is not dead. when you keep that amount of data, nothing is cheap. 3-4 TB is nothing compared having multiple copies of 20 TB of ****. this is getting expensive. this is a business expense enyway

Like BH sad, there is probably 25% of that crap he can probably get rid of.


----------



## DBA (Jan 31, 2013)

He may benefit from an online service like CrashPlan. I use their business "1 PC unlimited data" plan (I currently have ~350GB backed up) which is ~$7 per month. I've been told that if you have a large amount of data to backup the first time you can send them the hard drive(s). They'll transfer the data to your account and ship the hard drive(s) back. (I'd hate to attempt to backup 20 TB over the internet). Plus if you loose all your local files they can also send you hard drive(s) with your data on it, instead of downloading 20 TB of data.
Enterprise Data Backup ? Online Backup Software, Cloud Backup ? CrashPlan PROe

Another option would be to store a set of backup hard drives in a bank safety deposit box. This obviously wouldn't work as an instant backup but a way to store the majority of the files off site.


----------



## bhop (Jan 31, 2013)

20TB of files?  Does he save every snapshot?  That's crazy..


----------



## cgipson1 (Jan 31, 2013)

For that much data, he needs to look into standard commercial SCSI array solutions. They aren't that expensive... compared to what the loss of the data would be. 

There are also some Commercial Online Storage / Data Center facilities that will not be going away.. that provide both storage and offsite backup capability. And the cost will be significantly less than trying to put something together himself. Dell, for instance.. has the capability of providing that service (as they already do to a lot of really LARGE players, as well as small ones! They have proven to be reliable and cost-efffective.) For 20 TB, or more, they could arrange to have the data loaded wherever needed, and he could then do more data appends via the internet as needed.

It all depends on what he wants, actually needs, and what he can afford.  

Home level NAS systems are not something I would trust with something really important.


----------



## ghache (Jan 31, 2013)

bhop said:


> 20TB of files?  Does he save every snapshot?  That's crazy..



i have 3 TB and im only shooting for a few years, not even full time.

I can imagine a full time photographer who switched to digital 10 years ago, shooting everyday, stacking that amount of data.


----------



## gsgary (Jan 31, 2013)

Do i need to back up my floppy discs ?


----------



## Mully (Jan 31, 2013)

He needs to edit... the need for many duplicates of older work is unnecessary.  He should back up 1T at a time because few machines can handle a 20T back up.


----------



## dmunsie (Jan 31, 2013)

Edit-delete-repeat as needed. The past year I have been saving every shot as well, and spent the last couple of days deleting images I didn't need. I can't imagine having to go back a decade or more. wow... makes me appreciate the need to spend a little more time on each shot and get it right the first time or so.


----------



## ph0enix (Jan 31, 2013)

dmunsie said:


> Edit-delete-repeat as needed. The past year I have been saving every shot as well, and spent the last couple of days deleting images I didn't need. I can't imagine having to go back a decade or more. wow... makes me appreciate the need to spend a little more time on each shot and get it right the first time or so.



Let's not make assumptions.  The guy is a fairly world-renowned photographer.  I'm not saying super-famous but pretty known.  I'm sure he gets it "more right" on the first try than most people do.  He's been doing it for decades and has made a pretty decent living at it.  I'm not going to question his ability or his workflow.   He said he just came back from a month-long trip to Africa and brought about a 100GB worth of photos back.  It's not that much if you think about it.  His wife is his partner (also a photographer).  If they both shoot 50 photos a day each for 30 days, that's 3,000 shots total.  Let's assume that they don't use super high MP cameras and the raw files are 30MB/pc, that's 90GB.  That's just the unedited versions.  If they want to keep all the RAWs ("negatives", if you will), the amount of data will add up pretty quickly.  There might be other photographers that work for/with them that I'm not aware of.


----------



## snowbear (Jan 31, 2013)

gsgary said:


> Do i need to back up my floppy discs ?


Just copy them to cassette tapes, and you'll be fine.  Now, where did I put that shoe box of punch cards?


----------



## IByte (Jan 31, 2013)

gsgary said:


> Do i need to back up my floppy discs ?



Better get on that laser disc son you are a we bit late.


----------



## dmunsie (Jan 31, 2013)

ph0enix said:


> dmunsie said:
> 
> 
> > Edit-delete-repeat as needed. The past year I have been saving every shot as well, and spent the last couple of days deleting images I didn't need. I can't imagine having to go back a decade or more. wow... makes me appreciate the need to spend a little more time on each shot and get it right the first time or so.
> ...



The assumptions were for my own limited skills and shots. I've quickly accumulated alot of images and I still...can't fathom how someone could go through decades worth of material. So for me personally...I've quickly learned that post organizing / processing is much more time expensive than I realized so instead of taking a plethora of shots each shoot, I'm spending much more time on composition and taking much fewer shots.


----------



## GerryDavid (Feb 5, 2013)

I shoot portraits, and I've been keeping every shot on three external hard drives, when those get full, I get 3 more.  I am thinking of changing things.  I rarely get reorders so it seems like I am keeping these for no reason.  Its just taking up space, money and time.  If I do get a reorder, it will probably be a print that I have already processed, not one that didnt make the first round of selections.

In my picture viewings, I will show them the pictures, say 100.  that gets narrowed down to 50 and then to 25.  I am tempted to just erase the ones that dont make the first and/or second round of edits freeing up a ton of space.

I am even considering keeping the files for 30 or 60 days and then anything that doesn't get ordered gets erased.  So out of the 100 files, I will be keeping between 3 and 20 pictures.  That way they wont think "Ill order them next year" which never comes.

Then the pictures that get ordered will take up a fraction of the space compared to all of them would have, and could even be backed up to a cloud.

Any thoughts?


----------



## D-B-J (Feb 5, 2013)

G-Technology 8TB G-RAID Professional High-Performance Dual-Drive Hard Drive - Apple Store (U.S.)

800 for 8 TB?  Not bad...


----------



## Ilovemycam (Feb 5, 2013)

ph0enix said:


> dmunsie said:
> 
> 
> > Edit-delete-repeat as needed. The past year I have been saving every shot as well, and spent the last couple of days deleting images I didn't need. I can't imagine having to go back a decade or more. wow... makes me appreciate the need to spend a little more time on each shot and get it right the first time or so.
> ...



Reminds me of Moose Peterson. He has about that much stuff. Saves everything. 

 I don't think there is a photog on earth that has that many great keepers to fill up so many drives. 

For my own stuff, I try to just keep exceptional photos and some sentimental stuff here and there. (I don't always keep up with the cleaning out though.) But every photog is different with the picture hoarding. 

He could make 4 x 6 master prints of some of them and scan the back up prints if his drives die. Or he could back up on gold DVD pairs and have a ton of DVD's. 

I'm glad I'm not in his shoes. He inspires me to go clean up some files!


----------



## Raian-san (Feb 6, 2013)

Don't really know why someone would back up 20 tb of data. You just save some of your favorite ones and delete the rest. I shoot video so every wedding is 200-400 gb of files. This is why I give my clients the raw footage and they can save it. If I need it in the future I can ask them for it but no point of my saving it.


----------



## bratkinson (Feb 10, 2013)

Unfortunately, computer storage technology has been changing faster than one expects.  30 years ago, the first hard drives were RLL and MFM (I've since forgotten what the letters mean), SCSI also came along about then.  Then IDE hit the street.  Made hard drive management lot easier as I didn't have to worry about interleave-factor, low-level formatting, etc.  In the past 7-8 years, we've gone through SATA, SATA II and now SATA III. Who knows what there will be 10 years from now?  

As for off-site storage, it used to be 160K (yes, K!) floppies, then 360K, then 1.2m, 2M ZIP drives, various-size tape drives, CDs, now DVDs, and finally, 'the cloud'...if you can trust someone else to keep your data YOUR data, and, to BE THERE 10 years from now...

For my lowly 1TB requirement, I have a 1TB USB drive.  If I need more, I'll buy another 1TB for less than $100.  At least the price keeps going down.  2 years from now, a 10TB will probably be in the $100-200 range.  When the price and the technology changes again in 3-4 years, I'll probably go to that technology.


----------



## Patriot (Feb 10, 2013)

He waited too late IMHO. That's something he.should have started when his collection was smaller.


----------



## vtf (Feb 10, 2013)

try beta


----------



## rexbobcat (Feb 10, 2013)

bratkinson said:
			
		

> Unfortunately, computer storage technology has been changing faster than one expects.  30 years ago, the first hard drives were RLL and MFM (I've since forgotten what the letters mean), SCSI also came along about then.  Then IDE hit the street.  Made hard drive management lot easier as I didn't have to worry about interleave-factor, low-level formatting, etc.  In the past 7-8 years, we've gone through SATA, SATA II and now SATA III. Who knows what there will be 10 years from now?
> 
> As for off-site storage, it used to be 160K (yes, K!) floppies, then 360K, then 1.2m, 2M ZIP drives, various-size tape drives, CDs, now DVDs, and finally, 'the cloud'...if you can trust someone else to keep your data YOUR data, and, to BE THERE 10 years from now...
> 
> For my lowly 1TB requirement, I have a 1TB USB drive.  If I need more, I'll buy another 1TB for less than $100.  At least the price keeps going down.  2 years from now, a 10TB will probably be in the $100-200 range.  When the price and the technology changes again in 3-4 years, I'll probably go to that technology.



Pshaww, in 2 years everyone will have 10 TB of cloud storage and hard drives will be obsolete.

You won't need a personal laptop or anything because you'll be able to go to any computer anywhere and log into your cloud account to access your desktop, programs, and files.

THE FUTURE!!!!!!!


----------

