# Is there a middle-ground for archiving between RAW and JPEG compression?



## RAW (Mar 2, 2014)

Hi everyone 

I am trying to consolidate the better half of a terabyte of data from an external hard drive and my laptop's internal disc storage as well... and just a few sporting events I recently photographed are taking up nearly 100gb of data, alone.

*QUESTION: I was wondering if there is a nice in between file-type I could convert all of my RAW images to, before either storing to my hard drive(s) and/or burning onto physical media like DVD-Rs, so I don't use all of my storage so quickly?* My RAW images are between 20-30mb each, and it adds up ridiculously quick. I know RAW is king when it comes to loss-less quality and photo editing, so I'll keep the images I really like in their native format... but what about everything else? 

JPEG downsizes everything *SEVERELY *from 25mg to 1mb, so I'm wondering _*if there's something I'm missing that I can convert my photo-archives to so I can compress them without TOTALLY sacrificing quality, in case a friend or client wants copies of them in the future etc.?*_

THANK YOU SO MUCH EVERYONE!


----------



## 480sparky (Mar 2, 2014)

Memory is cheap.  Just buy a 3tb hard drive.


----------



## vimwiz (Mar 2, 2014)

RAW said:


> JPEG downsizes everything *SEVERELY *from 25mg to 1mb, so I'm wondering _*if there's something I'm missing that I can convert my photo-archives to so I can compress them without TOTALLY sacrificing quality*_*[*/QUOTE]
> 
> 
> Depends, what JPEG options are you using?
> ...


----------



## 480sparky (Mar 2, 2014)

Taking the time to shoot raw, only to save everything as a JPEG, is kinda like shoot film, making 4x5 prints, then tossing out the negatives.


----------



## The_Traveler (Mar 2, 2014)

Throw out the bad pictures that you'll never look at or need.


----------



## RAW (Mar 2, 2014)

480sparky said:


> Memory is cheap.  Just buy a 3tb hard drive.



That's really not helpful, at all. 



vimwiz said:


> Depends, what JPEG options are you using?
> 
> A full-res jpeg image at best quality is still tens and tens of megs, you must still be compressing heavily?



I use Irfanview for a good deal of my compression - it is pretty strong compression. Maybe rather than saving to a specific file-type, I could simply use Irfanview or many other programs to choose a file-SIZE in stead... opting for about 10mb for each file, cutting my storage usage in half (?)



480sparky said:


> Taking the time to shoot raw, only to save everything as a JPEG, is kinda like shoot film, making 4x5 prints, then tossing out the negatives.



Sparky, I'm not sure if you're just not inclined to give new members a cold welcome, or if you didn't take the time to read my entire post before offering your sarcastic short-sighted remarks? Let me explain it again, with some more detail. I have a lot of external storage, however, I don't want to use it all on keeping 100% OF MY RAW images in their raw format. However, what WOULD like to do, is keep my favorite images in their RAW format, but for other images that aren't quite deserving of RAW but are still useable and worthy to be saved, convert/compress those somehow with whatever option is best, to a lower FILE SIZE. Since I've taken a few thousands photographs just this year already at different events and outings, even the terabytes of data I do have are quickly being absorbed... SO... for the images that I need/want to keep on file for clients or for my own purposes and projects that are not in need of remaining RAW, what would a great option be to perhaps cut their size in half.


----------



## Light Guru (Mar 2, 2014)

480sparky said:


> Taking the time to shoot raw, only to save everything as a JPEG, is kinda like shoot film, making 4x5 prints, then tossing out the negatives.



Yup. 

Hard drives are CHEAP!!!

I love hearing people complain about photo file sizes. I shoot 4x5 film and a good scan of one of my negative is around 1GB.


----------



## RAW (Mar 2, 2014)

The_Traveler said:


> Throw out the bad pictures that you'll never look at or need.



This could take... maybe a month or two, if I went through them all one by one.

I'm beginning to think I might be better off going elsewhere for what certainly is a simple answer? I can't imagine some of the more senior members (or any member at all) of all the other forums I'm on or moderate, ever taking something so simply (downsizing images) and injecting their own snide useless tongue-in-cheek rubbish... NONE of which gets the new member an inch closer to figuring out the very normal and very basic question they posed. 

1. Buy more space! (it's cold, and my money tree hasn't been blooming properly)
2. Delete your pictures! (oh my gosh, NO JOKE, REALLY? This is amazingly insightful and helpful)
3. What are you an idiot, shooting in raw and then converting? (you didn't even read my whole post, because a few sentences is hard to get through I suppose)
4. I love hearing people complain about storage space, I shoot 4x5... droning on (oh, so now I know some of your interesting life story, and still have zero answers to MY ISSUE -- but please, tell me what else you love and how much space your negatives take up. By the way, it's 2014, I could also say "Using negatives in 2014 is like using a donkey to pull a BMW to work!" But, I'm not rude for no reason like EVERY person here so far)

Nice... thanks!!!


----------



## BillM (Mar 2, 2014)

What about taking a bunch of those raw files and zipping them for storing.


----------



## RAW (Mar 2, 2014)

BillM said:


> What about taking a bunch of those raw files and zipping them for storing.



Hi Bill, thank you for the first useful and sincere post. This is a great idea.


----------



## 480sparky (Mar 2, 2014)

RAW said:


> 480sparky said:
> 
> 
> > Memory is cheap.  Just buy a 3tb hard drive.
> ...



I didn't realize giving factual, honest answers is considered 'cold'.  My point is: instead of trying to save memory by saving smaller files (which is a result of tossing out data), why not just buy another hard drive?  Memory is cheap.  A couple terrabytes would probably cost less than the time it would take you to convert all your images.

OK, what are you future plans for the images?  Will you even look at  them again?  Or is it possible you may get into selling your current  archive of work?  If you have any sort of future plans for your images,  keep the raw files.  All of them.  I've sold images I _personally _thought were crap and never even star rated nor edited it.  And had I not had access to the original raw file, trying to edit an 8-bit JPEG would have failed miserably and cost me far more money in a lost sale than another hard drive did.  Had I tossed out all the images I didn't star rate, I'd have an archive of only a couple thousand images.  And much fewer sales.

Yes, I delete the OOF, blurry, over- and under-exposed shots.  But if it's sharp and well-exposed, I keep it.  But that's just me.

_Short-sighted_?  Wow.  I thought we were talking about how to properly  archive your work.  But if you want to toss most of your hard work away, that's up to  you.


I guess if you're only willing to listen to what you're predisposed to hearing, I'll leave now.


----------



## Light Guru (Mar 2, 2014)

RAW said:


> I'm beginning to think I might be better off going elsewhere for what certainly is a simple answer?



The simple answer is what you want to do is not the best option. 



RAW said:


> By the way, it's 2014, I could also say "Using negatives in 2014 is like using a donkey to pull a BMW to work!" But, I'm not rude



Yes it is 2014 and yet no digital camera on the market can produce a image with the resolution of a large foe at negative plus the movements of a large foe at camera give you more control over your image then any DSLR.


----------



## 71M (Mar 2, 2014)

Raw's _the one_ for correcting white balance, denoising, knocking-out colour aberrations etc. The other formats are problematic in these respects. Using TIF wouldn't be much of an economy generally. If you're happy with how things look, you could opt for 100% quality JPEG, (4:4:4, YCbCr?).


----------



## table1349 (Mar 2, 2014)

OP.  If you want a simple answer perhaps it would help if you told us what answer you want to be given and someone here can parrot it back to you.  

You have been given several simple answers for doing it the correct way.  The fact that you don't like any of them is not the fault of the posters.  Storage space IS CHEAP.  If you don't want to just buy more storage and save the original files then pick a format.  Jpg with no compression, dng, tif, what ever you want and use it.  


Be advised, a few weeks from now when you come back asking how to take a photo or photos in that format you choose and fix some issue with it because you want to print it in a bigger size and it looks bad when you do, or you want to send it in to some contest and you can't make the file size big enough for them with out ruining the quality of the photo.  You were given recommendations for the best options and choose to ignore them.  Then you will actually see "cold" "not helpful" answers.


----------



## Derrel (Mar 2, 2014)

Are you shooting in 12-bit, compressed RAW format? THAT can save a lot of disk storage space compared to say, 14-bit uncompressed RAW. Not sure what camera you have or what its options are. As far as JPEG storage size, Nikon offers a choice between a FIXED size, OR allowing the camera to writer as large a JPEG as is needed for maximum quality for the subject matter.

Adobe's DNG Converter saves a significant amount of disk space on Fuji S2,S3, S5 Pro .RAF raw format files, due to the very odd,odd way those files are made, but I don't think the same is true with "standard" Bayer-array files from Nikon and Canon and Sony d-slrs.

Around a decade ago, Kodak's professional FF d-slr's used to offer a hybrid RAW/JPEG file type, called *ERI JPEG*, for Extended Range Imaging, which allowed a significantly greater than normal recoivery/adjustment/editing capability compared with standard JPEG files, but I think those HAD TO be made in-camera with one of the Kodak cameras. The idea has probably been sold, along with so,so many Kodak patents in the recent fire-sale of Kodak IP, but I suppose you might be able to research ERI JPEG and see if that might be able to be applied as a conversion from other files.


----------



## jaomul (Mar 2, 2014)

You could download Adobe dng converter and process your raw files into lossy dng, basically a cross between raw and jpeg in that you still retain raw functionality once you use an Adobe based software to process but there is a degree of compression. It can result in files about quarter size

Edit.Derrel got there first


----------



## Gavjenks (Mar 2, 2014)

You don't need a middle ground, because archiving every photo you've ever taken is dumb, and the amount of space the actual keepers take up should be fairly trivial on hard drives and DVDs (or clouds).
And if it causes you any issue in space problems, archiving all of your keepers in RAW is also frankly a little dumb, too.  There are a lot of photos that you should pretty much know you'll never want to edit again.

So 3 categories of photos:
1) Non-keepers. Don't keep any copies of these anywhere.
2) Keepers that you have no earthly reason to want to edit again: JPEG
3) Keepers that you think you may need to re-edit (especially portrait/wedding client stuff): RAW

And the tiered nature of the above should easily keep sizes down to being very affordable. If, let's say, the ratios of the above are something like 60%, 30%, 10%, then the average amount of space a photo will eventually take up at the time of snapping = about 4 megabytes. Which means that you have to take about 1,000 photos per DVD, and something like *quarter million* photos per modern hard drive.



> Taking the time to shoot raw, only to save everything as a JPEG, is kinda like shoot film, making 4x5 prints, then tossing out the negatives.


That's a terrible comparison. You need negatives to make more prints. Whereas you do not need RAWs to make more prints.
And in fact, RAWs _do not lead to better prints than jpegs do_, unless you have some need of re-editing for a different look (in which case save the RAW by all means, see above)





Bottom line: Jpegs were specifically designed as the smallest format that isn't detectible as low quality by the human eye for photographic images. So as long as the image looks how you want it, save it in jpeg, and you'll be fine. RAW is specifically designed for widest editing latitude, so save RAW if you need to edit.

It's just a simple matter of using tools for their intended purposes.


----------



## Nevermore1 (Mar 2, 2014)

I've read in multiple articles from various sources that .tiff files are the best option.  They don't lose as much file info as the conversion to .jpg does.


----------



## Gavjenks (Mar 2, 2014)

Tiffs usually take up about the same space as RAW


----------



## cheshirecat79 (Mar 10, 2014)

I'd personally suggest one of the following: 

- Using Lightroom to flag your photos for Pick or Reject. Reject those that, as another poster stated, will never amount to anything. Usually, blur is the worst offender. Turn on auto-advance and just use the x/p buttons to cruise through your library. I was able to get through about 15k photos in around 3 hours. It's tedious, but it'll help cut down on clutter. 

- ZIP your RAW files as another poster mentioned. Keep high-quality JPEGs as your "live" photos wherever you may keep them. 

- I know you mentioned that you didn't want to get into this, but entertain the possibility of taking the RAWs off disk and enter them into cold storage. Get a blu-ray burner and start archiving- you can get 25gb on a disc in a pretty short amount of time. Keep these masters in a very safe place. 

Unfortunately, RAW is really the only option when it comes to no-compromise editing. Keeping the RAW files of your final pictures is good practice- you never know when you may need them or when you may learn a new processing method that might turn your old ho-hum picture into something really special. 

I wouldn't recommend TIFF as it's just an uncompressed graphic- your storage concerns will not be addressed with this method.

tl;dr: delete the useless photos and take the raws offdisk once you make a high-quality jpeg export.


----------



## vimwiz (Mar 10, 2014)

> I wouldn't recommend TIFF as it's just an uncompressed graphic- your storage concerns will not be addressed with this method.



Not true, TIFF supports losless compression, pretty big, sure, but less than a RAW.


----------



## robbins.photo (Mar 10, 2014)

RAW said:


> Hi everyone
> 
> I am trying to consolidate the better half of a terabyte of data from an external hard drive and my laptop's internal disc storage as well... and just a few sporting events I recently photographed are taking up nearly 100gb of data, alone.
> 
> ...



Have you considered compressing the RAW image files with either ZIP, 7ZIP or RAR?  7ZIP or RAR would give you the best compression, ZIP would be the easiest to work with because newer versions of windows would allow you to open the file easily and get the RAW image inside. 

Just a thought.


----------

