# Challenge: Noticeable differences between RAW and a jpeg edited in 16 bit mode?



## Gavjenks (May 21, 2013)

1) The human eye can't even see all of the colors possible with an 8-bit jpeg.
2) Considering #1, the only real advantages of RAW are to be able to do edits with the full data range (thus avoiding banding, etc.), or for LOWERING the dynamic range (e.g., you took a photo with almost entirely midtones, and wish to stretch it out to have full contrast).
3) The issue with lowering of dynamic range can be fixed in-camera in most models by selecting desired contrast settings, etc., and the camera will use the RAW to do this simple calculation without any extra loss.
4) For more complex edits like curves, dodging, burning, etc., Photoshop is able to convert an image and perform all edits at 16 bit, and many types of edits at 32 bit. This does not magically create data that isn't there, but it *DOES *prevent things like banding quite successfully, even if the original image only has 8 bits of data. Pretty much no matter what edits you do within the remote realms of reasonableness, it will not degrade visibly in quality.

*Conclusion:* There shouldn't really be any technical reason why it is more useful to edit a RAW than to edit a jpeg after first converting it to 16 or 32 bit.

The only reason I can think of for why RAW would have an advantage would be if people have just bothered to write better software for RAW converters than Adobe has done for 16 bit mode filters and adjustments.  But I somewhat doubt this is the case, since Adobe is a company with vastly more experience at this sort of thing than random proprietary RAW software developers.

I tried this myself just now, shooting several sample images with RAW + JPG save mode, and tried editing some with RAW software, and others with photoshop (converting first to 16 bit mode).  I was unable to do any sort of normal edits that yielded results where I was able to visually tell the difference.

Can I tell the difference between RAW edits and 8 bit unconverted jpg edits? Yes.  But not with the 16 bit edits.

I even went to various websites that explain the difference between RAW and jpg, and I used their demo images: the very images that they took specifically to show the difference between the two formats in the most dramatic ways possible (things like pure gradients, and images over or underexposed by several stops), and again, was able to achieve quite comparable results either way. (Note that this is WITHOUT the benefit of being able to set appropriate contrast settings, etc. in-camera) Differences were noticeable in the shots that were poorly exposed by 2+ stops, but not slightly badly exposed ones.

I am curious as to how I might be proven wrong, in reasonable situations where you didn't completely fail your photo?


----------



## Judobreaker (May 21, 2013)

When you capture a photo in raw format the data will be saved in 12 or 14 bit (in Nikon cameras at least), meaning that there will be more data in the shadows/highlights which can be brought back in post processing.
The jpg file will not have this data because it is only 8 bit so it will not be possible to bring back these details.

Sure, the camera can do some processing itself according to some settings which will make sure these details are sort of brought back before saving the actual jpg... However this is done in a process determined by the company that developed the camera and might not be in a way I want it.
I'd like full control over my files. Doing the post processing myself will make sure I get photos in my own style.

Converting a jpg file to 16 bit will not bring back the detail in blown out or heavily underexposed areas, that data will still be lost forever because it was clipped during the in-camera jpg save.
This is why I work with raw.


----------



## Gavjenks (May 21, 2013)

You misunderstand the in-camera settings I'm suggesting.

I'm saying that you should set your camera to have NO auto adjustments (in my Canon T2i, this is done by setting "picture mode" to "faithful" or "neutral").  This makes it no longer do the sharpening and contrast, etc. that it normally does by default in converting to jpg. Or only minimally.

Then you do your own sharpening, color, and everything in post processing in photoshop, using 16 bit mode, and advanced tools that will do a better job of sharpening, etc. than your camera would (for example, sharpening copy layers to do it nondestructively and channel-specific, or whatever you want to do).

You're right that changing it to 16 bit will not bring back any detail, but what it will do is stop you from LOSING any of the detail you still have.  *And since the human eye is not precise enough to detect even half the level of detail captured by a jpeg, all you really need to do is to preserve the 8 bit detail you have.*




For example, here are two images of black white gradients:
The first one was generated as a 8 bit, then squashed into a smaller tonal range using levels, then expanded back out again, all at 8 bit.
The second one was also generated at 8 bit, but then converted to 16 bit mode, and the same edits were done.



Notice a difference?  The bottom one is perfectly smooth still. In fact, if you were to make a 16 bit gradient in photoshop (I can't display one here, because the file uploader wont upload 16 bit formats), you would not visibly see ANY difference, because your eye is not precise enough to do so.

However, you can indeed see a huge difference vs. the first image, because doing your edits in 8 bit is lossy. When you smash down the tonal range, there aren't enough "slots" to hold all the data, so it gets banded into bins. Then when you expand out again, the data can't be recovered.

*Bottom line: Editing in 8 bit creates banding and issues.  But the original 8 bit itself is just as precise as 16 bit as far as our eyes can see. In the bottom image here, I didn't create any new data. I simply protected the sufficient data that 8 bit already provides.*

So by setting your camera to do minimal or no processing while converting to jpeg, all it will do is throw out data that your eye cannot physically see anyway, and then by editing in 16 bit, you avoid losing any of that remaining data.  And the final result is something that (if you are good at editing) should be able to always achieve equally high _PRINT_ quality as a RAW.



The only time when you should "need" RAW is if you are cropping out 50-80% of your image or if you shot at the *completely* wrong white balance, or if you overexposed by like 3 stops or something. In other words: if you horribly Horrible HORRIBLY botched all your images. In which case, you have much bigger issues to worry about than file format...

If you only slightly botched your image though (1 stop off on exposure, 10-20% off on zoom/composition, etc.), RAW shouldn't offer any visible differences vs. clever camera settings and 16 bit editing.


----------



## Judobreaker (May 21, 2013)

While you are sort of right this goes wrong on a scene with higher dynamic ranges.
If you have a scene with dark patches of shadows and very light highlights you have no other option then to underexpose the shadows or overexpose the highlights (or in extreme cases both).
With a jpg file all the data in those under- and overexposed shots would be lost. With a raw file you could pull back a lot of that data.
It's basically what you do with HDR imaging, only HDR allows for an even broader range than raw files.

Yes, you can do pretty nice edits with jpg files, I've worked with loads of jpgs and editing in general works just fine. It's just that raw will enable you to capture more in terms of dynamic range.


----------



## Gavjenks (May 21, 2013)

Canon : Picture Style <--Neutral picture mode in canon, for example, is specifically designed to have a main priority of not throwing away that dynamic range data when converting to jpeg. To allow you to deal with situations just like you describe.

On default settings, the difference between RAW and jpeg often matters, a lot.  Because shooting jpegs in standard mode, your camera will auto-boost the contrast so that you have full representation from blacks to whites.  In doing so, it will often clip off the ends and lose a lot of the extreme detail, whereas RAW doesn't do this.

But if you simply tell it to not do that (by setting neutral picture mode on canon), then it will have the same default grayness that RAWs have, and will shy away on the histogram from both highlights and shadows, allowing you to choose in post-processing what threshold you want for clipping both ends, depending on what balance you want between detail and contrast.

If you set it correctly in settings + exposure, so that all photos just baaaaaarely dont reach either the black or the white ends of the histogram, then you will have all the data you need later (with just jpeg) to render whatever details you want, with only 10-20% adjustment, which is not enough to bring the images under the resolution capacity of the human eye (i.e., you wont visibly be able to notice the adjustment).

RAW does give you more options to go in extremes in one direction or the other in post processing, but it won't help you more than jpeg in BOTH directions at once (see above: if you sample evenly with neutral mode, then jpeg will already have more than the human eye's resolution from shadows to highlights). Thus, this feature is only a benefit if either:
1) You make a dramatic mistake like overexposing by 2-3 stops (hopefully this is very rare if you aren't clueless), or 
2) You don't know what you want the lighting your photo to look like when you shoot it.  Which is very very bad practice anyway.  A good photographer should consider the light before anything else, before even composing the shot.  They certainly should not leave that decision to post processing (this encourages you to largely ignore light completely, and will thus lead to poor choices in subject matter and composition and poorer image quality overall).


----------



## Gavjenks (May 21, 2013)

Note: using custom options, you can even set contrast to LOWER than canon's "neutral" picture mode, by quite a lot.

Whereas zero (neutral) will sample evenly at all lightness values, negative values will actually aggressively / actively start to bunch up the pixels toward the midtones a bit.  In other words, it will sample more heavily from the high-density RAW data on the white and black ends than in the middle, when it decides what to include in the jpeg.  The opposite of what it does by default when it tries to make all jpegs look all shiney and pop-y for you.

So let's say for some reason you actually want to make your image grayer than it even is in reality.  Perhaps you are shooting something like a person silhouetted against a bright window from indoors and want to be able to see some detail outside and on the person (probably a bad choice without a flash, but whatever, it's an example), you can switch to a custom picture mode with -4 contrast and zero everything else, and you will retain oodles of shadow and highlight data in your jpegs (i.e. higher than 8 bit detail, even though its only an 8 bit format! Allowing you to spread it out a lot without visible artifacts), at the expense of minimal true midtone detail, which you don't want.


----------



## Garbz (May 21, 2013)

Before we get into an example why not start with a thought experiment. Lets start with 3 numbers somewhere around the middle grey region.

32760, 32770, and 32780. These three 16bit colours are a whole 30 shades apart. Now lets boost the contrast by 100 values around middle grey with the function (a-32767)*100+32767, effectively a big increase in contrast equally centered around middle grey.
32067, 33067, and 33968. These three colours are now 40 shades apart. Lets convert to 8bit a/2^8
125, 129, 133. Our final values nice and separate thanks to an increase in contrast. 

Now lets do it again
32760, 32770, and 32780. Why not try converting to 8bit first a/2^(16-8) 
128, 128, and 128. And now lets go back to 16bit a*2^8
32768, 32768, and 32768. And apply the same settings again (a-32767)*100+32767
32768, 32768, and 32768. And back to 8bit we go a/2^8
128, 128, 128.

Now these are just numbers but they illustrate a point. If you don't have the data to begin with yes photoshop doesn't clobber it's own functions, and yes you should always edit in 16bit regardless of what you start with, but it helps if you have real meaningful data to work with to prevent posterisation. Now is it field relevant to an image that was exposed absolutely perfectly? Maybe not. But we live in a world of HDR, boosting incredible tones from shadows, shooting into the sun, and above all in order to prevent over exposing we shoot to the left which often requires we brighten up the most butchered data (dark values thanks to a gamma curve). One day you may actually need it. 

More to the point, the extra data allows a lot of latitude for playing with colours. This may not normally be a problem, unless your camera fouls up the autowhitebalance, or you had the wrong manual white balance set. There's very clear side by side differences between a JPEG and RAW shot in daylight, and then post processed to tungsten whitebalance. 


But hey why not do a real world example from my life:

Let's start with a perfectly exposed image of the Great Carina Nebula. Perfectly exposed as there's no clipping on black (noise) and only very slight clipping on one channel (24pixels on blue channel). I.e I can't shoot any brighter without clipping data.





Now lets process this bad boy but first convert to 8bit then back to 16bit.





Uah YUK. Lets try again in 16bit all the way through.





Not bad!

By the way being the perfectionist I ended up redoing this 5 minute exposure some 48 times over a few hours and then stacking them together to create one whopping big 64bit file. Here's the end result of my editing:





many more stars visible now.


----------



## bratkinson (May 21, 2013)

Gavjenks said:


> I'm saying that you should set your camera to have NO auto adjustments (in my Canon T2i, this is done by setting "picture mode" to "faithful" or "neutral"). This makes it no longer do the sharpening and contrast, etc. that it normally does by default in converting to jpg. Or only minimally.



I'm concerned with the accuracy of your statement.  For starters, unless you have access to both the internal chip-level micro code and firmware within the camera, there is no basis upon which to state this as a fact.  Having written internal chip-level logic, I can tell you that there may be an incredible level of 'unseen' processing taking place between the RAW and JPG images produced, regardless of user settable options.  

I'm not out to 'bust your chops' on this, but rather attempting to keep what seems to be a highly accurate, well thought out discussion from going astray.  Unfortunately, my level of 8-bit, 14-bit, whatever-bit depth image information is somewhere near zero.  All I know is that there is more 'tricks' available in Photoshop or whatever with RAW files than JPGs.  So I do most of my editing with the RAW.


----------



## Helen B (May 21, 2013)

Gavjenks said:


> 1) The human eye can't even see all of the colors possible with an 8-bit jpeg.



That isn't even close to being correct if you are referring to sRGB or Adobe RGB.



> 2) Considering #1, the only real advantages of RAW are to be able to do edits with the full data range (thus avoiding banding, etc.), or for LOWERING the dynamic range (e.g., you took a photo with almost entirely midtones, and wish to stretch it out to have full contrast).



Raw is often used to increase the dynamic range above what is represented in the JPEG. As Judobreaker says, there is more (in many cases much more) luminance information in a raw file. You are probably going to say that an in-camera JPEG conversion could be selected that would include the entire usable dynamic range of the sensor, but if it was in sRGB or Adobe RGB gamma it would be very low contrast and there would be bad banding if you tried to restore contrast, whether you worked in 8 bit or 16 bit. If you know how to use a raw converter these effects will be easy to recreate.


----------



## Benco (May 21, 2013)

What about JPG artefacts and such? sure you're using 16 bit but it's still a lossy format and if you're editing a JPG that'll become an issue won't it?


----------



## cptkid (May 21, 2013)

JPEG is silly. RAW is serious.


----------



## Judobreaker (May 21, 2013)

Jpg artifacts aren't too much of a problem if you save high-quality jpg.


----------



## 480sparky (May 21, 2013)

Judobreaker said:


> Jpg artifacts aren't too much of a problem if you save high-quality jpg.



Editing alone can cause the problem, long before you save it.


----------



## amolitor (May 21, 2013)

16 bits might not be enough either. Scientists have discovered that you can use practically infinite dynamic range for internal calculations. The trouble is that as you apply processing steps, you might increase or decrease a pixel value outside of the dynamic range available, and then put it back with another step.

So, you need lots and lots of dynamic range internally.

Also, JPEG is as noted a compression format (although there MIGHT be an option somewhere in some of the versions to do no compression?) so in general the image is being disturbed. It is being disturbed in ways that are generally invisible or at any rate hard to see for the human visual system.

As a general rules, a good 8 bit JPEG is just fine as an output format. The dynamic range and color gamut available are generally more-or-less better than most output media. The JPEG compression applied at this stage, if applied lightly, is invisible to the human eye.

JPEG is NOT a good choice for internal/intermediate images, for similar reasons to 8 or 16 bit being a poor choice internally. Applying it once is no problem, applying it over and over again can be. Just as arithmetic errors can pile up, leading to problems, compression artifacts can pile up. You open the file, you edit a bit. You blow out a couple pixels without noticing it. You save it. JPEG compresses it a little, introducing some perturbation. You open it again, and fix the thing that blew out the pixels (but that data is gone now) and you do some more work, and you save again, JPEG recompresses it.

Do this half a dozen times.

At some point, depending on a tremendous number of variables, visible artifacts will begin to appear.


----------



## Heitz (May 21, 2013)

Regardless of if my eyes can sense differences between adjacent values in any color space, I will never stop shooting RAW.  When I bought my $3000 camera, and paid it off over 6 months, it was to produce the highest quality images possible within technological and creative limits.  Why, then, would I ever consider moving to jpg?  If even 1 out of 500 images could benefit from raw, losing that one to jpg is too many.  I'm not lazy in my editing either, meaning I'm fine importing hundreds of 75mb RAW files and editing, or at least viewing fully, each and every one. 

If there was REALLY no benefit to RAW editing, I doubt it would be so prevalent.


----------



## Judobreaker (May 21, 2013)

480sparky said:


> Judobreaker said:
> 
> 
> > Jpg artifacts aren't too much of a problem if you save high-quality jpg.
> ...



Jpg artifacts are only created during the jpg saving process. Editing a picture which already has jpg artifacts in them could enhance them but editing will never create artifacts.


----------



## amolitor (May 21, 2013)

Further:

In-camera noise reduction is theoretically better than offboard, and in practice it has been shown to be better at least some of the time. This is a case where having the camera produce JPEG (since, in general, you can't get the in-camera noise reduction without converting to JPEG) is a win. Point JPEG.

The OP's original point appears to be that, if you set the camera up with sufficient precision, and expose with sufficient precision, the JPEG output will frequently be indistinguishable from the RAW. This is probably true, most of the time. I take issue with the idea that the camera can be persuaded to correctly push the entire dynamic range into the JPEG 100 percent of the time, even with the most precise setting of controls, but anyways it's irrevevant.

The point is 'if you set the camera up with sufficient precision, and expose with sufficient precision'. Why do I want to do this? Why is this a good thing? I can fiddle around in menus and poke at tiny little buttons fussing around this way and that, just to get a result that -- as far as we can tell -- is darn near indistinguishable from the one I can have the easy way? Why on earth would I work harder to get something that's practically almost all the time mostly just as good?

There is an excellent case to be made for RAW+JPEG for noise reduction, at least for some models of cameras. There is an excellent case that JPEG (without a lot of fiddly effort to make it perfect) is perfectly adequate for a lot of pictures.

To try to make some blanket statement that JPEG is just as good in all cases, though, is just silly.


----------



## 480sparky (May 21, 2013)

Judobreaker said:


> ...........Jpg artifacts are only created during the jpg saving process........



Huh?

So I load a JPEG, choose the Red channel and crank the brightness up to 100%.







I haven't saved the image yet... just loaded and edited it.  This is how it looks on my monitor, I just took a screen shot.

You're telling me there are no editing artifacts here?  It only becomes an artifact when I save it?


----------



## Overread (May 21, 2013)

A few more thoughts to add:

1) White balance. RAW doesn't fix the white balance and lets the photographer set it themselves; this makes adjusting the white balance of a shot very easy in RAW editing; its just adjusting two sliders. Now if you take a shot in JPEG mode and don't have time to custom set the white balance or something happens and the white balance is off then suddenly you're in a world of pain as you have to spend a lot longer editing the JPEG to adjust the white balance of the shot to look proper - whilst if you had it  in RAW it would take a few moments to adjust. 
This also leaves you more creative options with RAW since you can easily adjust the white balance in editing for the best visual appearance you want for the shot.

2) Faithful in the camera is still applying a level of editing to the final JEPG. The image data that the camera captures needs somewhere to start when it comes to applying these settings. RAW processing software gives a starting point and lets you then effortlessly change the final value - JPEG on the other hand selects a point for you. Faithful is as neutral a value as typically possible based upon testing and such done by the designers; but its not "perfect" nor fully "faithful" to the scene as such. Heck you can see this fact yourself as you compare different RAW processing software options (or even different editions of the same software) and you can see that different start points by the software are selected on nearly all the parameters such as noise reduction, sharpening, contrast etc...

4) In camera noise reduction and sharpening is global - it hits the whole photo. Sometimes you'll have a shot that just doesn't need that over the whole photo, you want it in specific areas so having a RAW lets you take control (although of course, cameras do apply noise reduction and such during the capture process and bundle it into the RAW itself before JPEG processing). 

In the end if there were no real gain with RAW people would not advocate and use it so widely.


----------



## Judobreaker (May 21, 2013)

480sparky said:


> Judobreaker said:
> 
> 
> > ...........Jpg artifacts are only created during the jpg saving process........
> ...



No no, you didn't read my post did ya. 
I said editing can indeed enhance jpg artifacts, it can not however create them.


----------



## 480sparky (May 21, 2013)

Judobreaker said:


> No no, you didn't read my post did ya.
> I said editing can indeed enhance jpg artifacts, it can not however create them.



There were no artifacts before the edit.


----------



## Judobreaker (May 21, 2013)

Well if you created artifacts through editing they sure as heck aren't jpg artifacts!
Jpg artifacts are compression artifacts which are made during the saving process to decrease filesize. As long as you're editing the file there is no compression process running, that process only runs when you save the file.
Sure you can probably create artifacts through editing, but they are not really jpg artifacts. ^^


----------



## amolitor (May 21, 2013)

What IS creation, anyways?


----------



## Judobreaker (May 21, 2013)

Google said:
			
		

> Creation: The action or process of bringing something into existence: "job creation".



There.


----------



## 480sparky (May 21, 2013)

Judobreaker said:


> Well if you created artifacts through editing they sure as heck aren't jpg artifacts!
> Jpg artifacts are compression artifacts which are made during the saving process to decrease filesize. As long as you're editing the file there is no compression process running, that process only runs when you save the file.
> Sure you can probably create artifacts through editing, but they are not really jpg artifacts. ^^



They certainly ain't *raw *artifacts. :er:

Can't be TIF or BMP or PNG or NEF or DNG artifacts.

They don't look like 1800 BCE Mesopotamian artifacts.

I doubt they're grilled ham and cheese sandwich artifacts.


----------



## jowensphoto (May 21, 2013)

I was having a similar discussion today. Each file type has its own uses, I get that. BUT can anyone tell me why a wedding photographer would shoot solely in JPEG?


----------



## Gavjenks (May 21, 2013)

> Let's start with a perfectly exposed image of the Great Carina Nebula. Perfectly exposed as there's no clipping on black (noise) and only very slight clipping on one channel (24pixels on blue channel). I.e I can't shoot any brighter without clipping data.


This example is irrelevant unless you set your in-camera settings to be extremely anti-contrast. For example, in Canon, which I shoot, there is an option for custom "picture modes" and you can set contrast to negative if you like.  What this will do is sample a ton of data from the RAW on dark and light ends of ths histogram, and almost none from the middle relatively.

So the RAW is like this:

LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

A jpeg made with the default settings would be sampled like this:
LLMMMMMHH

A jpeg made with neutral settings would be sampled like this:
LLLMMMHHH

A jpeg made with inverse contrast settings would be sampled like this:
LLLLMHHHH

And the eye can only see something like
XXXXXX
(something like 3/4 the dynamic range of a jpeg... I mean specifically within the boundaries of sRGB yeah yeah.. see further down) 

The result is that you actually end up with much more than 8 bit data for shadows and highlights, even though your format is 8 bit, because it is essentially comandeering the data space that would have been used by midtones in order to store more shadow and highlight data (it will look extremely gray out of camera).

This is exactly the setting you would use to shoot astrophotography in jpeg.  If you used that setting, then you wouldn't have to do any anti-contrast edits like you did (or only very minorly), and so you wouldn't be stretching the jpeg out of its original state much, and you wouldn't get noticeable posterization.

RAW is only necessary here if you didn't set your camera for astrophotography before taking astro photographs.  Sure this is in some cases more convenient (to not have to think ahead), but ultimately, thinking ahead about lighting is something you should always be doing anyway, and setting your custom picture mode would just be a part of that process.

The result being that you don't have to fill up your entire memory card in 5 seconds or spend hours in post to get what you want.

ALSO: if you're already on a tripod and taking 68 photographs of the same scene, etc. then it would be pretty easy to simply do exposure bracketing for your jpegs. A 3xbracketed exposure will lead to even higher dynamic range than RAW, and will take up half the memory card space.



> I'm concerned with the accuracy of your statement. For starters, unless you have access to both the internal chip-level micro code and firmware within the camera, there is no basis upon which to state this as a fact. Having written internal chip-level logic, I can tell you that there may be an incredible level of 'unseen' processing taking place between the RAW and JPG images produced, regardless of user settable options.


No matter what the user settings, it has to make sampling decisions in order to go from 12 bit or whatever to 8 bit.  And it has to do jpeg compression, and various other things.  For sure.

But what's relevant to the photographer is not borign algorithm stuff for changing to a specific format.  What's relevant is what you can see, and the settings are sufficiently powerful to allow you to never see anything bad if you know what you're doing.

I can't claim to have years of experience or anything, since I just discovered this like a week ago (hence a "challenge" thread), but every stress test I've done so far turns out fine.



> Raw is often used to increase the dynamic range above what is represented in the JPEG. As Judobreaker says, there is more (in many cases much more) luminance information in a raw file. You are probably going to say that an in-camera JPEG conversion could be selected that would include the entire usable dynamic range of the sensor


No, not the entire range. That's impossible.  What you do is to retain the portion of the dynamic range that you care about for that photo, such that you don't need to dramatically change the contrast in post.  And as long as you don't have to do that, you will not see banding.  Photos never look banded right out of the camera at print size... there's a reason for that. it's only after you stretch them in editing.  And in-camera setting can prevent the need for stretching in post.

For example, in the case of astrophotography above, where an extreme negative contrast sampling should be used (very much the opposite of the default camera settings).



> 1) The human eye can't even see all of the colors possible with an 8-bit jpeg.
> That isn't even close to being correct if you are referring to sRGB or Adobe RGB.



Okay, fine. A more responsible way of putting it: Within the range covered by sRGB (and the internet and any but the most expensive prints), the resolution is plenty for your eye to work with, and you will not see any obvious color posterization.

The important point here is not the exact number of colors you can see versus print, etc.  It's that using a RAW isn't going to make your color space any more vibrant.  If you're going to print your photo anywhere other than a Sotheby's catalogue, or something, you're just going to convert it back ALSO to sRGB later anyway, because that is the standard of almost everything, and it is the closest you will usually get to print color accuracy.

So if it's going to end up in sRGB anyway, the ONLY advantage to RAW for color editing is therefore flexibility in choosing which bit of the color space is going to be sampled to sRGB.  And if you are thinking about color and light in the field (which you should be), then you shouldn't need very much flexibility.  The out of camera jpeg should already be pretty much where you want it, and whatever minor (10-20%) corrections you may want to do will not be sufficient to cause visible color posterization.

There are tons and tons of settigns in your camera for setting up a proper white balance.  There's custom compensation, there's pre-set WB for different standard lighting conditions, and there are options to calibrate using cards in the field. You have all the tools you need, most of which only need to be set once for an entire shoot.



> What about JPG artefacts and such? sure you're using 16 bit but it's still a lossy format and if you're editing a JPG that'll become an issue won't it?


jpegs out of your camera are not compressed very much.  Especially if you shoot Canon.  Nikon, by my understanding, has historically been more aggressive about compressing busy images to conform to a smaller file size, whereas Canon tends to just let the file be as large as it needs to be to have what they deem an acceptable level of compression for non-visible artifacts.

(This may no longer be the case nowadays. I can't afford to buy new bodies all the time, so I'm not up on latest developments)



> 16 bits might not be enough either. Scientists have discovered that you can use practically infinite dynamic range for internal calculations. The trouble is that as you apply processing steps, you might increase or decrease a pixel value outside of the dynamic range available, and then put it back with another step.


That's true.  If photoshop had a 32 bit mode that still had all the tools available, then I would use that.  But it doesn't. Although since RAW is only 12 or 14 bit, any discussion of hypothetical edits done with a currently nonexistent 32 bit or higher commercial photo editor is not really on topic (both jpeg and RAW would benefit equally from the extra processing range).




> If even 1 out of 500 images could benefit from raw, losing that one to jpg is too many.


The idea is that RAW slows down your workflow and your burst speed.

So for those 500 images, editing them in RAW might take 2-3 extra hours compared to editing them in 16 bit photoshop from jpeg.  During those 2-3 hours, you could have been out in the field taking another 100 photographs, of which many more than 1 would have been keepers.

So by spending unnecessary time in post, you are in fact losing would-be images by not being out shooting as much.  Just like time is money, time is keepers.

(Also, smaller buffer size means smaller bursts, which also will cause many sports photographers, etc. to miss shots)



> If there was REALLY no benefit to RAW editing, I doubt it would be so prevalent.


The system I am describing will not work if you leave your camera at default settings.  It also will not work if you do not consider the light and color before every shooting situation.

This means RAW is often more convenient.  But IMO, knowing how your camera works and considering light and color constantly are things one should do anyway.  And if you do, then it is fairly easy to compensate and not require major editing in post that causes all the problems. But people may legitimately disagree with me on the virtues of thinking carefully about light and color each time. *shrug*



> The point is 'if you set the camera up with sufficient precision, and expose with sufficient precision'. Why do I want to do this? Why is this a good thing? I can fiddle around in menus and poke at tiny little buttons fussing around this way and that


In my Canon Rebel T2i, even as a low end dSLR, the menu for this is not hidden at all.  There's actually a dedicated button (down button on the directional pad) which immediately brings you to the selection of your picture mode. i.e. telling the camera what portion of the RAW to sample to jpeg.

There are also I think infinite (?) or if not at least 3-4 custom slots available that you can set and then use within 1-2 seconds. And this is not something you would do before every shot.  It's something you would do before an entire shoot or major change of light.

E.g. "I'm shooting the night sky now, lemme switch to my night sky preset"  or "I'm shooting broad daylight, lemme switch to my neutral mode" etc.

Is this an extremely minor inconvenience?  Yes.  But the benefit = faster burst, 5-6x as many photos in your memory card, and much faster workflow when editing. I think that's majorly worth it.



> So I load a JPEG, choose the Red channel and crank the brightness up to 100%.


You're not following my instructions, which includes using your camera settings to get reasonably within the ballpark prior to editing. Cranking brightness up to 100% implies that you thought your original image was like 6 stops underexposed... You should never be doing that.


----------



## amolitor (May 21, 2013)

You should use your camera in whatever way best suits your needs.

Me, I shoot both ways. I use RAW a lot when I can't be buggered to worry much about the exposure _a priori_&#8203; which is quite often.


----------



## Garbz (May 22, 2013)

cptkid said:


> JPEG is silly. RAW is serious.



The only thing silly is that statement. JPEG has many uses. 



Gavjenks said:


> > Let's start with a perfectly exposed image of the Great Carina Nebula. Perfectly exposed as there's no clipping on black (noise) and only very slight clipping on one channel (24pixels on blue channel). I.e I can't shoot any brighter without clipping data.
> 
> 
> This example is irrelevant unless you set your in-camera settings to be extremely anti-contrast.



Why is the example irrelevant? Just because it doesn't fit your perfect idea of a normal photograph? The first photo I posted was exactly what I saw through my viewfinder. I just chose to enhance the shadow detail. Granted I enhanced it more than I normally would but that suddenly doesn't render my photo invalid. 

But now you're arguing semantics. The camera gives you a choice of picture modes does it? So suddenly we supposed to do some processing in the camera in a high bit depth just to get around your argument that there's no reason to use a high bit depth in post processing? So then why shouldn't I just edit in RAW? More importantly I directly challenge you to get the same effect in 8 bit and you may start with as much in camera processing as you can achieve. 



> That's true.  If photoshop had a 32 bit mode that still had all the tools available, then I would use that.  But it doesn't. Although since RAW is only 12 or 14 bit, any discussion of hypothetical edits done with a currently nonexistent 32 bit or higher commercial photo editor is not really on topic (both jpeg and RAW would benefit equally from the extra processing range).



That's why I use Pixinsight, though the interface is arcane it provides 64bit native processing in a linear colour space. 



> The idea is that RAW slows down your workflow and your burst speed.
> 
> So for those 500 images, editing them in RAW might take 2-3 extra hours compared to editing them in 16 bit photoshop from jpeg.  During those 2-3 hours, you could have been out in the field taking another 100 photographs, of which many more than 1 would have been keepers.



False and false. Nothing about RAW slows down your workflow. You can preset your import settings from RAW just like you can set your in camera JPEG settings. Heck if you use the manufacturer's RAW converter it will actually read the camera picture settings out of each individual file and apply them on import without any loss of data. Also 2-3 hours in the field taking photos of nothing because I miss my photo opportunities while messing with menus in the camera does not sound like time well spent to me. 

Oh and RAW doesn't slow down burst mode. It only slows down continuous shooting speeds after the buffer is full which rarely happens and never was a problem even when I was firing off a ludicrous number of frames at a motorsport event. 



> This means RAW is often more convenient.  But IMO, knowing how your camera works and considering light and color constantly are things one should do anyway.  And if you do, then it is fairly easy to compensate and not require major editing in post that causes all the problems. But people may legitimately disagree with me on the virtues of thinking carefully about light and color each time. *shrug*



As mentioned earlier it's one thing to know how your camera works and quite another again to have time to set it to the "perfect" settings. In a studio I would agree with you whole heartedly (and still shoot RAW), but in the real world where photo opportunities are fleeting moments, and the lighting or conditions change at any moment the camera stays at the default settings. Hell I'm not even a proponent of shooting manual for the same reason (though I would in a studio).


----------



## Benco (May 22, 2013)

Gavjenks said:


> jpegs out of your camera are not compressed very much.  Especially if you shoot Canon.  Nikon, by my understanding, has historically been more aggressive about compressing busy images to conform to a smaller file size, whereas Canon tends to just let the file be as large as it needs to be to have what they deem an acceptable level of compression for non-visible artifacts.



That's not the same as uncompressed though. The fact that artefacts are 'non visible' is disengenuous, depending on what you do they might remain invisible, with heavier editing they might not. If you're talking about JPG being a practical alternative that can equal raw then surely this is a problem.


----------



## Ysarex (May 22, 2013)

Gavjenks said:


> *....Conclusion:* There shouldn't really be any technical reason why it is more useful to edit a RAW than to edit a jpeg after first converting it to 16 or 32 bit.



The technical reason is that the camera JPEG processing software isn't adequately capable of rendering the photographer's intent and, after the fact with the raw data discarded, it is then usually impossible to realize the photographer's intent from the JPEG file.

In a response further back in this thread you said this: "And if you are thinking  about color and light in the field (which you should be), then you  shouldn't need very much flexibility.  The out of camera jpeg should  already be pretty much where you want it, and whatever minor (10-20%)  corrections you may want to do will not be sufficient to cause visible  color posterization.

There are tons and tons of settigns in your camera for setting up a  proper white balance.  There's custom compensation, there's pre-set WB  for different standard lighting conditions, and there are options to  calibrate using cards in the field. You have all the tools you need,  most of which only need to be set once for an entire shoot."

This is where you're completely missing it. If you can coax a JPEG from the camera software that is close to the photographer's intent, and I grant that this is in limited circumstance possible, then I'll agree that a camera JPEG once converted to 16 bit can tolerate light repair and remain serviceable (wince). The problem is that what you're suggesting as an approach to taking the photo isn't workable over a sufficient range of conditions -- at least not for me. And those tons and tons of settings available in our cameras are sub-crude and entirely inadequate to the task given what we know is possible once we have the raw file. So in part it's a case of because we can. We're always going to be pushing to go as far as we can. Using the camera processing software is like tying one hand behind your back, then tying your shoelaces together and trying to run.



Gavjenks said:


> I am curious as to how I might be proven wrong, in reasonable situations where you didn't completely fail your photo?



You asked for a challenge, here you go. I took this photo:








I'm happy with that photo. That's what I saw (photographer's intent) and what I expected to take away when I clicked the shutter. I am using a Canon camera and the day I unboxed it I set those tons of settings to null and the picture style to faithful as you have suggested. Here's the camera JPEG for that image:








And here's a link to that JPEG at full-res so you can download it and go to work processing it: Cannon_wetland The processing goal of course is to realize the photographer's intent and produce the same image I produced from the raw file. You can post it back here when you're finished.

You may want to argue that I could have used those tons and tons of camera settings to coax the camera software into producing a result closer to my intent and I might have better luck then trying to repair the JPEG. I know better -- not in lighting conditions like this.

Your very limited proposition that a JPEG once converted to 16 bit can tolerate light repair is valid but it makes me wince. I've never seen a camera processed JPEG that I couldn't repair and improve. I spent years chasing after this very proposition since I've always wanted a camera to carry with me everywhere. When digital cameras became shirt pocket size I started buying them and using them in precisely this way. They only produced JPEGs and I would work to get the best JPEG possible and then repair it in Photoshop. I went through 1/2 a dozen of those cameras constantly frustrated. I'd buy one and then when it bit me (impossible to repair JPEG and I knew a raw file would have worked) I'd ship it off to a niece or nephew and eventually get another one. I finally compromised on a slightly larger camera that would save raw files. I'm happy now and my production rate has soared with no more frustration.

Joe


----------



## Gavjenks (May 22, 2013)

Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.

The saturation is not a big deal, because upping saturation does not degrade data (there's already plenty of hue information in the jpeg.  They're just boring hues, but the data is there, and thus doesn't have to be invented or stretched causing banding).

The contrast, however, IS a big deal. In order to follow my suggested process, you should have shot this with a custom picture mode of [0 -4 0 0] ([sharpness contrast saturation color tone]).

Did you do that?  If not, then this is not a valid challenge photo, because I never claimed to be able to make whatever you want happen without any consideration of settings when taking it. In other words, you threw out the data that I needed and kept a bunch that I didn't need, when with the right settings, you could have done the opposite and made it possible to match.



Most people don't routinely do what I am suggesting (not necessarily because it's wrong.  Possibly just because it requires more thinking in exchange for time, and because it is not the default setting).  Thus, it is very unlikely that any existing photos are going to be valid challenge photos here.  If you don't routinely use picture style and white balance in camera before a shoot to push the jpeg as far as possible in the right direction, then none of your old photos will work for demonstration purposes.  You'd have to go take a new photo with optimal settings, and then post that as a challenge to match to the RAW processed version.


----------



## o hey tyler (May 22, 2013)

Upping saturation can introduce banding or posterization. If that's not degradation, I don't know what is.


----------



## amolitor (May 22, 2013)

Gavjenks said:


> Possibly just because it requires more thinking in exchange for time, and because it is not the default setting).



This makes no sense at all. You're essentially proposing that we do something in-camera rather than out of camera. You're also waving away all the technical issues with an airy "well, they're not visible, anyways" which is at best questionable.

Since when is doing something in-camera faster than doing it out of camera?


----------



## dbvirago (May 22, 2013)

jowensphoto said:


> I was having a similar discussion today. Each file type has its own uses, I get that. BUT can anyone tell me why a wedding photographer would shoot solely in JPEG?



The reason I have seen is the large number of images they shoot and the short time to process. However, since a couple of clicks can convert the RAWs to JPGs, I'm not sure I get it either.

For me the reason was simple. I shot JPGs the first six months or so after getting a DSLR until the day I came home from a full day of shooting landscapes with the WB set to Tungsten.


----------



## DBA (May 22, 2013)

Garbz said:


> Gavjenks said:
> 
> 
> > The idea is that RAW slows down your workflow and your burst speed.
> ...


What Garbz said. :thumbsup:

Unless you're uploading images [online] directly from your camera, I don't see how it'll slow down your workflow (maybe a minute or two while transferring them to your pc).


----------



## Gavjenks (May 22, 2013)

> Upping saturation can introduce banding or posterization. If that's not degradation, I don't know what is.


Upping saturation is a linear change.  taking an array [1 2 3 4 5] and adding 3 to all the numbers = [4 5 6 7 8].  The sample rate is the same, there is zero loss of precision, and it is entirely reversible.  This would be true even if you edited it at 8 bit. if you get banding from saturation changes, then you must be doing something fancier than just upping saturation linearly.  Are you perhaps upping saturation with a curve algorithm or similar? OR are you upping saturation to ceiling causing clipping and then reducing it again (silly to ever do, but would create posterization)?



> Since when is doing something in-camera faster than doing it out of camera?


Opening a jpeg OOC in photoshop takes about 1 second.  Processing a RAW into a jpeg even with automatic settings can take 10-20 seconds depending on the software you use (and for a full memory card of photos, that can add up to hours of difference.  Your camera does this, however, while you're out shooting, and it is also hardware accelerated for that specific task).  More importantly, if you are shooting RAW in the first place, you obviously intend to at least some of the time do something OTHER than automatic processing, otherwise there was zero reason to use up all that extra memory space, etc.  So you're also spending time fiddling with custom conversions some % of the time, just to get it into a photoshop compliant format. That could be anything from minutes to hours depending on what proportion of your RAWs you insist on hand crafting into rasterized form.  Especially since processing time on doing any edits in a RAW editor usually takes 2-5x longer in my experience than doing an equivalent edit in photoshop, even in 16 bit.

Setting aside the question of whether or not that helps you achieve a better photo, the fact that it slows you down is a pretty objective one.


> The reason I have seen is the large number of images they shoot and the  short time to process. However, since a couple of clicks can convert the  RAWs to JPGs, I'm not sure I get it either.


Why would you use up all that extra space and burst speed and tie up your computer for longer if all you're going to do is the exact same thing your camera already does for you without any of those drawbacks?

Well, you said why: so that you can correct your mistake if you have the wrong white balance for an entire shoot (something that a cursory glance at the histogram on your LCD for the first couple of photos would have highlighted just as well).  If that level of extreme convenience to not have to consider your color or lighting appeals to you, then sure, shoot RAW.  I've suggested that all along.  I'm simply saying that in MY opinion, you should be carefully considering your light and color and contrast anyway, regardless of the format you shoot, if you're going to come up with good photos as often as possible, so setting these settings correctly should come for free as a side effect of the mental work you should already be doing.



> "well, they're not visible, anyways" which is at best questionable.


This is a challenge thread.  Says right in the title.  Go out and shoot a photo in both RAW and jpeg, using the optimized settings as I have discussed to make the jpeg most closely match your desired artistic intent.  And then process the RAW as you see fit, and give the jpeg to me and I will attempt to match it.  Whether the differences become observable or not is an empirical question.

(note: it would be most fair if you gave me the settings you used in your RAW converter too.  This is a technical possibilities challenge, not a "can Gavjenks guess what edits you did" challenge)


----------



## amolitor (May 22, 2013)

I am utterly uninterested in taking your challenge, and actually, so is everyone else. Your challenge is, as near as I can tell, 'Go do this complicated thing that's a pain in the ass and provides no benefit whatsoever, and prove to me that it's not almost as good as what you already do'. Why on EARTH would anyone want to do that?


----------



## Gavjenks (May 22, 2013)

> Also 2-3 hours in the field taking photos of nothing because I miss my  photo opportunities while messing with menus in the camera does not  sound like time well spent to me.


Setting your picture style and white balance is something you would do only once for a given lighting situation (e.g. set it once when you start in mid day, then maybe reset at sunset, or if you switch to indoors, etc.), outside of extraordinary circumstances.

And since you should already be reconsidering ALL of your settings anyway when you switch to a new lighting situation, this does not require any greater presence of mind than you already need.  It's just one more step added to the routine "recheck everything" you already have to do when switching environments.



> Oh and RAW doesn't slow down burst mode. It only slows down continuous  shooting speeds after the buffer is full which rarely happens and never  was a problem even when I was firing off a ludicrous number of frames at  a motorsport event.


Depends on your camera body.  Maybe if you have a fancy pro grade full frame with 10 FPS already or something, and some huge buffer, it's not an issue.  On my Rebel T2i, though, the difference is very noticeable.

Shooting jpeg, i was able to fire at full speed for something like 50 shots in a row before I gave up and assumed it was just infinite.

Shooting RAW only, my camera stopped after 7 shots and had to sit there and think for like 2 seconds.  Then each subsequent photo was about 1 per second instead of 4, with thinking in between each time.





> I am utterly uninterested in taking your challenge, and actually, so is  everyone else. Your challenge is, as near as I can tell, 'Go do this  complicated thing that's a pain in the ass and provides no benefit  whatsoever, and prove to me that it's not almost as good as what you  already do'. Why on EARTH would anyone want to do that?


Your strawman version is not interesting, no.

The actual thing I am describing however, is.  "Go do this thing that requires a bit of extra complexity, but not as much as dealing with RAWs, and let's see if it has any impact on image quality."

At *WORST*, this is equally as complex as dealing with RAWs.  Even in optimal circumstances for your side of the argument, where somebody simply auto converts every single RAW to jpeg without touching their keyboard (using it only as a fallback safety net for catastrophic mistakes), it still equates to pushing a couple extra buttons vs. pushing a couple of other extra buttons.

Even then, it is useful to know about and keep in mind in order to choose which set of strengths and weaknesses is best for a given situation. "Do I need burst speed/amount or am I worried about filling up my memory or do I not have easy access to a computer with RAW software on it to clear out my cards?  Jpeg with in camera optimized settings set once per lighting situation."  "Do i not need those things right now and prefer to have a layer of extra security?  RAW."


----------



## Benco (May 22, 2013)

Gavjenks said:


> Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.
> 
> The saturation is not a big deal, because upping saturation does not degrade data (there's already plenty of hue information in the jpeg.  They're just boring hues, but the data is there, and thus doesn't have to be invented or stretched causing banding).
> 
> ...



Most people don't do that because there's no need to, if you shoot in raw then none of that matters. Let's approach it from a different angle: what's the advantage of shooting and editing in JPG over raw? I really don't see any, if you do that you have to go to way more trouble to achieve results that might nearly be as good.


----------



## amolitor (May 22, 2013)

I mostly shoot JPEGs. JPEG is fine stuff, I use it a lot.

When I want to move more of my decision making in to post, then I shoot RAW. That's what RAW is for, it's for deferring decision making and (as a special case of that) recovering from errors.

Claiming that JPEG is "just as good" as RAW is simply wrong, though. It's good stuff, it's really quite good, make no mistake. I'd really have to work at it to cook up a test case with interestingly visible issues, and I'm a pretty technical boy. I pretty much know where JPEG buries its bodies. But I could do it. I know I could do it because I understand the technologies involved. I don't have to drop a small cannonball and a large cannonball from the tower in Pisa to prove that they'll fall at the same rate, and I am certainly not going to climb up a great big tower with two cannonballs and a stopwatch just because some guy on the internet asks me to. Neither do I need to fiddle around with custom picture settings and so on.

My camera actually does a bang up job making JPEGs when I set it to "standard" mode and let it do the metering. No custom picture modes involved. They're not *quite* as good as what I can get with RAW and post processing, though.


----------



## Gavjenks (May 22, 2013)

Benco said:


> Most people don't do that because there's no need to, if you shoot in raw then none of that matters. Let's approach it from a different angle: what's the advantage of shooting and editing in JPG over raw? I really don't see any, if you do that you have to go to way more trouble to achieve results that might nearly be as good.



I've outlined them multiple times:
1) RAW takes up 5x more memory on your card than jpeg does (16 bit is 2x + the light compression of OOC jpeg). Doubly problematic if you're on vacation and don't have a computer with you with proprietary software for converting RAWs (thus, can't clear out your card at the end of the day).

2) RAW dramatically reduces the number of photos you can take in a burst on low-mid range camera bodies (in mine, RAW alone will limit you to 1 FPS after just 7 shots, and RAW+jpeg will limit you to 1 FPS after just *TWO *shots. Jpeg allows 4 FPS continuously seemingly forever, by comparison)

3) RAW cannot be used OOC even if you dont want to edit it at all.  Websites and printers can't read it, so you are obligated to process it through photoshop or whatever other RAW software. This is a disadvantage if you are taking normal photos of friends or something, for instance, and are not trying to win the pullitzer prize in photojournalism. Oftentimes while just out and about taking photos, I will have about a 30/70 mixture of funny snapshots that I want to share with somebody on facebook as a joke vs. real images I intend to be beautiful. With jpeg, I can pick out the silly ones and post without processing.  With RAW, I'd have to painstakingly process them all first OR sit there and constantly switch back and forth between formats in camera, OR use up even more memory and cripple my ability to take burst shots with RAW + jpg.

4) For those images that you do want to process regardless, RAW takes much longer to load into photoshop than jpeg. At best this is annoying (if automated/uniform conversions) and I have to go eat a sandwich or something instead of editing photos like I want to, while it processes slowly.  At worst, this wastes *tons *of my time, if I am actually attempting to do photo-by-photo optimized RAW conversions and am sitting at the keyboard the whole time.


The next two are more subjective musings than cold hard disadvantages, but:


5) In my opinion, the safety net of RAW encourages a photographer to be a little lazy and not worry about lighting or coloration in the field.  Lighting, color, and exposure are all part of composition!  And if you don't consider them carefully, then you won't even take the same photos that you would have if you did.  i don't mean that you would have used different settings.  I mean that if you don't consider these things, you will not even end up pointing your camera at the same scenes as if you do.  And usually, what you do end up pointing at will be substandard composition.  No amount of post processing can ever fix poorly conceived composition.

IF it were the case that RAW was the only way to achieve technical clarity and vividness in a print, then this would be a temptation that you'd just have to suck it up and learn to avoid.  But IF you can actually achieve the same quality with jpeg (point of this thread is that I believe you can), then the ability to force yourself to not give in to that temptation to be lazy is a very nice feature.  And if you're thinking about these things anyway as you shoot, then the picture style settings, etc. will be second nature side effects, and not a hassle at all.

6) Depending on how much you take editing of RAWs into your own hands, you may be shooting yourself in the foot.  This varies depending on how you use them.  If you edit RAWs in high quality, custom made RAW converters by major brand name software companies, then you're probably fine.  But if you attempt to do basic things like noise reduction and sharpening yourself using sub-optimal software, you are almost guaranteed to end up doing worse than the professional Canon or Nikon engineers do in modern cameras, unless you spend half an hour fine tuning every little thing specifically to each image and have years of practice. In short: RAWs used naively by amateurs who don't use the right tools can lead to worse results than even a jpeg OOC.


----------



## amolitor (May 22, 2013)

So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.

Just don't pretend that there isn't any advantage to RAW, eh?


----------



## Gavjenks (May 22, 2013)

amolitor said:


> So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.
> 
> Just don't pretend that there isn't any advantage to RAW, eh?



There is certainly not always an advantage in every instance.

But in some situations, there pretty much objectively is.  For instance, if I'm spending all day shooting action sequences of athletes where the interesting part happens very rapidly (like pole vaulters or gymnasts), and I'm in a gymnasium with consistent lighting, using a normal person camera under $3000, then it is pretty much flat out wrong to be using a RAW.  RAW would limit my bursts to the point where I will simply miss action that I need to capture. It will fill up my cards due to shooting all day, requiring me to invest unnecessary money in a bunch of memory. I have no reason to require the flexibility of adjusting exposure in post, because the conditions are not rapidly changing, and I can just nail it all in camera within 5 minutes of walking in the door. And working with the huge volume of images would be unnecessarily unwieldy when i get them on a computer, for no real advantage.

For situations like that, it is important to know how your in camera settings work for WB and picture style, to correctly use a more advantageous jpeg format.

In many other situations, it is more ambiguous than that, and in some, RAW is clearly superior, i admit.


----------



## o hey tyler (May 22, 2013)

Gavjenks said:


> There is certainly not always an advantage in every instance.
> 
> But in some situations, there pretty much objectively is.  For instance, if I'm spending all day shooting action sequences of athletes where the interesting part happens very rapidly (like pole vaulters or gymnasts), and I'm in a gymnasium with consistent lighting, using a normal person camera under $3000, then it is pretty much flat out wrong to be using a RAW.  RAW would limit my bursts to the point where I will simply miss action that I need to capture. It will fill up my cards due to shooting all day, requiring me to invest unnecessary money in a bunch of memory. I have no reason to require the flexibility of adjusting exposure in post, because the conditions are not rapidly changing, and I can just nail it all in camera within 5 minutes of walking in the door. And working with the huge volume of images would be unnecessarily unwieldy when i get them on a computer, for no real advantage.
> 
> ...



Ok, no one cares. Shoot how you want. Others will do the same. Get over it.


----------



## Ilovemycam (May 22, 2013)

Just post pix. Pix is what we deal with, not words. Lets see jpeg vs raw.


----------



## Helen B (May 22, 2013)

Gavjenks said:


> amolitor said:
> 
> 
> > So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.
> ...



That's why some people use raw and some use JPEG, I guess, and it isn't because they are lazy or ignorant.


----------



## Gavjenks (May 22, 2013)

> But if you try to tell people that there's no advantage to RAW


I didn't tell anybody that. Of course there are advantages.

I like your analogy with the food. It is a good one.  

But very few chefs go out and slaughter their own chickens and grow their own wheat in the back yard, etc.  There's obviously a balance that every chef chooses between maximum possible intervention and customization and convenience.  Basically what I'm saying in this thread is:

1) That balance is probably closer to the pre-prepared side than is typically talked about on these forums.  Many situations certainly are best with RAW, but probably not as many as people make out to be the case. Especially non pros who don't know everything their cameras can do and haven't considered all the angles (main intended audience of this thread)

2) Here are some tools you can use to shift that balance more toward the preprocessed and convenient side, without making as many sacrifices in final quality as you may have traditionally expected.


The exact balance point will be different for every person, but if you never considered this topic or these strategies before, you may well consider shifting yours from wherever it is to a relatively more preprocessed point (some greater % of borderline situations now shooting jpeg).



> it isn't because they are lazy or ignorant.


Laziness is a *possible *pitfall of using RAW.  Not a guaranteed one. I tried to make this very clear.  The disadvantage is having to deal with the temptation, which takes some effort.  Some less disciplined or more newbie photographers are likely to succumb to the temptation.  Seasoned pros can probably handle it just fine and not.


----------



## Ysarex (May 22, 2013)

Gavjenks said:


> Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.
> 
> The saturation is not a big deal, because upping saturation does not degrade data (there's already plenty of hue information in the jpeg.  They're just boring hues, but the data is there, and thus doesn't have to be invented or stretched causing banding).
> 
> ...



OK, got home and snapped a photo of my neighbor's Victorian house. Here's my version processed from the raw file:








Canon camera with the picture style set to faithful and the contrast set to -4 as you require.

Here's your JPEG: Victorian house

Waiting breathlessly.

Joe


----------



## jake337 (May 22, 2013)

amolitor said:


> So shoot JPEG. What a good idea! Virtually everyone shoot JPEG some of the time.
> 
> Just don't pretend that there isn't any advantage to RAW, eh?



My D90 hasn't produced a jpeg since.....Well I can't remember.

I produce my jpegs via post processing.




I guess we could send all that film we shoot to Walmart to get developed as well.......


----------



## Benco (May 22, 2013)

Gavjenks said:


> Benco said:
> 
> 
> > Most people don't do that because there's no need to, if you shoot in raw then none of that matters. Let's approach it from a different angle: what's the advantage of shooting and editing in JPG over raw? I really don't see any, if you do that you have to go to way more trouble to achieve results that might nearly be as good.
> ...



Yes your first four points are some disadvantages of raw, none of them however make JPG an equal to raw for post processing. You digress.


----------



## Ilovemycam (May 22, 2013)

Ysarex said:


> Gavjenks said:
> 
> 
> > Your intentions here, clearly, were to 1) reduce the contrast of the actual scene, and 2) up the saturation. Simple histogramming of your jpeg versus your intended shows a huge, two-peak-with-valley-in-the-middle mega contrasty pattern on the jpeg, versus a fairly even distribution in the intended.
> ...




Put them side by side. Best JPEG PP and Best RAW PP of the same image.

I know RAW will win with big prints. JPEG only has about 6% of the info a RAW file has. (6% image size anyway in mb)

Don't know about small prints and monitor with JPEG / RAW debate.


----------



## TruckerDave (May 22, 2013)

o hey tyler said:


> Ok, no one cares. Shoot how you want. Others will do the same. Get over it.



Ding ding ding.


----------



## 480sparky (May 22, 2013)

I guess we need something to bicker about since the Nikon v. Canon thing has been beat to death.

Perhaps we should debate Chevy v. Ford.


----------



## Gavjenks (May 22, 2013)

Okay here you go.  Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally, and put them on imgur:
*imgur: the simple image sharer*

Click back and forth from image #1 and image #2 at the top of the screen.

Or look below (these are smaller for some reason than the imgur ones).  My version from the jpeg on the left, raw version on the right.


----------



## bratkinson (May 23, 2013)

I'll bet the audiophiles argued long and hard about the "advantages" of digital recording over analog. Analog wins, hands down.  Still does.  So do tubes.

Enter digital photography. In the pre-high pixel days, film won, hands down. Nowadays, digital wins both for color depth and pixel density, in my book. And for what it's worth, L-JPG and RAW-->LR4-->L-JPG look to be th same from my 5D3. But then, I'm an old geezer with glasses not looking for scientific and/or marketable results.

Maybe someday, someone will create a sensel that accurately differentiates every 0.0001 angstrum of light wavelengths and we'll have 10 to the 99th power of bit-level color depth.


----------



## amolitor (May 23, 2013)

Gavjenks said:


> Okay here you go.  Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally,



These two images look radically different. Was that your intention? What I take to be yours seems to lack shadow detail, and has quite ugly artifacting on the small awning on the extreme left of the frame. The colors seem less appealing, as well, but that might be simply that you like more saturation.


----------



## Helen B (May 23, 2013)

Gavjenks said:


> Okay here you go.  Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally, and put them on imgur:
> *imgur: the simple image sharer*
> 
> Click back and forth from image #1 and image #2 at the top of the screen.
> ...



Both the shadow and highlight detail in the raw version are better, and it is not difficult to see the difference. The clouds in the JPEG version have featureless patches, and the shadows are blocked up. Many of the mid tones have that 'stretched too much look'. 

Are you happy with the JPEG?


----------



## Garbz (May 23, 2013)

Gavjenks said:


> Setting your picture style and white balance is something you would do only once for a given lighting situation (e.g. set it once when you start in mid day, then maybe reset at sunset, or if you switch to indoors, etc.), outside of extraordinary circumstances.



Yes except when a subject suddenly moves into the shade, or it so happens the next person you took a photo of is wearing a different coloured shirt. Man if you could suggest three different settings that would get me remotely in the ballpark every time then I and everyone here would be eternally grateful. But life doesn't work like that. Heck I'm sitting here at my desk and I see 4 different light sources with different colour balances on each and I don't even have daylight coming in from outside to mess things up. I think you're wasting valuable field time that you could be spending at home



Gavjenks said:


> Shooting jpeg, i was able to fire at full speed for something like 50 shots in a row before I gave up and assumed it was just infinite.
> 
> Shooting RAW only, my camera stopped after 7 shots and had to sit there and think for like 2 seconds.  Then each subsequent photo was about 1 per second instead of 4, with thinking in between each time.



Not quite sure what the argument here is. If you routinely find yourself hitting the buffer limit then maybe you should have bought a better camera given the one you have is going to have a wreaked shutter quite early in its life. Action isn't continuous and those 7 bursts followed by gaps will in any normal scenario give you a break after a few shots to let your camera catch up. Even at the races where my goal was to capture as many racers as possible in an attempt to sell as many photos as possible you typically only fire three shots, recompose, fire three shots, recompose, etc. 



> The actual thing I am describing however, is.  "Go do this thing that requires a bit of extra complexity, but not as much as dealing with RAWs, and let's see if it has any impact on image quality."
> 
> At *WORST*, this is equally as complex as dealing with RAWs.  Even in optimal circumstances for your side of the argument, where somebody simply auto converts every single RAW to jpeg without touching their keyboard (using it only as a fallback safety net for catastrophic mistakes), it still equates to pushing a couple extra buttons vs. pushing a couple of other extra buttons.



Again I disagree. What you were describing is getting in the ballpark using camera settings and playing in camera menus in the field, often on a hard to see LCD screen (if shooting in the sun) and then doing some work in photoshop when you get home. How is this MORE difficult than just doing work in photoshop anyway. At *BEST* YOUR method is equally as simple as dealing with RAWs. And then you confirm it in your arguement saying it's pushing buttons vs pushing buttons. Yes, but you're not pushing buttons in the field viewing on an uncalibrated display, in uncontrollable light sources, when you should be taking photos.

My final thoughts on this are, if you're going to touch up the photos anyway why not get a program that treats RAWs and JPEGs identically? Honestly I open up Lightroom and if it weren't for the fact that some of the sliders don't work as well, and some options are missing I wouldn't know if I were working on a JPEG instead of a RAW.


----------



## Helen B (May 23, 2013)

The colour space issue seems to have been pretty much ignored, even though it opened this thread. Getting correct single-point white balance at the time of shooting does nothing to enlarge the color space itself.


----------



## Ysarex (May 23, 2013)

Gavjenks said:


> Okay here you go.  Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally, and put them on imgur:
> *imgur: the simple image sharer*
> 
> Click back and forth from image #1 and image #2 at the top of the screen.
> ...



Nice try, but you clipped the highlights. I did not clip the highlights. Clipping the highlights is photo felony #1. I *NEVER* clip the highlights. Helen saw it right away, "featureless patches in the clouds."

In your defense the highlights were clipped in the JPEG when you got it so the best you could do is reduce some of that clipping to "featureless patches" since there is no fix for blown highlights. Even though you did that you still clipped the highlights for example on the top of the white hand rail. You talk like you know something and I was anticipating you would reject the JPEG once you noticed the highlight clipping and we'd have to get into a discussion about exposure. It's worth stressing here that since the highlights are not clipped in my version then they must not have been clipped in the raw capture therefore the exposure was good. It was in fact textbook nailed.

In your original post you came to this faulty conclusion and posted it as a challenge, _"*Conclusion:* There shouldn't really be any technical reason why it  is more useful to edit a RAW than to edit a jpeg after first converting  it to 16 or 32 bit."_ In my response I told you why your conclusion was faulty. I'll repeat myself, _"The technical reason is that the camera JPEG processing software isn't  adequately capable of rendering the photographer's intent and, after the  fact with the raw data discarded, it is then usually impossible to  realize the photographer's intent from the JPEG file."_

That was high contrast lighting, but nothing really extreme. It's sunny sidelight just starting to shift to backlight. I knew from experience that this level of moderate stress was all it would take for the camera JPEG processing software to fail. I told you that as well in my original post, _"The problem is that what you're suggesting as an approach to taking the  photo isn't workable over a sufficient range of conditions -- at least  not for me. And those tons and tons of settings available in our cameras  are sub-crude and entirely inadequate to the task given what we know is  possible once we have the raw file."_

And the camera software did fail with your recommended settings applied. If we crank up the stress level by notching up the lighting contrast even more, the camera software will fail even more. I like to be able to work effectively under those more difficult conditions. Using your methodology you have to sit on the sidelines. You could argue now that you had the option in this example photo to reduce the exposure and help the poor camera software not clip the highlights. I really hope you have enough sense not to try and make that argument.

In defense of shooting JPEGs: there are those of us in the varied sub-disciplines of photography who have to shoot camera JPEGs (sports/action, journalism). For those specialists the controls in the cameras to assist in achieving a best possible result are invaluable and important tools. I'm glad those tools exist for them and I can adjust my expectations appropriately when I view their work. I hope the camera manufactures continue to improve those tools for their benefit.

But right now those tools place an artificial and unnecessary limit on what we can do when we don't for example have to meet a publication deadline. Why would you chose to tie your shoelaces together before you start a race?

Assuming you have a JPEG with all the info you need and can effect a light repair having converted it to 16 bit, will it hold up reasonably well? Yes. Getting that JPEG is however another matter and your estimation of what the tons and tons of controls in the camera provide is way off mark. Judobreaker told you that in the very first response to your post and as far as I'm concerned he settled it right then and there. Now you have empirical proof.

Joe


----------



## amolitor (May 23, 2013)

The JPEG pulled together by the OP suggests to me that he or she simply doesn't see technical minutiae as well as some of us. There's nothing wrong with this, it doesn't make the OP evil. JPEG is specifically designed so that the average viewer can't see that stuff. It does mean that stuff that bugs YOU or ME isn't going to bug the OP.

I am almost certain that an image much more like the one from the RAW could be pulled out of the JPEG, but you gotta be able to *see* the stuff before you can make the stuff happen.

Hence my other thread.


----------



## o hey tyler (May 23, 2013)

Well, that clears that up. 

Everyone on the forum other than GavJenks: 1

GavJenks: 0


----------



## TruckerDave (May 23, 2013)

480sparky said:


> I guess we need something to bicker about since the Nikon v. Canon thing has been beat to death.
> 
> Perhaps we should debate Chevy v. Ford.



 Or,  Coke v Pepsi....vhs v betamax...mac v windows.....iphone v android.....


----------



## Gavjenks (May 23, 2013)

amolitor said:


> Gavjenks said:
> 
> 
> > Okay here you go.  Images wouldn't paste full size into the forum, so I made them both the same resolution as the raw one you gave me originally,
> ...



All the saturation is added, so that is not the fault of jpeg editing, as I could have simply added less.  The difference is only about 2% in most areas of the image... I added more than I would have normally for sure in an attempt to try to match his photo.  I'd say 2% is a pretty good match.  Try loading up both and eye droppering them yourself.

The sharpening (artifacts) was also added, and I used more than I would have personally, again to match the photo.  Notably the bricks.  I did sharpening as a layer, non destructively, so I could have simply gone back and erased the awning part upon noticing artifacting, which I didn't.  And it would have fixed that.  Admittedly, sharpening is the trickiest part.  However, even with a RAW, if you want best quality, you need to learn to do some difficult sharpening techniques ANYWAY, because the raw converters will not do things like apply separate levels or types of sharpening to areas with different artistic sharpening needs in the same image.

As for highlight and shadow clipping, here are the luminosity histograms to show that the clipping is not really as bad as all that in any of the pictures, and not very different either. Almost identical highlight clipping profiles:



Nor do I see much difference in the actual photos.  I can see more details like reflected leaves in the RAW, but that's because I played them down on purpose due to considering them environmental pollution (it's a house, not a tree).  And the clouds may have the illusion of being more clipped, because I made them whiter (rather than what i thought to be a less desirable blue cast and wimpy appearance).  Seeming additional clipping in the clouds is an illusion of them being white, I think.  Histograms don't lie.  Nor is it that the histogram is measuring two different things.  Spot checking the clouds in particular with eyedroppers and local histograms reveals neither being very clipped at all (the railings are closer than the clouds are).

Also if you examine both clouds very closely, the RAW version's does not actually have more "steps" or shades to it than the jpeg version. The transition is not smoother.  It's just shifted a little bit differently, and again, more noticeable due to being bluer, so that you have color and luminosity cues together.



And please note again: *This is not a challenge of "can gav guess the settings you used when you don't tell him the settings (which I asked for btw)."* We aren't testing my eyes or my color or artistic judgment.  We're testing the technical limitations of jpeg editing.  And *the technical clipping is indeed no worse than the version made from a converted RAW. Nor is there color banding, etc.*  The two photos are not IDENTICAL, because I had to guess a lot of things (and also, he seems to have done a lot of edits other than just converting from RAW, such as keystoning for perspective, etc. So even more unnecessary guessing). But they are pretty damn close, enough so IMO for purposes of pointing out that you can do the same SORTS of things to the jpeg without getting posterization or other major issues.  Most complaints here seem to be based on artistic complaints that arose from me guessing things or in some cases attempting to improve slightly.


----------



## amolitor (May 23, 2013)

Oh, sorry. I didn't look at the histograms, I just looked at the pictures to see how similar they were. Now I know.


----------



## Gavjenks (May 23, 2013)

Also btw for those looking at the histograms: the reason that the raw and the jpeg both pile up on the left side so much more than the original jpeg is because of the weird solid black border that he included in his image for some reason, and that I copied in attempting to match it. So most of those are human-added feathered solid black pixels, not really part of the image per se.



> Oh, sorry. I didn't look at the histograms, I just looked at the pictures to see how similar they were. Now I know.


The histogram proves that the data still exists, which is the point of this thread.  The point is not to prove how good of an art forger I am, or how well I can guess editing steps that were not told to me, or artistic intent that was not told to me.

Regardless of whether you like my version better, the data still exists.  It is not actually clipped any more than the raw version, and it is not any more posterized. So if you prefer subtler, bluer clouds like he had, then you can simply make them bluer, and then lower the lightness a bit and play with the curves until you get the more visible texture you desire. The information is there to allow you to do that without ending up with any more posterization than his version has.

it's just that apparently, based on people's reactions in this thread, the diffrence between 238/238/238 and 233/233/233 is not as visible to the human eye as the difference between 220/220/245 and 215/215/240.  It is equally as visible to a computer, though, and the computer should be perfectly capable of shifting it over to where you can see it, without any loss of data, if that's what you want.  I just thought it didn't look as good.


----------



## Helen B (May 23, 2013)

I just saw featureless patches in the white clouds (that's not the same as clipping) and poor shadow detail. When I select the same patch of what I see as featureless (by putting both images into the same document and making the relevant parts coincident, then making a selection) and look at the histogram it tells me exactly the same story. The raw histogram is broader and has a greater standard deviation (12 against 2.4 for the area I selected). Technical issues aside, it is visually obvious that there is more subtle gradation and hue in the highlights of the raw-originated file, and the appearance is what matters. Isn't it obvious to you?


----------



## Benco (May 23, 2013)

Helen B said:


> I just saw featureless patches in the white clouds (that's not the same as clipping) and poor shadow detail. When I select the same patch of what I see as featureless (by putting both images into the same document and making the relevant parts coincident, then making a selection) and look at the histogram it tells me exactly the same story. The raw histogram is broader and has a greater standard deviation (12 against 2.4 for the area I selected). Technical issues aside, it is visually obvious that there is more subtle gradation and hue in the highlights of the raw-originated file, and the appearance is what matters. *Isn't it obvious to you*?



Clearly it's not. :banghead:


----------



## Gavjenks (May 23, 2013)

> Isn't it obvious to you?



1) It is obvious, yes.
2) I like it better with less gradation and bolder whites.  The raw version clouds look "wimpy" to me. I did this on purpose, whether subconsciously or not.
3) Even though the raw version has more visible gradation, in the brightest parts of the clouds, the gradation only occurs in 3-5 distinct steps/shades.  I actually think this looks a little bit posterized. Making it brighter in my opinion also makes it look better by masking the posterization a bit (people can't tell differences as well at brighter ends of the range, even if its the same number of steps in a computer), without looking completely blown out. "nearly but not quite blown out" looks better to me than "gradation, but in visible steps." At least for white objects like clouds.  Apparently I'm in the minority on that.
4) How do you measure standard deviation? That's a neat trick. And if it is indeed that much different, then maybe there was some significant data loss after all.  It didn't look nearly that different to me from the graphs, but I don't actually have real statistics like that.  Just eyeballing the curves.  

If you tell me how, then I will try to run some statistics on the raw converted image versus the original jpeg for the whole cloud region and post them.  That would be more in line with the technical claims in this thread, while avoiding issues of artistic taste.


----------



## amolitor (May 23, 2013)

I don't understand at all now.

You're saying that you can take the JPEG version of a photo and make a picture out of it that's different that the one Joe made out of the corresponding RAW file? I don't think anyone's gonna argue with you on that point.

I truly don't understand what on earth the thrust of this argument can be. I thought you were saying that you could get the same results out of JPEG as out of RAW, if only you were careful in exposing the JPEG, and in processing it afterwards. Now you're apparently not saying that? Or what?


----------



## Ysarex (May 23, 2013)

Gavjenks said:


> ....And *the technical clipping is indeed no worse than the version made from a converted RAW.*



Blatantly wrong. Shadow clipping is a misdemeanor at best. Highlight  clipping is a felony. There's one thing worse than highlight clipping  and that's filling in those blown highlights with a flat dead tone.  Here's a threshold test of the camera JPEG that shows the highlights  that the camera software clipped.



Your version of the photo and your histogram doesn't show the same  clipping in the highlights because you filled in those blown areas with a  solid tone -- that's the felony that comes with a mandatory life  sentence. Blown highlights are ugly -- filling them in with a solid tone  is butt ugly.

There is no highlight clipping whatsoever in my version of the photo.  From my perspective that was the point of this exercise. The camera  software couldn't deliver a JPEG under moderately stressful side  lighting. That JPEG from the camera was damaged BEYOND REPAIR by the  camera software. *You shouldn't have even made the attempt to process it!*  To suggest that what you did filling in those highlights compares well  to processing the raw file in which those highlights are not clipped is  laughable.

*AND* Let's not forget that this  was moderately stressful sidelight. The lighting contrast can get much  higher and from this point on your camera can only fail worse. Your  single option shooting JPEGs in similar lighting is to reduce exposure  which is an even worse idea than trying to repair camera JPEGs. I chose  this scene and lighting contrast because I expected the camera failure  would be minor and I was curious to see how you'd react. What do you do  under higher contrast conditions?

You've been granted that, up to the limits of the camera software's  ability to deliver a JPEG that isn't badly damaged, some light repair  with the photo converted to 16 bit produces serviceable results. You can  work like that if you want to, knock yourself out. I'm going to  continue to work in a manner that permits me to photograph a whole lot  of stuff that you can't without trying to convince yourself that pasty  filled in highlights really do look OK. You've got an answer to your  original post. Don't make me go outside and take a backlit photo for  you.

Joe


----------



## Ysarex (May 23, 2013)

Gavjenks said:


> The histogram proves that the data still exists, which is the point of this thread.  The point is not to prove how good of an art forger I am, or how well I can guess editing steps that were not told to me, or artistic intent that was not told to me.



Your histogram proves you filled in the clipped highlights with a solid tone -- the featureless patches (I think posterization is a common word for that) that Helen is referring to. I just posted a threshold test for the original JPEG that clearly shows the highlight clipping caused by the camera. (I wouldn't call what you did forgery.)



Gavjenks said:


> Regardless of whether you like my version better, the data still exists.



The non-photographic dead tones you created exists and it looks out of place and just butt ugly. The original JPEG with it's clipped highlights is there for anyone to examine.



Gavjenks said:


> It is not actually clipped any more than the raw version, and it is not any more posterized.



There is no highlight clipping in the version I processed from the raw file and I didn't fake in any flat tones to fill in the clipping.

Joe


----------



## 480sparky (May 23, 2013)

You guys argue all you want:









I'm gonna go take some pictures......


----------



## Gavjenks (May 23, 2013)

Yes okay there are like 30 pixels of clipping when you mask it like that. I really disagree that that amount of clipping you show in the clipping mask you made above is anything very significant to worry about. I am sitting there myself straining to look at that exact part of the image, knowing full well what I'm supposed to be seeing that's so "butt ugly" and am unable to even MAKE myself see it as bad looking at all.  It's a white cloud, 0.5% of it is pure white, and the areas around that part gradient smoothly into that pure white, not abruptly .  So what?

The camera seems to have done a pretty darn good job in general in choosing a  sampling range from the RAW in order to meet your request of very low  contrast

However:



> The lighting contrast can get much  higher and from this point on your camera can only fail worse.



This is a fair point.  Based on the fact that it had any clipping here, I imagine that if you used 0,-4,0,0 settings on that first photo you took of the backlit tree, it would indeed have failed pretty badly even at those maxed out negative 4 settings.

I'll try to test it later when I get home with my camera, with a bright lamp in front of a black window shade or something, but I suspect that you're right, and it'ss going to be fairly bad.



If so, then yes, that would lead me to the conclusion that shooting RAW is helpful in normal circumstances practically.  However, this doesn't seem like it SHOULD be the case theoretically.  Rather, it is a failing of Canon's camera settings that you can't make them strong enough to meet your desires.  I.e., it only lets you have some small proportion of the influence that you should have over the RAW->Jpeg sampling pattern, for no apparently good reason.


----------



## Ysarex (May 23, 2013)

Gavjenks said:


> Yes okay there are like 30 pixels of clipping when you mask it like that. I really disagree that that amount of clipping you show in the clipping mask you made above is anything very significant to worry about. I am sitting there myself straining to look at that exact part of the image, knowing full well what I'm supposed to be seeing that's so "butt ugly" and am unable to even MAKE myself see it as bad looking at all.  It's a white cloud, 0.5% of it is pure white, and the areas around that part gradient smoothly into that pure white, not abruptly .  So what?
> 
> The camera seems to have done a pretty darn good job in general in choosing a  sampling range from the RAW in order to meet your request of very low  contrast
> 
> ...



That's a lot more than 30 pixels in the full res version of the photo. Bottom line: Camera software can't do it. We've known this for a long time. You can repair a JPEG with decent success if it's taken under less extreme lighting conditions but really why? Having a uniform workflow is helpful. Many of us find not having to deal with anything except evaluating the lighting and nailing the exposure is in fact a real advantage. And I would much rather process a raw file than try to repair a JPEG. I find it easier and I get it done faster. I like faster.

It was obviously a mistake to give you a photo where the camera only failed slightly. Yesterday when I went to take that photo I had to also take out the trash. So I started out in the back alley and the first thing I did was photograph my neighbor's backlit roses in the backyard. The camera software failed miserably, but I went ahead around front to get you that shot of the house where the camera fail was less obvious. My mistake. Here's the roses:




I'm not spending a lot of time with this so I just popped the raw file through Photo Ninja made sure the highlights weren't clipped (cause they're not in the raw file) did a slight rotate and crop and got the contrast about right. The camera JPEG is on the right. Compare the sky and the clouds in the sky. There is no clipping in the raw file and the exposure is excellent. The camera software failure is massive. You want to verify the JPEG (or process it) here it is: roses.



Gavjenks said:


> If so, then yes, that would lead me to the  conclusion that shooting RAW is helpful in normal circumstances  practically.  However, this doesn't seem like it SHOULD be the case  theoretically.  Rather, it is a failing of Canon's camera settings that  you can't make them strong enough to meet your desires.  I.e., it only  lets you have some small proportion of the influence that you should  have over the RAW->Jpeg sampling pattern, for no apparently good  reason.



It's not just Canon. I can assure it's all of them. Their position is reasonable. The point of a JPEG is that it's finished. If you're using the camera to deliver a finished JPEG then there have to be appropriate limits to how far the software adjustment controls can be pushed before you end up with something that doesn't look at all finished. For those who require the control, insist on being able to shoot difficult conditions and intend to edit their photos the camera manufacturers have provided the appropriate access.

Joe


----------



## amolitor (May 23, 2013)

Joe makes an excellent point.

JPEG is not intended as a format that "contains all the sensor information, so that you can process it into a final image", there's already a format for that which is much better suited to that role. It's intended as a SOOC picture format, or at least to render acceptably SOOC.


----------



## Gavjenks (May 23, 2013)

In light of the apparent failure of software, I propose a new intermediate format available from camera software, described in this thread:
http://www.thephotoforum.com/forum/...w-nor-jpeg-hypothetical-idea.html#post2962371

Check it out!

As mentioned, jpeg is intended as a finished SOOC.  RAW is intended for hardcore editing with no compromises or any attempt at all to cut down on pointless data.

Which leaves a pretty obvious unfilled niche of an in between format that compresses things which don't matter, but intentionally preserves data in the areas that do. See thread for a proposal to do exactly that.

It would require no fiddling in the field to work right.


----------



## o hey tyler (May 23, 2013)

Facepalm.jpeg


----------



## 480sparky (May 23, 2013)

o hey tyler said:


> Facepalm.jpeg


----------



## Benco (May 23, 2013)

Gavjenks said:


> In light of the apparent failure of software, I propose a new intermediate format available from camera software, described in this thread:
> http://www.thephotoforum.com/forum/...w-nor-jpeg-hypothetical-idea.html#post2962371
> 
> Check it out!
> ...



For there to be a niche there would need to be something wrong/difficult/compromised about raw...but there isn't.


----------



## Helen B (May 23, 2013)

There's already compressed raw, which works out at about 8 bits per pixel - and if it is raw from a Bayer or X-Trans sensor there is only one channel per pixel.


----------



## Derrel (May 23, 2013)

Who here knows anything about Kodak's ERI or extended range imaging JPEG format? They had that in the 14n, and I think the SLR/n and SLR/c models as well.

It's been over a decade, but it seemed like ERI JPEG format was going places. Buuuut then again, Kodak was "going places" back then too...like down the drain...

Rob Galbraith DPI: Kodak ERI technology brings power of RAW format to JPEGs


----------



## Ysarex (May 23, 2013)

Gavjenks said:


> In light of the apparent failure of software, I propose a new intermediate format available from camera software, described in this thread:
> http://www.thephotoforum.com/forum/...w-nor-jpeg-hypothetical-idea.html#post2962371
> 
> Check it out!
> ...



No data is pointless until a human being has evaluated it and made a determination. In the end this is really all about just one thing. F****ing software is f****ing blind. I can see, software can't. Keep the f****ing software the h*ll away from my data. Thank you.

Joe

edit: had a couple beers.


----------



## Garbz (May 25, 2013)

Gavjenks said:


> Which leaves a pretty obvious unfilled niche of an in between format that compresses things which don't matter, but intentionally preserves data in the areas that do.



You mean like Compressed NEF which even offers you the choice of a reduced bitrate to save space and a choice between lossy and lossless RAWs and is available on Nikon cameras, and I'm sure Canon has an equivalent? 

I don't understand your need to re-invent the wheel. Either you want the data for editing or you don't. If you want a little data then compress the RAW and drop the bitrate down.


----------



## Gavjenks (May 25, 2013)

Guys... I linked to the thread about this topic, and you're discussing it here.  that's not a great way to get responses back on comments or questions.  

Very briefly: Compressed NEF and other RAW compressions are not at all the same.  They are either lossless, or blind to the image details (like ERI or the block based cRAW), and thus their loss can potentially be catastrophic compared to RAW if it happens to be the wrong kind of image that doesn't optimize them.  I am not particularly surprised that they failed, for this reason.

What I am suggesting is a lossy compression, but where the loss is *custom targeted* to the areas of the data that have few or no pixels in them and that therefore don't actually matter or require high data density.  The data that was being "lost" was data that wasn't needed by the photographer for anything.  This is the only compression format mentioned here thus far, as far as I can see, that actually uses the image to intelligently figure out where to compress, instead of just applying the same algorithm to every pixel or every 16 pixels or whatever.

For example, if you have 3500 different values of green shades that NO PIXELS in your image have, then why the hell are you taking up bandwidth (in the form of requiring larger color codes to potentially code for that color) when don't even exist in your image??? None of the methods mentioned above would compress that out. 

Getting rid of those would be considered "lossy" in hardcore computer science terms, but would not in fact correlate to any loss of data that the photographer actually cares about.  This is how my algorithm works (though very slightly more aggressively, as it doesn't have to be compeltely empty of pixels to be compressed, but pretty close to this, honestly).


Anyway, further discussion of that thread should go in that thread, and I don't intend to check back here to follow up on this post (where in fact another equally promising idea already exists in a pixel compression format instead of rgb/l based format).  Link again:
http://www.thephotoforum.com/forum/...w-nor-jpeg-hypothetical-idea.html#post2962371


----------

