Help - Search - Members - Calendar
Full Version: Connecting Exposure, ISO and Noise
Luminous Landscape Forum > Equipment & Techniques > Digital Cameras & Shooting Techniques
Panopeeper
I am sure the vast majority of posters of these forums understand the connection between exposure, ISO and noise, but there may be many readers believing popular misconceptions like "my shots are noisy even at ISO 100", or "look how few noise is in this shot at ISO 12800".

I put a demonstration together showing the relationship between noise, exposure and ISO: The Source Of Noise

I ask for constructive critique of the content, form as well as language.


Thanks
Luis Argerich
Great article, very clear and I love the demonstrations. I know one shouldn't have the need to demonstrate the truth but people many times say "I don't believe it" so the examples are great.

thanks!
Luigi
PeterAit
QUOTE (luigis @ Feb 25 2009, 12:46 PM) *
Great article, very clear and I love the demonstrations. I know one shouldn't have the need to demonstrate the truth but people many times say "I don't believe it" so the examples are great.

thanks!
Luigi


And sometimes people don't believe the truth even after you demonstrate it!

Peter
Guillermo Luijk
Hi Gabor. I find the article very good if one already knows the interconnection between exposure, ISO and noise, but not so good for those who still don't understand it. You demonstrate the relations between those variables rather than explain them.

If I were to measure noise on my RAW files and would like to use RAWnalyze, this document would be just the perfect source of information.

However for those who are still finding out the connections you try to explain, the article could be confusing. The main problem I always found when talking to people about these subject is:

"Increasing the ISO setting decreases the noise up to a certain ISO level; further increasing the ISO setting is practically useless for the raw image."

This is a very advanced conclusion for someone who believed until now that: "the higher the ISO the more noise you get".

People are used to set ISO on their cameras and have apperture/shutter automatically adjusted by them. In that situation the variable ISO has not been isolated (which is actually the right way to analyse it), and I think this is the first thing that must be explained before saying to anyone that the higher the ISO, the higher the SNR. Otherwise they will not understand what you're talking about IMO.

BR
dwdallam
QUOTE (luigis @ Feb 25 2009, 05:46 PM) *
Great article, very clear and I love the demonstrations. I know one shouldn't have the need to demonstrate the truth but people many times say "I don't believe it" so the examples are great.

thanks!
Luigi


Well people contradict their own beliefs in spite of truth all of the time. For instance, in the USA most people believe in the Constitutional phrase that "All men are created equal," which really means that all people deserve the same treatment. The same people who believed the equality phrase took until 1921 to see that women must be allowed to vote, or they were holding contradictory positions. The "Truth" I speak of here is not in the phrase itself, but is in the contradiction. Even though all contradictions are false (which is true), people still hold on to contradictory belief systems.
dwdallam
QUOTE (GLuijk @ Feb 26 2009, 12:51 AM) *
Hi Gabor. I find the article very good if one already knows the interconnection between exposure, ISO and noise, but not so good for those who still don't understand it. You demonstrate the relations between those variables rather than explain them.

If I were to measure noise on my RAW files and would like to use RAWnalyze, this document would be just the perfect source of information.

However for those who are still finding out the connections you try to explain, the article could be confusing. The main problem I always found when talking to people about these subject is:

"Increasing the ISO setting decreases the noise up to a certain ISO level; further increasing the ISO setting is practically useless for the raw image."

This is a very advanced conclusion for someone who believed until now that: "the higher the ISO the more noise you get".

People are used to set ISO on their cameras and have apperture/shutter automatically adjusted by them. In that situation the variable ISO has not been isolated (which is actually the right way to analyse it), and I think this is the first thing that must be explained before saying to anyone that the higher the ISO, the higher the SNR. Otherwise they will not understand what you're talking about IMO.

BR



That sounds like a very astute and valid objection and observation.
bjanes
QUOTE (GLuijk @ Feb 25 2009, 06:51 PM) *
However for those who are still finding out the connections you try to explain, the article could be confusing. The main problem I always found when talking to people about these subject is:

"Increasing the ISO setting decreases the noise up to a certain ISO level; further increasing the ISO setting is practically useless for the raw image."

This is a very advanced conclusion for someone who believed until now that: "the higher the ISO the more noise you get".

People are used to set ISO on their cameras and have apperture/shutter automatically adjusted by them. In that situation the variable ISO has not been isolated (which is actually the right way to analyse it), and I think this is the first thing that must be explained before saying to anyone that the higher the ISO, the higher the SNR. Otherwise they will not understand what you're talking about IMO.

BR


This is an interesting dialog between two of the most knowledgeable and perceptive forum members. I agree with Guillermo that Gabor has brilliantly demonstrated some basic characteristics of noise and its sources, but without some theoretical considerations of the major sources of noise--read noise, shot noise, and PRNU (pixel response non uniformity)-- most readers will have trouble following the analysis. I would suggest that interested persons should read Emil Martinec's essay on noise and dynamic range in conjunction with Gabor's demonstration.

For a given exposure, the effect of ISO on noise is related to the contribution of read noise. Read noise is further complicated by whether it is expressed in electrons or ADUs, which are related to the sensor gain. Read noise can be reduced by increasing the ISO up to a certain point, beyond which ISO makes little difference. Gabor's demonstration is for a Canon camera, and the relationship between ISO and read noise is different for Canon and Nikon cameras as shown in Figures 15 and 15 of Emil's paper. It would be interesting to repeat the analysis for a Nikon D3 or other latest generation Nikon DSRL.

As Gabor has shown, noise in the color channels is related to exposure. Since current sensors are most sensitive to green light, the red and blue channels are relatively under-exposed with daylight exposures. With tungsten exposures the blue channel may be severely underexposed, leading to more noise. This may cause confusion for some.

Bill
marcmccalmont
I sure would like to know for a given situation which settings would give me minimum noise!
If I have a choice for a longer exposure but at a lower ISO where is my minimum noise?
When I lose an (or all) engines in an airplane, the flight management computer calculates an optimum airspeed to maximize my drift down distance.
Why can't my camera (or chart) calculate my optimum settings for minimum noise? I know ISO 100 for daytime shots, but in low light where am I better off? 1/2 sec @ISO 200 or 1/8 sec @ ISO 800? I would like to see some sort of "Minimum Noise Curve" say ISO vs. exposure time?
Marc
Luis Argerich
Longer exposure is always the way to go -If possible- in your example go with 1/2 sec if you can. The more light the sensor can get the better the signal/noise ratio will be.
The problem is that a long exposure is not always possible because subjects move, you may be without a tripod, etc. Only when you can't increase the exposure you can push the ISO to reduce noise.
Your camera has no idea if you are shooting a landscape or a person, handheld or tripod mounted so it can't take those decisions for you.

Luigi

QUOTE (marcmccalmont @ Feb 26 2009, 07:35 PM) *
I sure would like to know for a given situation which settings would give me minimum noise!
If I have a choice for a longer exposure but at a lower ISO where is my minimum noise?
When I lose an (or all) engines in an airplane, the flight management computer calculates an optimum airspeed to maximize my drift down distance.
Why can't my camera (or chart) calculate my optimum settings for minimum noise? I know ISO 100 for daytime shots, but in low light where am I better off? 1/2 sec @ISO 200 or 1/8 sec @ ISO 800? I would like to see some sort of "Minimum Noise Curve" say ISO vs. exposure time?
Marc

ejmartin
QUOTE (marcmccalmont @ Feb 26 2009, 04:35 PM) *
I sure would like to know for a given situation which settings would give me minimum noise!
If I have a choice for a longer exposure but at a lower ISO where is my minimum noise?
When I lose an (or all) engines in an airplane, the flight management computer calculates an optimum airspeed to maximize my drift down distance.
Why can't my camera (or chart) calculate my optimum settings for minimum noise? I know ISO 100 for daytime shots, but in low light where am I better off? 1/2 sec @ISO 200 or 1/8 sec @ ISO 800? I would like to see some sort of "Minimum Noise Curve" say ISO vs. exposure time?
Marc


The basic rule of thumb in most situations is

1) Choose the minimum aperture for the required depth of field.
2) Choose the minimum shutter speed required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

There is one major exception to this rule of thumb, when item (3) suggests an ISO higher than about 1600. In this case, one is usually better off underexposing at ISO 1600 at the aperture and shutter speed suggested by (1) and (2), since there is little to no improvement in S/N above ISO on most current cameras. The threshold for making such a decision varies from camera model to camera model; for the 5D2, which has a lot of high ISO banding, it may make sense to continue raising the ISO to 3200 in step (3), since it lowers the banding noise relative to signal, while for the new Sony sensors such as in the D300 and D3x, the benefits of raising the ISO tail off somewhere between ISO 800 and 1600.

There are also a few footnotes to the above, such as the fact that there is almost nothing to be gained from using "intermediate" ISO's (the 1/3 stops in between ISO 100/200/400/800/1600 etc).
Luis Argerich
#1 should be maximum aperture (lowest F-number)
#2 should be maximum shutter speed (the longer the better)

QUOTE (ejmartin @ Feb 26 2009, 10:50 PM) *
The basic rule of thumb in most situations is

1) Choose the minimum aperture for the required depth of field.
2) Choose the minimum shutter speed required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

There is one major exception to this rule of thumb, when item (3) suggests an ISO higher than about 1600. In this case, one is usually better off underexposing at ISO 1600 at the aperture and shutter speed suggested by (1) and (2), since there is little to no improvement in S/N above ISO on most current cameras. The threshold for making such a decision varies from camera model to camera model; for the 5D2, which has a lot of high ISO banding, it may make sense to continue raising the ISO to 3200 in step (3), since it lowers the banding noise relative to signal, while for the new Sony sensors such as in the D300 and D3x, the benefits of raising the ISO tail off somewhere between ISO 800 and 1600.

There are also a few footnotes to the above, such as the fact that there is almost nothing to be gained from using "intermediate" ISO's (the 1/3 stops in between ISO 100/200/400/800/1600 etc).

Panopeeper
QUOTE (ejmartin @ Feb 26 2009, 05:50 PM) *
for the 5D2, which has a lot of high ISO banding, it may make sense to continue raising the ISO to 3200 in step (3), since it lowers the banding noise relative to signal

Where do you see this happening?
scubarob639
QUOTE (dwdallam @ Feb 26 2009, 07:38 AM) *
Well people contradict their own beliefs in spite of truth all of the time. For instance, in the USA most people believe in the Constitutional phrase that "All men are created equal," which really means that all people deserve the same treatment. The same people who believed the equality phrase took until 1921 to see that women must be allowed to vote, or they were holding contradictory positions. The "Truth" I speak of here is not in the phrase itself, but is in the contradiction. Even though all contradictions are false (which is true), people still hold on to contradictory belief systems.


The point you make I understand, however the phrase "All men are created equal" is not in our constitution, and a women's right to vote is not an unalienable right as defined by our Declaration of Independence.

Rob
ejmartin
QUOTE (Panopeeper @ Feb 26 2009, 09:05 PM) *
Where do you see this happening?


Since banding noise is a "feature" of the ISO amplifier/ADC, and thus not amplified when the ISO is increased, the signal is boosted relative to the banding noise by increasing the ISO. The standard deviation of noise does not improve relative to signal beyond ISO 1600 in Canons, but on the other hand, banding noise is not a significant contributor to the standard deviation. Better measures of the amount of banding noise involve either vertical/horizontal averages, which don't decrease the vertical/horizontal banding but average out the spatially random noise; or use of the Fourier transform of black frames to isolate the banding component. I've not tested the 5D2, but I see this behavior on my 1D3.
ejmartin
QUOTE (luigis @ Feb 26 2009, 08:04 PM) *
#1 should be maximum aperture (lowest F-number)
#2 should be maximum shutter speed (the longer the better)


Yes on #1 -- minimum f-stop for required DOF.
On #2, the slowest exposure time in seconds that can freeze the action etc.
Panopeeper
QUOTE (ejmartin @ Feb 26 2009, 08:48 PM) *
Since banding noise is a "feature" of the ISO amplifier/ADC, and thus not amplified when the ISO is increased, the signal is boosted relative to the banding noise by increasing the ISO.

I guess any improvement should be apparent on the masked area, but I don't see that; the following captures show the masked area of the same sensor, with high contrast:


dwdallam
QUOTE (scubarob639 @ Feb 27 2009, 04:14 AM) *
The point you make I understand, however the phrase "All men are created equal" is not in our constitution, and a women's right to vote is not an unalienable right as defined by our Declaration of Independence.

Rob



That is true enough. The point is that people believed in equality for all people, but denied females the same right they themselves enjoyed, which presents a contradiction. And you are right about unalienable rights too, but if you take your position to it's logical conclusion and run a country by it, you get a very Draconian State. But this is exactly what I am talking about: It takes some people a very long time to come to the conclusions that are not self contradictory. For instance, it is neither an unalienable right for us to talk freely on the internet. Take that to it logical conclusion and it would be just dandy for our government to censor this medium, just like China does for its populace. Having a right to be female and have an education isn't unalienable right either, and you get my point.

I think your comment that "A woman's right to vote is not an unalienable right" is a fact, but would create an extremely specious argument form.
james_elliot
I think this sentence is more or less the translation of the fundamental sentence of the "Déclaration des droits de l'homme et du citoyen" of the french revolution.
The french sentence is very often translated in English by "All men are created free and equal" (the translation of "Tous les homme naissent libres et égaux"). However, the french sentence is indeed "Tous les hommes naissent libres et égaux en droit" which means "All men are created free and with equal rights".
However, women's right to vote in France was only established in 1944 (end of WWII, 150 years after the french revolution)... As Orwell's wrote: "all animals are equal but some animals are more equal than others" (Animal farm)

QUOTE (scubarob639 @ Feb 27 2009, 05:14 AM) *
The point you make I understand, however the phrase "All men are created equal" is not in our constitution, and a women's right to vote is not an unalienable right as defined by our Declaration of Independence.

Rob

marcmccalmont
QUOTE (ejmartin @ Feb 26 2009, 03:50 PM) *
The basic rule of thumb in most situations is

1) Choose the minimum aperture for the required depth of field.
2) Choose the minimum shutter speed required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

There is one major exception to this rule of thumb, when item (3) suggests an ISO higher than about 1600. In this case, one is usually better off underexposing at ISO 1600 at the aperture and shutter speed suggested by (1) and (2), since there is little to no improvement in S/N above ISO on most current cameras. The threshold for making such a decision varies from camera model to camera model; for the 5D2, which has a lot of high ISO banding, it may make sense to continue raising the ISO to 3200 in step (3), since it lowers the banding noise relative to signal, while for the new Sony sensors such as in the D300 and D3x, the benefits of raising the ISO tail off somewhere between ISO 800 and 1600.

There are also a few footnotes to the above, such as the fact that there is almost nothing to be gained from using "intermediate" ISO's (the 1/3 stops in between ISO 100/200/400/800/1600 etc).


Well I put your rule to the test with several low light exposures, 5DII, color chart and measured the noise with noiseware professional (and my eye), the 100 ISO 1 minute exposure was much cleaner than the 4 second ISO 1600 exposure!
Marc
ejmartin
QUOTE (marcmccalmont @ Feb 28 2009, 03:13 PM) *
Well I put your rule to the test with several low light exposures, 5DII, color chart and measured the noise with noiseware professional (and my eye), the 100 ISO 1 minute exposure was much cleaner than the 4 second ISO 1600 exposure!
Marc



Here's my test: 1D3 at ISO 100, 100% crop



and at ISO 1600 with the same Tv/Av:



the lower left square of the color chart is about 9 stops down from RAW saturation on the ISO 100 shot, IIRC.


Panopeeper
QUOTE (ejmartin @ Feb 28 2009, 02:34 PM) *
Here's my test: 1D3 at ISO 100, 100% crop

and at ISO 1600 with the same Tv/Av:

Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.
marcmccalmont
quote name='Panopeeper' date='Feb 28 2009, 03:07 PM' post='263679']
Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.
[/quote]

True but same result!
I thought that noise built up over time would exceed the noise in the analog amplifier at some point, I guess not. This was with dark frame noise reduction on, to make it a good real world test.
Marc

Click to view attachmentClick to view attachment[
dalethorn
QUOTE (scubarob639 @ Feb 26 2009, 08:14 PM) *
The point you make I understand, however the phrase "All men are created equal" is not in our constitution, and a women's right to vote is not an unalienable right as defined by our Declaration of Independence.

Rob


When the DOI said "all men" was it really referring to all persons, or just to males? You'd think that the founding fathers, some of the most astute and educated men in history, would have the same understanding of that issue. From what I've read, they had the same understanding of each other's biases, and that's what they best understood. Capt. Kirk said "no man" - Capt. Picard said "no one". So, if we had several alternate or independently-produced versions of the DOI at hand, we just might find a Kirk/Picard dialectic there. Don't you suppose people are short-changing themselves by reserving their utmost respect for only the "official" version of history?

None of this is intended to be political speech - just an illustration of understanding. Today, while photographing a Golden Tamarind at the zoo, I got a few decent exposures when the critter was in good light, and a few under-exposures when it moved away from the good light. Not surprisingly, the under-exposures were much noisier than the good exposures, although all of them were taken at ISO 400. But that's been my experience with any camera.
Fine_Art
QUOTE (ejmartin @ Feb 26 2009, 08:51 PM) *
Yes on #1 -- minimum f-stop for required DOF.
On #2, the slowest exposure time in seconds that can freeze the action etc.


I'll add that around the exposure time the same as shutter/mirror vibration time, blur is created. That duration should be avoided.
ejmartin
QUOTE (marcmccalmont @ Feb 28 2009, 07:16 PM) *
quote name='Panopeeper' date='Feb 28 2009, 03:07 PM' post='263679']
Somehow I have the feeling that your demo has nothing to do with what Marc posted. If I understand you correctly, you are comparing a 4EV underexposed shot @ ISO 100 with one using the same exposure but ISO 1600.

Marc compared two properly exposed shots, one @ ISO 100, the other @ 1600.


True but same result!
I thought that noise built up over time would exceed the noise in the analog amplifier at some point, I guess not. This was with dark frame noise reduction on, to make it a good real world test.
Marc


We didn't read each other's posts carefully enough. I didn't notice that you were changing the exposure in your test, and you didn't interpret my rule of thumb correctly. The rule of thumb was

1) Choose the minimum f-stop for the required depth of field.
2) Choose the minimum exposure time required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

If you were exposing at 1 min and ISO 100, and got a good histogram and satisfied conditions 1 and 2, then a second exposure at 4 sec and ISO 1600 violates #2 -- since you were able to get a good image at 1 min, that is an lower bound to the exposure time, and 4 sec is faster than optimum.
marcmccalmont
QUOTE (ejmartin @ Feb 28 2009, 07:56 PM) *
We didn't read each other's posts carefully enough. I didn't notice that you were changing the exposure in your test, and you didn't interpret my rule of thumb correctly. The rule of thumb was

1) Choose the minimum f-stop for the required depth of field.
2) Choose the minimum exposure time required to freeze the action, prevent camera shake, etc.
3) Choose the ISO such that the histogram is pushed to the right as much as possible while keeping the highlights that you want to preserve unclipped.

If you were exposing at 1 min and ISO 100, and got a good histogram and satisfied conditions 1 and 2, then a second exposure at 4 sec and ISO 1600 violates #2 -- since you were able to get a good image at 1 min, that is an lower bound to the exposure time, and 4 sec is faster than optimum.


It was a test, I'm agreeing with you!
Marc
Dale_Cotton2
Gabor, Guillermo, Emil (the Three Tensors?), et al.: for me the digital camera is an expressive artistic tool. To fully exploit it I need to understand how to use it; to understand how to use it I need to understand at least the rudiments of what's going on under the hood. For this reason I can't thank you three enough for your contributions to the on-line community. IMHO, the posts here from you three are one of the very few reasons this forum shouldn't be shut down by the absurdity police. ;) I also very much appreciate you're maintaining a pH-neutral tone ... which stands in marked contrast to the endless acidity of the prima donna set.
tlooknbill
Well I guess I came late to the discussion. After the email I got from Gabor on this topic, hadn't realized it was started over here at LL.

One thing that surprised me about Gabor's fascinating article was the fact that not only did noise crop up regardless of ISO, but different color patches exhibited varied amounts of noise according to their color whether low or high ISO.

Very informative article, Gabor. A bit hard to follow and keep organized in my head on what was affecting what because of the varying change ups between different color patches and ISO setting parameters and all the scrolling back and forth on my 20' wide format LCD I had to do, but I finally figured it out.

I experienced pretty much what you illustrated in a shot taken of a white goose in somewhat dark water in blazing bright mid-afternoon sunlight with my Pentax K100D. Since I stood the chance of loosing the shot shooting manual, since the goose was swimming away, which would've forced me to chimp the histogram adjusting exposure to prevent blowouts, I let the camera's metering choose with Av using Center Weighted Averaging and of course it chose to severely under expose the white goose where the water turned to black when viewed in ACR. It was shot at the camera's base ISO of 200, f/6.3, 1/1600s.

When I lightened the image in ACR so I could see the water, there was quite a bit of noise both chroma and luminous throughout the water going into the regions of the shadows of the white goose, but not enough that couldn't be fixed in ACR. There was plenty of light during the shoot and if I had shot manual I could've ETTR'ed maybe by as much as 1/3 to 2/3 stop increased exposure, but from my experience from shots like this with my Pentax, the dynamic range of that scene on my camera's sensor wouldn't have allowed ETTR to reduce the noise in the water or make it look lighter.

But I did get a nice shot of that goose.

Dale,

What do you mean by implying the Luminous Landscape forum shouldn't be shut down? Huh?! Are their talks of this being considered? Hope not.
tlooknbill
Gabor,

Something I considered as an afterthought derived from my goose shooting illustration above.

Have you tested for noise levels adjusting exposures using too much light where you have to underexpose to accomodate a dynamic range that happens to be too large for the camera's sensor?
Panopeeper
Thanks you all for the critique as well as for the appreciation. I will rework the entire paper. >I admit I have a big problem with assessing how the "average" reader understands the underlying issues. Most critique I received (in emails as well) suggested, that it is too steep, but I received the opposite as well.

I guess I will put the *conclusions* to the front, and then try to prove them; those, who are not interested on the technical details, do not need to go through them. Then I'll try to explain the technicalities in more detail.

QUOTE
Have you tested for noise levels adjusting exposures using too much light where you have to underexpose to accomodate a dynamic range that happens to be too large for the camera's sensor?

I don't understand the question. The noise does not depend on how and why you arrived at a certain exposure. Furthermore, it does not depend on the exposure per se, but on the illumination of the selected area (of course that does depend on the exposure). In other words. you may have overexposed the shot hopelessly and still have high-noise areas in it. As long as sensor blooming does not play a role, the overexposed and underexposed areas have nothing to do with each other.
marcmccalmont
I once read a math (or was it an engineering) book that had the main body on the right side pages and the detailed explanations (deep math) on opposing left side pages. I always thought that was a brilliant presentation when in printed book form. Perhaps there is a similar way for web presentation?
Marc
tlooknbill
Panopeeper,

Then why did I get noise in the water and into the shadows on the goose when there was a ton of light? It was broad daylight? You couldn't get any more light than what was in this scene.

It was just underexposed caused by the camera's metering. It ended up being a very dark image with a histogram as scene in ACR similar to a linear file. I've shot similar scenes shooting manual with proper exposure and didn't get noise in dark water.

Could this be caused by shooting a scene whose dynamic range is greater than the sensor's capability? IOW can dynamic range mismatch cause noise levels to increase.
Panopeeper
QUOTE (tlooknbill @ Mar 2 2009, 12:55 PM) *
Then why did I get noise in the water and into the shadows on the goose when there was a ton of light? It was broad daylight? You couldn't get any more light than what was in this scene.

Why don't you just upload your goose in raw format, so that we can see specifics, instead of speculating?
fike
In some ways, this demonstration is pointing out the obvious, but in a very sophisticated way that had never occurred to me.

It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions. If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range. The only difference is what is doing the gain increase--the camera or Photoshop. The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter. Higher ISOs provide a more agressive gain table, in a way. Eh??

Do I have that right?

Now that I think I have a more sophisticated understanding of this, I need to figure out how this changes my work in the field and at the computer. I am having trouble figuring out how this can help me in practice.

All I can think of is that it reinforces the well-known lesson that I need to find ways to get more light to the sensor, whether with longer exposures or wider apertures.

Can I draw conclusions about how to do better noise reduction? All this seems to do on that front is to reinforce the fact that noise reduction reduces and smears detail, because that detail wasn't all that certain to begin with.
Panopeeper
QUOTE (fike @ Mar 3 2009, 12:14 PM) *
It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions. If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range. The only difference is what is doing the gain increase--the camera or Photoshop. The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter. Higher ISOs provide a more agressive gain table, in a way. Eh??

Do I have that right?

Yes or no, depending on the actual camera and ISO.

Emil's advice above is very good, as generous consideration. I find it generally usefult to explain something from different angle of view for a better understanding. I use now the Canon 5DMkII for the explanation.

Down below are the noise graphs at the different *real* ISOs, i.e. ignoring those, which are only numerical derivatives, as they are never useful for the raw data. The horizontal lines represent a given level of noise; the intersection with the graph of an ISO value shows, how great the dynamic range can be with that level of noise. The difference in EV between two ISO settings at a given level of noise shows, how much you gain (or lose) when selecting this or that ISO.

For example if you say you are accepting up to 40% noise (see the examples for how 40% looks), then see that it intersects the ISO 3200 graph at -7.5 EV and the ISO 1600 graph at -8.5 EV. The difference between these is one full stop (like with a fake ISO). This means, that if you choose ISO 3200 over 1600 (and use one stop lower exposure), then you lose exactly one stop of the dynamic range. In other words, if you reduce the exposure but leave the ISO at 1600 (underexpose ISO 1600 by 1 EV), then the shadows will look equally good - or equally bad - but you don't run the risk of clipping one stop more highlights than with 1600 (because the increased ISO cuts off the top EV of the highlights, independently of the effect on the shadows).

In cleartext: don't use ISO 3200 and above with the 5D2, except if the embedded JPEG (the preview) is important, or if you are recording raw + JPEG.

Next step: ISO 1600 vs 800. The difference between those is roughly 0.7 EV, i.e. that is, what you lose when shooting with ISO 1600 vs. ISO 800 (where ISO 800 would be shot with one stop higher exposure). Turn it around: with ISO 1600 you expose by one stop less, but lose only 0.7 EV. The economics is not great, but if you can't efford the higher exposure and the potential of highlight loss is not a concern, then choose ISO 1600 over the underexposed ISO 800.

Now, ISO 800 vs 400: the difference is 0.5 EV. That's not a bad tradeoff for one stop lower exposure.

ISO 400 vs 200: roughly 0.25 EV. This means, that if the DR of the scenery is not very large, you can reduce the exposure practically without penalty.

ISO 200 vs 100: the difference is tiny. Use ISO 100 only if you really want to max out the dynamic range.

ISO 100 vs 50: the graphs conform, what we know already: there is no difference between these; ISO 50 is simply an overexposed ISO 100, there is no reason to use it; in fact, there is good reason not to use it: avoiding overexposure.

Now, all the above was for the case that you are not free to choose the exposure. If you can efford that, then use the lowest ISO (but not the overexposing one).

Note, that the differences between the ISO steps are the same with all the relevant noise levels with this camera and many others, i.e. it does not matter which noise level you accept, the considerations remain the same.

ejmartin
QUOTE (fike @ Mar 3 2009, 02:14 PM) *
In some ways, this demonstration is pointing out the obvious, but in a very sophisticated way that had never occurred to me.

It never really occurred to me to equate an underexposed image that is pushed (in Photoshop, for example) to make a 'good' exposure and a high-ISO image under the same conditions. If I understand this correctly an underexposed image that is pushed to be lighter in Photoshop will have similar noise attributes as a high-ISO image with the same resulting exposure range. The only difference is what is doing the gain increase--the camera or Photoshop. The limiting factor remains the number of photons that are read at the sensor, whether post-processing does the gain increase or the camera does it with an ISO, it really doesn't matter. Higher ISOs provide a more agressive gain table, in a way. Eh??

Do I have that right?


Yes gain can be done either in the camera hardware, in the camera firmware (as is the case with Hi ISO extensions in Canons, where ISO boost beyond the highest standard ISO is done digitally by camera firmware), or in the RAW converter by EV compensation. While it is true that it doesn't matter for the photons captured, they are always the same, it does matter for the noise. Every electronic component adds a bit of noise to the signal. There are two possibilities:

1) If that noise is added prior to amplification, then it gets amplified along with the signal, and so S/N stays the same;
2) If it is added after amplification then increasing the amplification increases the S/N since S is boosted but N is not.

When using analog gain in the camera via a choice of ISO, one gains an advantage because all the noises from electronics that process the signal after amplification (such as the amplifier itself and the A/D converter) come post-amplification and are of type 2. If one uses a lower ISO and amplifies in the RAW converter by applying EV compensation, then all noises are of type 1) for this amplification, and no S/N benefit accrues.

So a major lesson to take away is that any ISO that is a digital manipulation of a lower ISO by the camera firmware, provides no benefit in S/N while at the same time losing extra stops of highlights because image areas that are not blown out after the analog amplification can be made so by the additonal amplification applied in the firmware.

A cautionary note -- it seems that some converters (ACR seems to be one) the exposure compensation is not done all that accurately, and shadow detail is lost. See a thread over at DPR running at the moment:

http://forums.dpreview.com/forums/read.asp...essage=31188392

This is not a problem with the RAW file or the principle of pushing an underexposure for ultra-high ISO, it's an issue of the converter.


QUOTE
Now that I think I have a more sophisticated understanding of this, I need to figure out how this changes my work in the field and at the computer. I am having trouble figuring out how this can help me in practice.

All I can think of is that it reinforces the well-known lesson that I need to find ways to get more light to the sensor, whether with longer exposures or wider apertures.

Can I draw conclusions about how to do better noise reduction? All this seems to do on that front is to reinforce the fact that noise reduction reduces and smears detail, because that detail wasn't all that certain to begin with.


Yes, it's always best to get as many photons as possible.

Noise reduction is a separate discussion regarding the art of post-processing. One point though -- different RAW converters do better or worse with regard to noise. It is worth experimenting to see which ones give the best starting point for further processing.


tlooknbill
Panopeeper,

Here's the link to the goose shot. It is in its original Pentax PEF format. Let me know if you need it converted to DNG if you find you can't open it.

http://www.yousendit.com/download/U0d6a3ZFNXZBNkZjR0E9PQ

Just curious do you find what you understand about noise level behavior to be the same among other brands of camera's or are you only basing your studies and research on the Canon 5D Mark II?

Let me know what you find. And thank you for your time and insightful information.
Panopeeper
QUOTE (tlooknbill @ Mar 3 2009, 10:37 PM) *
Here's the link to the goose shot. It is in its original Pentax PEF format. Let me know if you need it converted to DNG if you find you can't open it

(No problem with the PEF)

1. The exposure was quite good; you could have exposed higher only 1/3 EV without clipping, see the raw histogram. With +2/3 EV a few spots would have clipped, see the second attachment (magenta indicates green clipping, red shows that the blue too would have clipped).

2. The third attachment is only for orientation, to show where I made the selection.

3. The fourth attachment shows the noise: the red channel was below the eightth stop. You see the vertical strips? That light ewas not enough for the camera. (In cleartext: the DR of the camera is about 8 stops without such noise).

Had you had made the shot with ISO 100, the noise would have been much less.

QUOTE
Just curious do you find what you understand about noise level behavior to be the same among other brands of camera's or are you only basing your studies and research on the Canon 5D Mark II?

The "effectivity" of the ISO steps is highly model dependent.

tlooknbill
Thanks for the analysis, Gabor.

So the dynamic range of the scene was somewhat beyond the capabilities of my camera. Wish I could've shot at ISO 100 but the Pentax K100D only goes to 200. So noise levels are different between models and brands of camera's based on shooting the same scene.

The Canon 5D Mark II would not show noise shooting a similar scene, correct?

Thanks for your time.
Panopeeper
QUOTE (tlooknbill @ Mar 4 2009, 12:02 PM) *
So the dynamic range of the scene was somewhat beyond the capabilities of my camera

That's right. I did not know the K100D does not go lower than ISO 100.

In such case you have no choice but to sacrify something: some bright white spots of the feather, or the darkest areas of the water. I think your shot is good so. It could have been increased by 1/3 EV without compromising the highlights, but that is practically impossible without bracketing. In critial cases I make several shots; I use a "neutral" setting, thus the histogram and the clipping indication show very reliably how exosure *was*, but it does not make the exposure correct, so I have to "circle on". Back home I look at the shots with Rawnalyze, if there was clipping then I see what and how much, look at the noise and select the best version.

QUOTE
The Canon 5D Mark II would not show noise shooting a similar scene, correct?

Here are two samples (smooth, uniform areas) with ISO 100 and 200, from the same intensity as the area I selected in your shot: around -8.6 EV. Although the 5D2 does show strong pattern noise (like all Canons do, on account of the rather "raw" raw data), that appears more than one stop later.

The next two captures show the noise with ISO 100 at -9.84 EV and -11 EV.
LEPING
Great. This is exactly what DxO found (0.7Ev DR decrease from ISO 800 to 1600, 1Ev from 1600 to 3200, etc.) I do understand that DxO Labs use 1:1 (0dB) SNR for the DR definition that differs from yours, but in the top right portion of your chart the curves are quite parallel to each other...

Click to view attachment
Someone have to go ahead and test a Nikon... Not the D3/D700 but the D3x, for which the DR keeps increasing by around 0.8Ev/ISO stop all the way to ISO 100, if the measurements are accurate. As someone mentioned Nikon and Canon use vastly different methods for readouts and as seen from many many examples (Lloyd, etc.) the D3x's read noise is way lower than the Canons and even their own D3. I have the raw data from DxO lab to share with the interested.

Both the 5DII and the D3's DR curve bend flat at ISOs below 800, while the D3x's keep raising almost linearly.

And, with little doubt, the D3x's red channel is a lot cleaner than the Canons.

The result: ETTR for the Canons, ETTC or even ETTL if you have important highlights on a D3x.

QUOTE (Panopeeper @ Mar 3 2009, 10:55 PM) *
Yes or no, depending on the actual camera and ISO.

Emil's advice above is very good, as generous consideration. I find it generally usefult to explain something from different angle of view for a better understanding. I use now the Canon 5DMkII for the explanation.

Down below are the noise graphs at the different *real* ISOs, i.e. ignoring those, which are only numerical derivatives, as they are never useful for the raw data. The horizontal lines represent a given level of noise; the intersection with the graph of an ISO value shows, how great the dynamic range can be with that level of noise. The difference in EV between two ISO settings at a given level of noise shows, how much you gain (or lose) when selecting this or that ISO.
......

Panopeeper
QUOTE (LEPING @ Mar 4 2009, 11:34 PM) *
Someone have to go ahead and test a Nikon... Not the D3/D700 but the D3x

Well, someone should make the test shots, like Erik Kaeffehr did with the A900 and Marc McCalmont with the 5D2.

QUOTE
I have the raw data from DxO lab to share with the interested

Upload a sample please, I take a look at if and how far I can use it. For example Imaging Resources files are not useful for *this* purpuse. Their color checker and gray step cards are downright dirty, and the exposure is too high. I can't use nice, well exposed shots, I need by three stops underexposed ones.

Btw, if you have tham, one from the A900 would be very interesting; not for measurement (Erik's shots are excellent), but perhaps I can show, why DxO misevaluated the DR and noise.

QUOTE
with little doubt, the D3x's red channel is a lot cleaner than the Canons

I don't have a monochromator (nor the camera) to make a full spectral response analysis, but I did compare the filter transmissivity on white, on the primaries and on a few other patches of the color checker card (on IR's images, with reliably constant illumination).

I don't see a *huge* difference (I see a larger difference in the blue). However, one needs to add the difference of the ISO sensitivity (the D3X "utilizes" the exposure better than the 5D2), then the difference is relevant.
LEPING
Hi Panopeeper,

Thanks for replying. I only have Excel spreadsheet from DxO Lab with the measurement numbers. Did you play and test a D3x first handed?

Please show us how DxO "misevaluated the DR and noise", despite that their DR curve and yours are almost exactly the same if one knows how to read these charts properly.

Thanks,
Leping

QUOTE (Panopeeper @ Mar 5 2009, 05:50 PM) *
Upload a sample please, I take a look at if and how far I can use it. For example Imaging Resources files are not useful for *this* purpuse. Their color checker and gray step cards are downright dirty, and the exposure is too high. I can't use nice, well exposed shots, I need by three stops underexposed ones.

Btw, if you have tham, one from the A900 would be very interesting; not for measurement (Erik's shots are excellent), but perhaps I can show, why DxO misevaluated the DR and noise.

Panopeeper
QUOTE (LEPING @ Mar 5 2009, 10:06 AM) *
I only have Excel spreadsheet from DxO Lab with the measurement numbers

I understood your I have the raw data from DxO lab that you have the raw data, i.e. the raw files.

QUOTE
Did you play and test a D3x first handed?

No, and I don't have good raw images for any evaluation.

QUOTE
Please show us how DxO "misevaluated the DR and noise", despite that their DR curve and yours are almost exactly the same if one knows how to read these charts properly

I referred to the A900 specifically. I will post about that within a few days. The documentation of my finding (i.e. proving it) is extremely labour intensive, and it will repulse most readers for being extremely pixel peeping, so I am pondering about what to include at all.
ErikKaffehr
Hi,

Would be most kind if someone would shoot the D3X raw images Panopeeper needs for evaluation.

- I used a syntetic Color Checker downloaded from ftp://www.rags-int-inc.com/fromrags/photo...CC_Calc_Lab.psd
- Printed this on semigloss A4 paper, important to avoid fingerprints and dust
- Illuminated by two cheap halogen lamps about 45 degree angle on each side, trying to avoid reflections
- Underexposed 4 steps

I sent the files to Panopeeper using yousendit com. Please note, it has taken some iterations, Mr. Panopeeper is not easy to satisfy ;-)

Best regards
Erik


QUOTE (Panopeeper @ Mar 5 2009, 08:53 PM) *
I understood your I have the raw data from DxO lab that you have the raw data, i.e. the raw files.


No, and I don't have good raw images for any evaluation.


I referred to the A900 specifically. I will post about that within a few days. The documentation of my finding (i.e. proving it) is extremely labour intensive, and it will repulse most readers for being extremely pixel peeping, so I am pondering about what to include at all.

This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2010 Invision Power Services, Inc.