A new site by DxO Labs has appeared: DxO Mark, comparing sensors and cameras with a rigurous approach (they claim to focus on pure RAW undemosaiced data).

Some of the statements such as: "Contrary to conventional wisdom, higher resolution actually compensates for noise" (copy & paste: www.dxomark.com/index.php/eng/Insights/More-pixels-offsets-noise!) make me thing it can become a quite interesting place for reference, contrarily to common JPEG sites such as DPreview.

Just the overall DR tests, not per-pixel DR (the Y-scale seems rounded but plots seem correct). All new FF's have similar DR but into that group the A900, with the highest pixel count, won:



BR

I found this introduction to that site on the web.

oops! I did a search about dxomark.com on the forum but forgot to check the main page. Anyway this thread can be used for discusion.

Good performance by the A900, although this is probably within measuring errors of the other 3.

By the way, is that pixel level DR or image level DR? I would think that pixel level DR is more important since image level DR will make the larger image look better at a given print size thanks to the smoothing effect of downsizing on shadow noise, right?

Cheers,
Bernard

I noticed that this site uses the cameras lowest ISO, which they list as ISO 100 on the A900. ISO 200 is the actual base ISO of the A900, and most tests I've seen give ISO 200 nearly a +1EV stop of DR compared to ISO 100, so the A900 may actually have even better DR than this new test shows. What do you think?

Never mind. They tested a range of ISOs. Strange that ISO 100 is higher than 200

I think DR measures are at image level, they speak about that here: www.dxomark.com/index.php/eng/Insights/More-pixels-offsets-noise!
(just copy & paste the address including the ! in the end since it seems the LL forum engine does not allow the ! symbol for links).
Otherwhise the 1Ds3 and A900 would surely have worse per-pixel SNR and DR than the Nikons.

I thought this fact had been conventional wisdom for quite some time and that only a few diehards like Gabor (Panopeeper) objected to trading resolution for lower noise.

This new site seems an excellent idea to me. I'm all in favour of more rigorous and more meaningful testing which relates more directly to the final product, which is usually a print, or image on screen, of a particular size. You can't have a sizeless image or print.

Iliah Borg over at dpreview doesn't seem to think a lot of the results from dxomark so far...

http://forums.dpreview.com/forums/read.asp...essage=30054057

As Emil says, it's a brand new site in beta stage and we still don't know deeply what methods they are using.

Also for example if you plot DR of the 40D vs 50D (the so much criticised review in DPreview), for both cameras DR is slightly higher for ISO200 than for ISO100. I can only think of that happening if saturation points at those ISO values vary so much to compensate SNR loss at a given RAW exposure when going from ISO100 to ISO200. Looking at the SNR plots I rather would think they didn't make a very accurate calculation in the low end (deep shadows). Perhaps they applied the same data processing to a big group of samples (they have reviewed many cameras) and the measures for some of them were not enough accurate at that critical point. Or simply they did not have data reaching the low end and some approximation was done.

See here the Canon 50D strange behaviour in the low end:




And for the case of the Nikon D90 there also seems to be an unexpected behaviour in the 4th curve (ISO 1600) that could lead to a such suspicious DR curve plot:




So surely there are things to be fine tuned in the results and methods, after all it's a massive beta version of the site, but IMO the beginning seems promising compared to other comparisions using procedures sometimes out of any logic (DPreview).

Let's see if Emil and Iliah hang around here.

I'd like to see popular MFDB's included in the test results.
Marc

Canon DR usually goes up from ISO 100 to 200 in current models for a combination of two reasons: (1) read noise goes down by almost a factor of two; and (2) the sensor saturates well before the max raw level 16383 at ISO 100. This means that one loses less than a stop at the highlight end in going from ISO 100 to 200, while getting nearly a stop at the shadow end.

It does seem that there are some glitches in the measurements at low signal in their SNR plots. I would think it better to use the measurements to fit the data to the universal noise vs signal formula, taking as many points are required to get a good chi-squared. Then plot the universal formula with the fitted parameters as the curve, with the data points to show the goodness of fit. And use the fit rather than the measured data to determine the DR etc, so that individual data points (especially in shadows) don't skew the results. My guess though is that they're using the measured points directly, or with simple spline-type fits rather than fits to the universal noise formula, and so if they don't have enough data they'll have more substantial errors in the results.

Since the D3 and D700 use the same sensor but get different DR evaluations, I see an obvious problem with this site's credibility.

I also wonder why anyone would care if their camera is better or worse at some extraordinarily negligible level of detail. When cameras had clear image quality advantages, then comparisons were useful, but at this point it is not worth the efforts some are putting into trying to determine which camera is "better".

Emil,
This seems to be very useful information to have. I think that most photographers would tend to assume that best results with respect to tonality, DR and noise will be achieved at base ISO. As a consequence, one might choose a merely adequate shutter speed at ISO 100 for the sake of better DR, in preference to a shutter speed twice as fast at ISO 200. The faster shutter speed is likely to produce at least marginally sharper results. However, if the lower ISO of 100 has no benefit with regard to DR, one might wonder what purpose is served by using it, unless one needs a slower shutter speed for a particular effect.

This DR test over the new FF cameras demonstrates that regarding DR, the differences among them are negligible. And that's a _very_ useful information to focus on other characteristics to make a choice. If you plot the result of the Canon 350D, or the Fuji S5 Pro vs these FF cameras, you will see important differences in DR.

I don't see a problem in the D3 and D700 comparision, their DR results are so close that could well be due to differences in the units under test (just look at the Y-scale). The point is not only which camera got a higher or lower DR in the test, but how much higher or how much lower. Again the conclusion is 'they have the same DR' and that allows to pay attention to other parameters (e.g. price).

Emil, please correct me if I am wrong: what you mean with this is that:

1. These Canon cameras _almost_ double their S/read noise ratio when going from ISO100 to ISO200, and that is equivalent to having very close curve plots in the low end (never crossing curves however), and that is equivalent to admitting that almost all read noise is produced after the ISO amplification.
2. On the other side, the difference in saturation point (lower at ISO100 than at ISO200) means more extra DR in the right end than what we loose in the left end when going from ISO100 to ISO200.

Being * the saturation points for ISO100 and ISO200 this is what we are talking about:



Questions:
1. If the saturation points were the same, we would _never_ get more DR at a higher ISO no matter how much improvement in S/read noise we could get from ISO100 to ISO200. The more we could expect in that case would be to get the same DR. Right?
2. If it is true that for a given camera a higher ISO provides more DR, unless we need extra long exposure times, it would be completely stupid to use the lower ISO for any application (this is what Ray asked).
3. Why is that big difference in the Canon's saturation points? at ISO100 my cheap old 350D saturates at 4095, while the 5D saturates at 3692. I never understood this. It seems as if the output of the ISO amplification gets clipped before entering the ADC, and even that clipping point depends on the ISO set.

BR

Agreed. In fact, I'd say that, other than making the choice between the lower noise of the D3 vs. the resolution of the 1Dsiii/A900, there really isn't much between the FF cameras. It seems that price, handling, camera options and system options are the more relevant reasons to make a purchasing decision.

And there you have it in a nutshell.

Yes the curves shouldn't cross; the examples you showed where they do represent flaws in the data.



Correct.



Because, though you lose a bit of DR at ISO 100 on Canons (at the cost of a stop in Tv/Av as well), you gain in S/N throughout the range. Any particular feature in the image will be less noisy.



It's to do with the native ISO of the sensor relative to the gain applied by the variable gain amplifier. Canon has chosen their lowest gain setting such that the sensor clips before raw saturation in the newer sensors. Since the newer ones are more efficient, if the same amplification is used and the well capacity remains about the same, the photosite will saturate more rapidly (one might say that the "native" ISO of the sensor is higher on the newer camera sensors). Of course these things are never kept fixed from models to model (due in particular to changing pixel pitch), but it is true that the native sensor ISO has increased and so one has the possibility that the sensor saturates earlier for a given ISO setting.

I see, but then if a given sensor saturates at ISO100 at (let's say) ADU=13000, which is less than 1 f-stop far from RAW saturation (ADU=16383), any other ISO should clip at RAW saturation. Does this happen?
For instance the 40D which saturates at 13823 (from my tests) at ISO100, should saturate at 16383 for any other higher ISO value. Does it?

Close enough. John Sheehy did some analysis of the 30D a while back and the clipping levels were not exactly 4095, but close enough that it doesn't affect things on the log scale of EV. The difference between 13823 and 16383 however is close to 1/4 stop. I did some 40D tests which gave clipping at 16383 for the green channel at ISO 200 and up, IIRC.

1. The DR of both the Canon 40D and 50D is larger at ISO 100 than at ISO 200.

2. The saturation level plays no role in the DR; the ISO 100 values are scaled down, probably from the base ISO, which appears to be 1/4 EV over 100.

Looks like I'll have to carry out some testing, when I have the time, comparing shadow and highlight detail at ISOs 100 and 200. I know that cameras do vary from copy to copy (of the same model), just as lenses do. Years ago, after collecting my new 5D, I wasn't happy with the severe banding in the darkest shadows. I returned it and tried another copy which was better. I wouldn't say much better, but noticeably better.

Forgive me for continuing to piss all over this thread, but I find these discussions not to be useful to me and question their purpose.



It's more fundamental than whether or not DxO is "accurately" measuring the characteristics of these sensors. What is the point of dissecting RAW data absent any conversion of that data? If fixating on differences in sensors makes any difference at all to the outcome of your photographs, then equal attention should be paid to RAW converters: http://forums.dpreview.com/forums/read.asp...essage=30062493 You can't simply divorce the data coming from the sensor from the data that comes from conversion of that file; after all, the later is what you will ultimately be working with when you craft your photograph. This leads me to question what some people in these photography forums are attempting to craft: better photographs, or better ways of measuring RAW data. For me, the tools I care about are the ones that deliver files I can use to make better photographs with, and DxO Mark is a diversion from that goal.

Tony has a point. We all have to use a RAW converter of one type or another and that choice of RAW converter might be based on lots of factors that are not directly related to getting the last ounce of detail from highlights or shadows. We might use a converter because we like the results, the work-flow, and like the ease of getting those results, or perhaps because we are simply familiar with the interface.

We should not forget that DXO are in the business of creating their own RAW converter. Their tests, which they are now sharing with everyone, are presumably necessary for their own purposes.

Because it provides a measure of the camera's capabilities rather than the sum total of the camera's capabilities filtered through a particular raw converter's capability. Usually the scientific method is to strip away those aspects of the subject under study that don't pertain to the subject under study. If you got a lousy result from camera A with raw converter X, you can't know if the issue is camera A or raw converter X. If you study the raw data of camera A without going through conversion, and you find it to be poorer quality than that of camera B, then no raw converter is going to make camera A's output look better than camera B's provided it is doing an equally competent job on each. The latter issue should be the subject of a separate investigation, which to my knowledge is yet to be done in a scientific manner for commercial raw converters.

Better to isolate the variable under study than to lump it in with all sorts of other unknowns.



I agree that equal attention should be paid to raw converters, and that hasn't been done yet. That doesn't mean it can't be done, and it doesn't mean that raw data analysis is without meaning or worth.

Well, obviously the purpose of pleasing you is failed miserably. I guess you need to ask Michael to close the thread.



It is called abstraction. The information gained this way reflects the characteristics of the camera, independently of the raw converter.

If this is not done, then one could believe, that the camera can do only as good as it appears from certain specific conversions. For example DPReview conducted tests of the 50D with the preliminary version of ACR, which contained some error (IMO it still does). The 40D raw data interpretation was wrong from the outset. I reported the error, which has been turned into another one. Measuring the DR based on that interpretation of the raw data leads to an incorrect result (but DPReview can not measure the DR anyway).

Do you find it useful to evaluate a camera based exclusively on a specific interpretation of the raw data?

No need to be flippant; I'm just expressing my point of view. No, I would never presume to ask Michael to close any thread in his forums.



I'm certainly not advocating that; in fact I'm thinking you and I agree more than you think we do as far as that is concerned. However, a camera's RAW files are only as good as what they can be converted to. While that may require a better RAW conversion algorithm; if none exists then that becomes very problematic to a photographer who doesn't have the knowledge or means to create an ideal RAW converter -- then you move on to another camera -- but I don't think this a really a problem with the recent DSLRs and associated RAW converters we currently have available to us.



Yes, specifically with the "interpretation" from the converter I would be using -- which for me would be the one that delivered the best results. Some people complain that superior results from a particular converter are attained because of NR being applied (often they are wrong), but I don't care as what matters to me is that the level of noise and the amount of detail are optimized, and what I find using NX with my D300 is that I have both less noise and more detail than what I get with other converters. Should I dismiss the results I know I can get because some website uses a different (putatively neutral) converter and then claims that some other camera does a better job than mine using that other converter? Should I care if someone claims another camera gets better DR than mine because some "abstract" analysis says so, particularly when I can get what I consider extremely good results with the camera I have?

The strength of the DxO Mark method and similar methods is that it allows for accurate characterization of a sensor and/or optical system.

The weakness of the method is that final image quality (e.g., what is seen in a print or on a display, such as in a web gallery) depends on much more than raw-level characterization.

For example, a raw-level characterization may show more promise for the Panasonic DMC-LX3's sensor compared to a Canon PowerShot G10's sensor, but raw files rendered using these makers' respective software (SilkyPix and Digital Photo Professional) may yield opposite results. This may seem initially like a bogus comparison because different software products are being used, but on the other hand it's a common scenario that will apply to many users (i.e., owners of these two cameras will simply use the software that comes with the camera).

This is the classic tradeoff of the scientific method. By isolating specific variables, you can get a very good understanding of that variable within a limited context. The downside is that it's insufficient to explain the big picture or the broader context, without a (much more difficult and time-consuming) examination of the whole context. For example, we have detailed models of the human eye, which can explain certain visual phenomena but is insufficient to model color appearance. As another example, there has been much study of the individual contributions of various nutrients (vitamin B, antioxidants, etc.) to health, which unfortunately are poor predictors in general because they do not take into account the much larger complex system (i.e., how all of these elements interact).

Don't get me wrong. I applaud DxO for their efforts in launching this site and for making the (summarized) data available. It is certainly the right way to go when developing raw conversion software. But if you don't develop raw conversion software, and you're just a good ol' photographer, you have to ask yourself if the numbers/data are going to be relevant to your workflow. Or, put another way, if the doctor tells you to eat fewer eggs because lower cholesterol means a smaller chance of a heart attack, you should ask if that is really the factor that matters.

As a Sony user, I can say that the two worst RAW converters I've seen for the A900 files are ACR and....Sony's own IDC! Since nearly every review site uses one of those two converters for RAW development with the Sony, I think it's good to show what the sensor is capable of. Aperture and Capture One are two examples of converters that don't mush the Sony RAWs, and I'm hoping the new Bibble acts similarly.

The point of an analysis without the raw converter would then be to determine the potential for improvement in raw conversion.

This kind of data may not be useful to regular photographers per se, but it could be useful to those making raw converters or otherwise working directly on raw data. It could also be useful to camera sensor makers.

I agree that results based solely on measures over RAW data providing an objective view of the hardware of the camera, need some extra interpretation effort to extrapolate results to what we can expect to achieve in our real photographs. However I prefer by far these results than those more obvious DPreview samples. After all we are not always going to use our camera to take JPEG pictures of a stamp with the queen of England.

BR

Emil, I was thinking that perhaps they already corrected the saturation point of the camera at each ISO to fit 100% in the X-axis of their plots, contrarily to your plots that strictly display ADU levels on that axis where the right end is always RAW saturation (ADU=2^14-1).
In that case their curves could cross, and in fact this would show in a very intuitive way in the plots what you explained on how some cameras can reach a higher DR for a given higher ISO, but at the cost of worse SNR in most of the range.

I think it could be a good idea that you show the sat point for each ISO (typically just ISO100 should differ from 2^14-1) in your graphs, specially for proper DR measures on them.

BR

I agree, but I don't know the saturation point for some of the cameras.

Saturation and clipping are more complex than one might initially think. To test the saturation of my Nikon D3, I grossly overexposed an image of a lightbox at various ISOs with the camera set to record losslessly compressed NEFs. The indicated exposure for ISO 200 was 1/1000 s at f/2.8 and I used 2 s at f/2.8, so I am reasonably certain that all pixels are saturated. I then used Iris to separate the Green 1, Green 2, Red and Blue channels and examined the histograms of the ISO 200 and ISO 3200 images in ImageJ:

Click to view attachment

Clipping can occur when the sensor is saturated or when the ADC overflows. The maximum value for a 14 bit data number is 16383. The green channels at ISO 200 never reach 16383. Due to PRNU (Pixel Response Nonuniformity), the pixels saturate at various levels as shown in the histograms. The camera gain is such that the ADC does not overflow. For the red channel at ISO 200, the gain is such that at pixel saturation, the ADC overflows for all pixels as shown in the histogram. For the blue channel, the gain is such that some pixels cause overflow of the ADC but others do not and PRNU is again observed.

I thought that these results were interesting and worth posting.

Bill

The strangest saturation point I have found is that of the R sensor in the Fuji Super CCD. Looking at the histograms obtained with increasing exposure one could think it never saturates since blown information never concentrates at any particular point but along a wide range of values in the form of a gaussian distribution. See here the EV histogram (each of them normalised to its maximum) with subsequently increasing exposure of +1EV:




If we look at the last histogram in linear mode we see all the saturated areas concentrated in a gaussian bell distribution in the right half:




One could think that's information compression (which would mean a huge DR!), but unluckily it is not, it seems to be just noise. Yes, noise in the highlights but noise after all with no texture or detail on it. And the three RGB channels of that bell are aligned in the RAW data (i.e. prior to WB which is the way I calculated these histograms), that is why once WB is applied, R and B relative exposures increase with respect to G and this causes the typical Fuji magenta highlights for having G shortage that becomes visible when applying a strong exposure correction down (what others call highlight recovery). So when a magenta cast appears on a Fuji image it just means one thing: saturation of the R sensor (of course in this situation the S sensor saturated time ago).

These images were obtained from the same RAF file, first with ACR and strong exposure correction down by -4EV, second using a not so strong exposure correction to try to avoid the magenta cast, and last with Zero Noise (DCRAW) using as saturation point the beginning of the gaussian bell, i.e. deliberately clipping to pure white all the 'information' that could be packed into that gaussian distribution:



The Zero Noise result displays the same information as ACR with the -4EV correction down but without the channel misalignment thanks to the specific saturation point set in DCRAW. Also ACR and DCRAW show differences in the WB applied to the R sensor, when both apply the same WB to the S sensor (i.e. the shadows of any RAF).

This particular behaviour of the R sensor in the Super CCD can easily lead to being optimistic (~0.5 f-stops) in the calculation of the DR for the Fuji cameras if this effect is not properly taken into account.

Guillermo,

These data are interesting, and I am surprised that no one has commented. I'm not that familiar with the Fuji super CCD and the R sensor component, but I would imagine that it would behave similarly to other CCDs with only one type of pixel.

In performing a sensor analysis of my D3 (CMOS sensor) using paired images at various exposures using the method of Roger Clark, one can make some interesting observations. I used Iris to separate the green 1 channel and cropped the image to 201 x 201 pixels. I then subtracted one pair from the other and added an offset of 200 to avoid negative numbers and determined the standard deviations. The standard deviations of the single frames represents total noise, while the difference standard deviation removes variations due to non-uniformity of the light field (e.g. due to vignetting in the lens), dust specs on the sensor, pixel to pixel sensitivity variations and other effects.

This table shows the results of some of the data. Sigma3 is the standard deviation of the subtracted frames and it is repeated for the second pair.

Click to view attachment

When the sensor saturates, there is an abrupt decrease in noise since all pixels are driven to their maximum value. The standard deviations of the individual frames represents mainly pixel nonuniformity and this is removed when one looks at the difference image:

Click to view attachment

The individual image histogram has a double Gaussian appearance since the D3 uses multiple readout channels. I think that the histogram of the subtracted frames represents quantization error.

It would be interesting if you would perform a similar analysis on your data and let us know the results. From the appearance of your histograms, I assume that you are using demosaiced data from DCRaw. It might be better to use data directly from a raw channel if this is possible.

Bill

There is something seriously wrong with the conclusions this website offers. I was reading a post at another photography forum that referred to the D300 ISO performance on this page: http://www.dxomark.com/index.php/eng/DxOMark-Sensor and when I went there I noticed that the D90 ISO performance (which uses the same sensor) was rated about half a stop better. I believe people relying on this sort of information in making a DSLR purchase are going to be seriously mislead as there is simply no way the D90 is a half stop better than the D300, and this makes me highly suspicious of all of the charts offered by this website. To put it bluntly, this site as it currently stands is simply bogus.

Tony,
How do you know there is no way? What inside information do you have to support this conclusion. What test comparisons have you done?

A reputable reviewer has done the tests: http://www.bythom.com/nikond90review.htm

I know who I trust.

Hi,

There are amplifiers in the signal path. The Sony chip has on chip ADCs but Nikon normally bypasses these and uses a 14 bit external ADC instead. Even the chip itself could be better as the D90 is a later model than.

I don't think that DXO is a bogus site just because you or I don't understand their methodology or don't agree with their results. On the contrary, I really appreciate that their share their results.

Best regards
Erik

The comparison is to the D90, another Nikon camera using the same sensor and it only records 12 bits whereas the D300 can record 14 bits; I believe both use a 16 bit pipeline, I know the D300 does. To quote the relevant portion (for the purposes of this discussion) of Thom Hogan's review of the D90:

"...for the main types of noise and high ISO values, the D90 is essentially a mimic of the D300....Go read what I wrote in my D300 review about noise handling; the D90 is nearly identical."

That sure doesn't look like there's a half stop difference to me -- perhaps Thom Hogan just doesn't see the differences that DXOMark is asserting are there.

These geeky, technical analysis' of cameras are already flawed even if they were accurate: http://theonlinephotographer.typepad.com/t...l-vindicat.html. However, the data presented by DXOMark is flawed and that makes it particularly egregious to offer it to for the purpose of "informing" some unwitting person who wants to buy the "better" DSLR.

A note without any further qualification/comparison: the sensors of the D90 and D300 are not identical. Although the overall dimension in pixels is the same, the masked areas are different as are the spectral responses.

This does not preclude, that the the sensors' electronics are the same. On the other hand, the different noise characteristics speak against the "same sensor" even regarding the electronics.

Okay, you prefer to believe some numbers presented by DXOMark over what Thom Hogan observes. It appears to me that in the real world (the world based on knowledgeable photographer's experiences with the cameras) the D90 and D300 have very nearly identical image quality; only in fantasy land does the D90 have noticeably superior image quality.

For instance, in the fantasy land that is the DPR Forums
http://forums.dpreview.com/forums/read.asp...essage=30005750

I prefer to believe my own measurements over what Thom Hogan observes. I came to said result and posted it much before this DXOMark comparison appeared.

It has to be noted, that my measurements relate to the noise at a given ISO, but that is only half of the subject; the other half is, how correct the nominal ISO settings are. When shooting in low-light situation, it does not solve the problem that one camera has much less noise @ ISO 1600 than the other, if the first camera requires one stop higher ISO in the same setting. (Of course, one full stop difference would be quite extreme). I can not measure this; I am using raw files made by others, mostly by Imaging Resources. I guess DXOMark publishes such a comparison as well.

On the other hand, from the point of maximum dynamic range, only the noise counts.

Anyway, the possible incorrectness of the nominal ISO values does not mitigate the fact, that the sensors of D90 and D300 are not the same; in fact they are very different.

Samples of my measurements are for example in D90 vs D300, a layered TIFF.

I see that you both are referring to comparison files from Imaging Resources. First thing I notice about these files is that they are exposed differently; for instance, at ISO 800 the D300 is exposed at 1/125 while the D90 is exposed at 1/100, that's a 1/3 of stop difference. I suspect that I do not shoot the same way Imaging Resources shoots, I utilize uni-WB to try to achieve a maximum EV and then adjust EV with the RAW converter. Nonetheless, despite the 1/3 EV difference, looking at the ISO 800 images I see practically nothing to distinguish them from each other -- so digging around in shadows to isolate differences is an exercise in pixel peeping that does nothing to elucidate which camera would be a better choice for an individual photographer -- as I stated earlier in this thread, all of these cameras will deliver more than adequate files for the vast majority of photographers under the vast majority of applications.

I wish all of you enjoyment in your pursuit of whatever it is you are searching for (my suspicion is that real photography is not the goal of these pointless exercises). I just hope that people looking for a "better" camera choice are not swayed by these flawed analysis'.

I'm happy for you that you don't need to push the performance of your cameras to the limit. BTW, uni-WB does nothing to the RAW file. It may help you to set the exposure if you're not in the habit of metering the scene before you shoot, by checking the histogram after you shoot.

There is a difference, it's quite obvious in the IR images, and much more than the 1/3 stop difference in exposure. Agreed though that not all people need the difference, any of the cameras in the thread I linked is more than capable of making great images at low to moderate ISO.

Nonsense. I do push my cameras to their performance limits, and that is more than what I typically need. How do you find the current cameras lacking?



It definitely helps, it is essential. What's more, other settings are involved too, including most importantly brightness. I find that the true native ISO of the D300 is closer to ISO 125 compared to what the D200 does.



Laughable. I have an IR converted D200 and wouldn't waste my time doing IR shots with a CMOS sensor camera.

Perhaps you meant Imaging Resource, well I find their shooting conditions to be uncontrolled as demonstrated by using different EVs, different focal lengths, and who knows what else is different. With so many differences, it's difficult to determine what the sensor's actual differences are.



That's my point, for the vast majority of users all of these cameras are more then up to the job.

This is totally irrelevant regarding the noise, as is the lens, the distance, the subject, the illumination. ("Calibrating" the ISO is a different issue.)



I'm afraid you are not in the position to judge this; you did not ask the "vast majority" of photographers, nor did I. You are suggesting, that noise and dynamic range are non-issues with these cameras; I don't doubt that this is true regarding your photography, but looking around on several forums I am convinced, that this can not be said generally.

People are shooting in low light like in concerts or with very short shutter time without professional lighting like indoor sports in a high school gym, thus they are forced to go with high ISO. Some of the shots are still strongly underexposed; some areas can be in the seventh, eighths or ninths EV (from saturation), and in that region the noise is unavoidable with today's DSLRs (at ISO 1600 and higher), the question is only how bad. These nonsensical looking tests are analysing just that: how is the noise in the very deep shadows.

The other side of the same coin is the dynamic range. I am very often in situations, that highlights are already clipping but other areas of the image are far underexposed @ ISO 100 or 200. How much I can reclaim from the shadows while showing the details is limited by the noise. I can determine with my measurements the difference between two sensors in stops at a given noise level.