Detail preserved after downsampling?

[LKaven]

One thing that has pleased me about high-resolution sensors is the way in which high-frequency detail appears to be preserved, even after downsampling to prints size, or even to web size.  The difference between my D3x and my D3s was striking even at small sizes.  I can see it in skin, hair, leaves, textures, etc.  I have no experience with the larger sensors, but have noticed the extra detail present in many images posted here.

But what are the factors that explain that?  I had a hypothesis -- that the cutoff of the low pass filtering was different in each case.  I ventured that this was not just a function of the OLP filters in each respective camera (or the lack thereof in some cases), but also a function of the total regimen of oversampling followed by downsampling.  Somewhere up near the Nyquist limit, the D3s starts to fragment.  In comparison at the same output size (12MP or less in this case), the D3x would produce good high-frequency detail.  This was illustrated in a quick experiment that Lloyd Chambers did with these cameras, photographing the label on a soup can and noting the smoothness of the text in the D3x capture when downsampled to 12MP, in comparison to the fragmented text D3s capture (at it's native resolution).

In an exchange with Bob Newman over at DPR, Bob suggested that I need look no further than the difference between the OLP filters on these two cameras.  The D3x does of course have a more gentle filter considered as a function of sampling frequency.  But I wonder if Bob's claim is true?  I asked Bob if a D3s with no OLP filter would produce output as detailed -- or more so -- than a D3x image downsampled to the same resolution.  Bob believes the answer is yes. 

Is that correct?  I had surmised that there were more factors involved with this.  For example, I had always thought of a downsampling filter as being in essence a LPF, with a cutoff that might vary somewhat in slope, depending upon the method.  I feel that surely I am getting additional benefits from supersampling and other processing that manifests itself in the retention of detail after downsampling.  But I don't have either the empirical or analytical tools at hand to resolve this question. 

Among those of you here who have investigated in this area, do you have either an empirical or analytical answer to this, or just a good hypothesis? 

[LKaven]

Hmm...  Too wordy?  Wrong forum?

[eronald]

Hmm...  Too wordy?  Wrong forum?

I think anyone capable of answering that question is on vacation

Actually, I don't know if anyone has done the necessary tests to determine the transfer characteristics of the D3x and D3s. I know that some people have been using 5D2 cameras and even Nex7 without  filter, and of course many of us have MF without such.

Edmund

[BartvanderWolf]

Hmm...  Too wordy?  Wrong forum?

Hi Luke,

It's hard to comment on something you have noticed but I have not, because I use different cameras. Theoretically, at the amount of downsampling ("even to web size") you mention, the only factors can be the MTF curves of both your capture systems combined with the downsampling method used. An optical low-pass filter will barely have any influence on the spatial frequencies that remain after downsampling.

Proper downsampling will eliminate as much as possible all spatial frequencies that cannot be reliably represented in the downsampled image, and do so before downsampling. The filter used for that can make a difference on how exactly the transition from wanted to unwanted detail is shaped, a Lanczos windowed Sinc filter is generally considered being close to a best compromise.

A free commandline utility like ImageMagick allows to do that (the default filter for downsampling is Lanczos), and the result can even be improved by adding a Gamma linearization before the downsampling, in the same Convert command.

I'd be a bit surprised if there were significant differences visible between two almost identical images taken with the same lens and two different bodies. I do agree that it's hard to beat the look of a well downsampled image, almost without any of the possible aliasing artifacts.

Cheers,
Bart

[LKaven]

Hi Luke,

It's hard to comment on something you have noticed but I have not, because I use different cameras. Theoretically, at the amount of downsampling ("even to web size") you mention, the only factors can be the MTF curves of both your capture systems combined with the downsampling method used. An optical low-pass filter will barely have any influence on the spatial frequencies that remain after downsampling.

I think one can compare cameras more broadly.  I picked the D3s/D3x not just because I have them, but because they are the same sized sensors.  I'm not sure that matters though.

It did also seem to me that the OLPF would not explain the difference entirely, and that there would have to be a difference in the total MTF of the two systems.  But I'm not sure where that difference would be.  I'd like to see those two respective curves somehow. 

I did look up Lloyd Chambers' experiment.  Cereal box, not soup can.  It's about 2/3 of the way down the page:
http://diglloyd.com/diglloyd/2009-01-blog.html

Proper downsampling will eliminate as much as possible all spatial frequencies that cannot be reliably represented in the downsampled image, and do so before downsampling. The filter used for that can make a difference on how exactly the transition from wanted to unwanted detail is shaped, a Lanczos windowed Sinc filter is generally considered being close to a best compromise.

A free commandline utility like ImageMagick allows to do that (the default filter for downsampling is Lanczos), and the result can even be improved by adding a Gamma linearization before the downsampling, in the same Convert command.

Thanks for the recommendation.  I have had a hard time finding a convenient Laczos filter that I can incorporate into my workflow.  I was hoping ImageJ had it, but it doesn't seem to.  Irfanview does it, but only at 8 bits.  Too bad there isn't a photoshop plugin that can invoke ImageMagick calls. 

I'd be a bit surprised if there were significant differences visible between two almost identical images taken with the same lens and two different bodies. I do agree that it's hard to beat the look of a well downsampled image, almost without any of the possible aliasing artifacts.

I wonder if I shouldn't interpret that antialiasing as a kind of high-frequency detail retention as well.  A smooth curve in comparison to the jagged curve has additional high-frequency components, and it seems that's mostly it, or what?  I'm not sure I know why "antialiasing" as a concept does any more work than that, except to speak specifically about perception in a narrow, practical sense.  [And the same I suppose might go for color gradations as distributed across the image surface, which have frequency components of their own.]

To recap:
The image from a high-resolution sensor with H megapixels, after being downsampled to L megapixels (L < H), appears to retain more high-frequency information than an image captured with a native L megapixel sensor when both are examined at L megapixel resolution.

Hypothesis: This would be explained by a difference in the total MTF between the two systems, especially approaching the high-frequency cutoff.  But what are the differences?

[hjulenissen]

Theoretically, at the amount of downsampling ("even to web size") you mention, the only factors can be the MTF curves of both your capture systems combined with the downsampling method used. An optical low-pass filter will barely have any influence on the spatial frequencies that remain after downsampling.

I do believe that the lack of OLPF can cause aliasing into low frequencies (even "DC") that may be visible not matter what the downsampling factor is?

-h

[BartvanderWolf]

Thanks for the recommendation.  I have had a hard time finding a convenient Laczos filter that I can incorporate into my workflow.  I was hoping ImageJ had it, but it doesn't seem to.  Irfanview does it, but only at 8 bits.  Too bad there isn't a photoshop plugin that can invoke ImageMagick calls.

On the other hand, in Windows you can make a batch file in the "SendTo" folder, which will allow to Right Mouse Button click an image file and produce a smaller version (e.g. 800x800 pixels maximum for Web use). Or one can place a batchfile on the desktop and drag-n-drop image files on it, or use a batchfile to process an entire subdirectory with image files, and save the reduced size files to another subdirectory. 

To recap:
The image from a high-resolution sensor with H megapixels, after being downsampled to L megapixels (L < H), appears to retain more high-frequency information than an image captured with a native L megapixel sensor when both are examined at L megapixel resolution.

Hypothesis: This would be explained by a difference in the total MTF between the two systems, especially approaching the high-frequency cutoff.

That's most likely what you're experiencing, a higher MTF response near the Nyquist frequency.

Cheers,
Bart

[BartvanderWolf]

I do believe that the lack of OLPF can cause aliasing into low frequencies (even "DC") that may be visible not matter what the downsampling factor is?

Hi -h,

That's correct, but it's unlikely to be information that will help the downsampled image achieve that sharper look the OP is experiencing. Aliasing is a crap-shoot, you rarely get what you want.

Cheers,
Bart

[LKaven]

That's most likely what you're experiencing, a higher MTF response near the Nyquist frequency.

So would you agree with me that this is caused by factors above and beyond a difference in OLP filters?  In other words, if neither the L-resolution camera nor the H-resolution camera had OLP filters, there would still be an advantage (in high frequency detail, up near the Nyquist limit of L) to the H-resolution image even after it was downsampled to L-resolution?

[BJL]

A question and a couple of thoughts, not scientifically tested at all:

Question: how far are you downsampling and still sensing a difference?

1) 2x linear (4x in pixel count) is the minimum for the JPEG or TIFF or whatever to have even one pixel of each color in the collection used to produce each output pixel, going the lower pixel count image has it's pixel count reduced by les than 4x, I can see why it might have less sharpeners.

2) In practice, demosaicing algorithms use a weighted average of data from many nearby pixels, not just the nearest ones of the needed color, so could it be that the "footprint" of each output pixel covers date for, say three or four photosites in each direction, which would mean that only when one goes past about 6x to 8x linear downsampling (36x to 64x pixel count reduction) is the smearing due to this process completely gone?

3) Downsampling can increase the SNR of the output pixels, so could downsampling from more sensor pixels give more local contrast and thus perceived sharpness in some cases?


P.S. in the experiment you describe, of a 12MP RGB output file format (so R, G, B values at each of 12 million locations) from a 12MP Bayer CFA camera (6 million green, 3 million each red and blue) and a 24MP camera (12 million green, 6 million each or red and blue) it would not at all surprise me that the latter gives more resolution, because the 12MP RGB output file format can hold more information than either sensor delivers, and specifically more luminosity information than the 12MP sensor gives, since that is based mostly or entirely on values from green pixels. Also, in each case, the output luminosity values rely on raw data from locations other than that of the output pixel: even the 12 million green pixels of the 24MP sensor are not at the same places as those of the output file.

[ErikKaffehr]

Hi,

I may jump conclusions but I have two observations:

1) Both D3X and D3s have OLP (antialiasing filter), but the OLP on the D3X is optimized for 24 MP while the one on D3S is optimized for 12 MP. So the D3X will smear out detail far less than the D3S.

2) Downscaling an image will introduce aliasing, which my cause the perception of improved detail.

Best regards
Erik

One thing that has pleased me about high-resolution sensors is the way in which high-frequency detail appears to be preserved, even after downsampling to prints size, or even to web size.  The difference between my D3x and my D3s was striking even at small sizes.  I can see it in skin, hair, leaves, textures, etc.  I have no experience with the larger sensors, but have noticed the extra detail present in many images posted here.

But what are the factors that explain that?  I had a hypothesis -- that the cutoff of the low pass filtering was different in each case.  I ventured that this was not just a function of the OLP filters in each respective camera (or the lack thereof in some cases), but also a function of the total regimen of oversampling followed by downsampling.  Somewhere up near the Nyquist limit, the D3s starts to fragment.  In comparison at the same output size (12MP or less in this case), the D3x would produce good high-frequency detail.  This was illustrated in a quick experiment that Lloyd Chambers did with these cameras, photographing the label on a soup can and noting the smoothness of the text in the D3x capture when downsampled to 12MP, in comparison to the fragmented text D3s capture (at it's native resolution).

In an exchange with Bob Newman over at DPR, Bob suggested that I need look no further than the difference between the OLP filters on these two cameras.  The D3x does of course have a more gentle filter considered as a function of sampling frequency.  But I wonder if Bob's claim is true?  I asked Bob if a D3s with no OLP filter would produce output as detailed -- or more so -- than a D3x image downsampled to the same resolution.  Bob believes the answer is yes. 

Is that correct?  I had surmised that there were more factors involved with this.  For example, I had always thought of a downsampling filter as being in essence a LPF, with a cutoff that might vary somewhat in slope, depending upon the method.  I feel that surely I am getting additional benefits from supersampling and other processing that manifests itself in the retention of detail after downsampling.  But I don't have either the empirical or analytical tools at hand to resolve this question. 

Among those of you here who have investigated in this area, do you have either an empirical or analytical answer to this, or just a good hypothesis? 

[BJL]

To combine comments on OLP filters with mine about demosaicing algorithms: does it make sense that to avoid aliasing, the OLPF should be limited to the lowest spatial resolution of each color, meaning to the resolution of the 6 million red (and 6 million blue) pixels in a 12MP Bayer CFA camera? If so, green resolution gets squashed down to that level too, suggesting that downsampling to anything above about half the camera's pixel count will have resolution limited by the OLPF/sensor combination rather than the downsampled pixel count.

[hjulenissen]

To combine comments on OLP filters with mine about demosaicing algorithms: does it make sense that to avoid aliasing, the OLPF should be limited to the lowest spatial resolution of each color, meaning to the resolution of the 6 million red (and 6 million blue) pixels in a 12MP Bayer CFA camera? If so, green resolution gets squashed down to that level too, suggesting that downsampling to anything above about half the camera's pixel count will have resolution limited by the OLPF/sensor combination rather than the downsampled pixel count.

My guess would be that the OLPF is a compromise between the spatial resolution of each color channel (slightly blurry for the green, slightly aliasy for red/blue).

-h

[LKaven]

To combine comments on OLP filters with mine about demosaicing algorithms: does it make sense that to avoid aliasing, the OLPF should be limited to the lowest spatial resolution of each color, meaning to the resolution of the 6 million red (and 6 million blue) pixels in a 12MP Bayer CFA camera? If so, green resolution gets squashed down to that level too, suggesting that downsampling to anything above about half the camera's pixel count will have resolution limited by the OLPF/sensor combination rather than the downsampled pixel count.

Sounds like you are suggesting two possible sources for increased image fidelity, both of which seem plausible.  They seem to involve (1) Offset of R->R pixels, B->B pixels, and variously, G->G pixels, and (2) Blur radius of OLPF

In the case of (1) we're talking about a sampling frequency for each color component {R,G,B}, and suggesting that downsampling beyond that frequency offers a threshold of fidelity.

In the case of (2) we're talking about averaging out the effects of smearing in the OLPF to the point where they go from being consequential to being inconsequential.

Have I got this approximately right? 

[ErikKaffehr]

Hi,

MTF using Imatest based on D3X and D3S based on test images from The Imaging Resource.

First case: Both imported by default setting in LR

Second case: Both exported as JPEG in Lightroom, scaled to D3S image size using Lightroom, default settings.

Best regards
Erik

[ErikKaffehr]

Hi,

This compares LR conversion of D3S with LR converted D3X tiff downscaled with bicubic sharper. It seems that bicubic sharper sharpens significantly but also has very high MTF at Nyquist.

The enclosed crops show aliasing effects on the downsampled Nikon D3X image (the one on top). The bottom image is actual pixels from D3S. The image is at 200% for easier viewing.

Note: When the D3X arrived it outresolved the test target Imaging Review used, so they photographed it at doublie distance. The D3S was tested with a newer test target of higher resolution. For the MTF measurement this does not matter.

Best regards
Erik

Hi,

MTF using Imatest based on D3X and D3S based on test images from The Imaging Resource.

First case: Both imported by default setting in LR

Second case: Both exported as JPEG in Lightroom, scaled to D3S image size using Lightroom, default settings.

Best regards
Erik

[LKaven]

Erik, this is beautiful!  I'm sleep-reading them now.  Shh.  More when I wake up.

[BJL]

Erik,

    Thanks for all those graphs.

What do the red lines marked with suffix  "(corr)" signify?

Do the results showing sharpening with bicubic down sampling go with the comment in the thread on "N7 downsampled vs M9" http://www.luminous-landscape.com/forum/index.php?topic=60303.msg486061#msg486061 about the hazards of comparing downsampled to unsharpened images?

Or is it instead that by passing through JPEG conversion first, as in your first set of graphs, some sharpness advantage of the D3X files is lost?

Either way, it makes me realize how difficult resolution and sharpness comparisons are due to the intervention of different demosaicing algorithms and such.


P.S. I found the following reference explaining "(corr)". Those curves are with some standardized sharpening, which tries to compensate for the different sharpening in images from different sources. So this does seem the best comparison to use, and it also indicates the sharpening in bi-cubic down sampling from the D3X, since this sharpened red curve is no higher than the unsharpened curve.

http://www.imatest.com/docs/sharpness_comparisons/

[LKaven]

This really helps to address some key questions about the benefits or otherwise of high resolution sensors.  

I too was curious about MTF50(CORR).  I see Imatest has a reference page on measuring sharpness, but I couldn't find reference to it there.  Perhaps I needed to drill down further.
http://www.imatest.com/docs/sharpness/

I'm focusing on the TIF test, which doesn't involve questions about JPG artifacts and conversion.

It seems that the downsampled D3x image has significantly higher MTF than the native D3s image.  If I'm taking MTF50(CORR) to be the relevant comparison, there seems to be some advantage to the D3x image just beginning around 1000 LP/PH, and becoming somewhat significant by 1500 LP/PH.  I had not expected to see such a broad difference, but more of a difference up around the Nyquist frequency of the D3s sensor.  Am I reading it correctly?  It makes me wonder how other factors of conversion, downsampling method, sharpening, etc, figure into the results.

Do I also read correctly that the edge profile is sharper on the D3x, but also has a "ring," possibly an artifact of bicubic sharper?  I wonder how that would fare using Lanczos windowing?

This is really useful data, Erik, and I'm looking forward to seeing your interpretation.

[Chris Livsey]

I think I am correct in that the RAW files from Nikon (and others) are not "pure" data but are manipulated by the software in the camera before writing. In that case if the software algorithms between the D3x and the s are different, and given they are different sensors they should be, the differences seen when files are processed externally in the same way to the same file size may be partially down to the "optimisation" deemed correct by the software internally.
So is the effect of downsampling larger sensors yet retaining detail as reported found in other "families" eg the Sony range SLR the Phase One range MF ? Is it a provable effect or subjective? (Not against subjective BTW).