Will jpeg XR format emancipate us from RAW conversions at last?

[Jonathan Wienke]

The point, as I tried to make clear, is that those 2^32 nonlinearly encoded values are spread over a much larger linear range.  If the values in the linear range that are skipped over are not discernable due to noise in the system, it is just as well they were left out.  As I pointed out in my initial post, Nikon's "lossy" compression works to the extent that it is properly implemented because of this fact.

So what? What you're advocating here is a form of lossy compression, a gain achieved by quantizing multiple large integer values to the same floating-point value. In practice, there is no difference between doing that and using standard RGB integer values with a defined gamma/TRC curve. It's nothing more than an alternate method of gamma encoding. As such, it is just as undesirable to use JPEG XR as a RAW format as any other integer-based RGB file format with a gamma >1.

Let's go back to the question posed in the topic title--can JPEG XR eliminate the need for RAW conversion? Definitely not. Before the linear RAW data from the sensor can be presented as an image, a bunch of stuff needs to be done to that data. At a minimum, we must:

1. Demosaic the RAW data to linear RGB.

2. White balance (scale the demosaiced RGB channel values so that neutral colors have equal R, G, and B values).

3. Convert to an RGB editing space (which involves changing the TRC and using some sort of camera profile to convert from demosaiced RGB values to the appropriate destination space RGB values).

In-camera processing rarely does any of these tasks optimally. The best demosaic algorithms need more computing horsepower than is typically stuffed inside a camera. Shooting conditions can easily fool automated white balance sensors, and often the preferred white balance chosen for creative reasons rather than absolute technical accuracy. And given camera-to-camera variation, being limited to the standard factory color profile baked into the camera firmware is usually far from ideal. On top of that, there's the issue of level and curve adjustments that are best done on an image-by-image basis--no camera firmware is good at doing custom curve adjustments.

RAW/DNG is the closest thing you can get to the unadulterated original data straight off the sensor. Current lossless compression algorithms can shrink standard RAW file size to something fairly comparable to JPEG XR, but with no adulteration of the RAW data. The whole point of shooting RAW is to have the unadulterated RAW data to work with, so that you don't lose any DR or color gamut from in-camera processing, and have maximum flexibility to edit and adjust the image without causing unacceptably high levels of artifacts. JPEG XR doesn't change that one bit.

[bradleygibson]

Quick replies:

All, firstly I would like to say how much I appreciate the change in the tone of this discussion from where it started out to a much more respectful exchange.  I firmly believe forums such as this one should be a place of coming together and exchange of ideas.  When we discuss thoughts, opinions and ideas and yes, even disagree on some things but do it respectfully, I think that all participants can come away enriched from the experience.

Sandy, a JXR 16f encoding would provide 1024 levels for each stop encoded.  Values are determined starting at the sensor saturation point (mapping to 1.0) and halving the numeric value for each stop.  As Emil points out, the numeric range of these 1024 levels is not constant, but the precision does remain proportional to the relative numbers of photons being captured on a per-stop basis.

Sandy, Emil, it may have too much fidelity in the shadows, but overall being able to avoid clipping seems like a bigger fish.  That is the one we went after.  In fact it was not the efficiency of the encoding or the excess fidelity in the shadows that caused the most controversy, it was the 'only' 1024 levels in the highlights (the corollary).  Still, seems OK, and there's always 32f, or 16i if not.

Gabor, Agreed that it's all a bet on what photographers want.  Only time will tell if it catches on as envisioned, as two distinct types of files as you've suggested, or even at all.

Best regards,

[ejmartin]

So what? What you're advocating here is a form of lossy compression, a gain achieved by quantizing multiple large integer values to the same floating-point value. In practice, there is no difference between doing that and using standard RGB integer values with a defined gamma/TRC curve. It's nothing more than an alternate method of gamma encoding. As such, it is just as undesirable to use JPEG XR as a RAW format as any other integer-based RGB file format with a gamma >1.

It's not lossy compression when there is no information content to using a larger number of levels; quantizing noise is never an efficient use of bit depth, does not increase the informatino content of the image data, and linear integer data encoding does a lot of noise quantization over most of the range.  There is also a big difference between using nonlinear encoding on the output of a RAW conversion vs using it on using it on the RAW data itself.  

I am not advocating the use of floating point; as I mentioned above, the most efficient use of bit depth is to use a gamma=2.0 encoding together with a linear segment at low levels.  Similar to Nikon's original compressed NEF format.  But even though floating point is not optimal, it can achieve better data compression without loss of image information content than integer encoding without gamma transformation.  And it was your incorrect statement that there was no distinction (and the uncivil tone in which it was offered) that I responded to in entering this thread.

Let's go back to the question posed in the topic title--can JPEG XR eliminate the need for RAW conversion? Definitely not. [snip]

Agree with what you had to say here.

[joofa]

We chose s10e5 (one sign bit, 10-bit mantissa and 5-bit exponent) partly for just the reason you mention -- to cover the anticipated DR.

In my understanding having a higher DR is not necessarily synonymous with higher SNR all the way. In general floating point would provide higher SNR except at the highest signal levels. To use your numbers, the comparison of 16 bit uniform quantization and s10e5 (one sign, 10-bit mantissa and assuming hidden bit, and 5-bit exponent) would be:

16-bit-Uniform-Quantization: DR = 90 dB, and SNR=98 dB (sinusoidal input wrt quantization noise).
Floating-point, s10e5: DR = 253 dB, and SNR = 74 dB (sinusoidal input wrt quantization noise).

DR for floating-point is much higher. As far as SNR is concerned the floating point will have higher SNR if the signal intensity is below the maximum by more than 24 dB.
 
If you have chosen s11e4 you would have gotten 80 dB SNR, but, then the DR would have reduced to 163 dB.

[bradleygibson]

Agree with what you had to say here.

I guess we'll have to agree to disagree on that one, and see what happens.

[Jonathan Wienke]

It's not lossy compression when there is no information content to using a larger number of levels; quantizing noise is never an efficient use of bit depth, does not increase the informatino content of the image data, and linear integer data encoding does a lot of noise quantization over most of the range.  There is also a big difference between using nonlinear encoding on the output of a RAW conversion vs using it on using it on the RAW data itself.

Whenever you throw information away (which is what you are always doing when you map multiple input values to a single output value) you are not improving the fidelity of the digitized data. If the data you throw out is mostly noise, then fidelity is not being compromised much. But unless what you're throwing out is 100% padding, there's always some signal that is being lost, to the detriment of the fidelity of the final encoding. Whether this is relevant in practice needs to be determined on a case-by-case basis.