Ad
Ad
Ad
Pages: [1] 2 3 ... 12 »   Bottom of Page
Print
Author Topic: Does a raw file have a color space?  (Read 116185 times)
bjanes
Sr. Member
****
Offline Offline

Posts: 2825



« on: January 18, 2008, 07:14:33 AM »
ReplyReply

Quote
Raw has no color space. And it doesn't matter what you set your camera to (you're shooting Raw).
[a href=\"index.php?act=findpost&pid=163585\"][{POST_SNAPBACK}][/a]

Does a raw file have a color space?

The question is, “Does a raw file have a color space?” Some answer no for various reasons. Some say that the raw file can not have a color space because it is monochrome and contains no color. Of course, no file can contain color, since color is a perception and exists only in the eye and brain. What raw files and RGB files do contain is color information in the form of tristimulus color values that are perceived as color when projected on the retina. The RGB file contains three color values per pixel, for a complete RGB representation. The raw file of a Bayer array camera
contains only one color per pixel, and the other two are interpolated from adjacent pixels in the demosaicing process. Nonetheless, color information is present in the raw file and does not magically appear during demosaicing.

The recognition that the raw file does have a color space is useful in understanding how raw files are processed. Examples with code are available in DCRaw and Dng_verify, a command line program written by Thomas Knoll.

In the DNG program and DCRaw, 3 by 3 matrix conversions are used to convert from the camera space to an internal working space, in the same fashion as one would convert from ProPhotoRGB to Adobe RGB. In the case of DNG, the working space is linear ProPhotoRGB, while in the case of DCRaw, it is apparently CIE XYZ. DNG uses the dng_matrix & CameraToPCS () function to determine the white point x,y values. Thus, it is apparent that the raw file contains all the elements of a color space, either explicitly or implicitly in the form of metadata, camera profiles, or other information. Furthermore, in the source code quoted below, Thomas Knoll refers to the “camera native space” and “camera color space”, which would indicate that he thinks that the camera does have a color space that is represented in the raw file and the camera profile.

[span style=\'font-size:14pt;line-height:100%\']
Appendix[/span]

RGB Matrix profile (example is aRGB) :

Gamma 2.2
White point
K 6500
 x, y: 0.3127, 0.3290
Primaries
   x      y
R   0.6400,   0.3300
G   0.2100,    0.7100
B   0.1500   0.0600

Raw File
Primaries: determined by CFA filters and silicon spectral response, stored in camera profile

Gamma 1.0
White point: stored in tag to raw file
White point x, y: recovered by dng_matrix & CameraToPCS () function in DNG_verify


3 by 3 matrix for Canon 1Ds, used by DCRaw

6517,-602,-867,
-8180,15926,2378,
-1618,1771,7633
Code Extracts

/*****************************************************************************/
// Copyright 2006 Adobe Systems Incorporated
// All Rights Reserved.
//
// NOTICE:  Adobe permits you to use, modify, and distribute this file in
// accordance with the terms of the Adobe license agreement accompanying it.
/*****************************************************************************/

/* $Id: //mondo/dng_sdk_1_1/dng_sdk/source/dng_render.cpp#1 $ */
/* $DateTime: 2006/04/05 18:24:55 $ */
/* $Change: 215171 $ */
/* $Author: tknoll $ */

/*****************************************************************************/
for (int32 srcRow = srcArea.t; srcRow < srcArea.b; srcRow++)
      {
      
      // First convert from camera native space to linear PhotoRGB,
      // applying the white balance and camera profile.

const dng_matrix & CameraToPCS () const;

/// Return the XY value to use for SetWhiteXY for a given camera color space coordinate as the white point.
/// \param neutral A camera color space value to use for white point. Components range from 0.0 to 1.0 and should be normalized such that the largest value is 1.0 .
/// \retval White point in XY space that makes neutral map to this XY value as closely as possible.


DCRAW
/*
   Thanks to Adobe for providing these excellent CAM -> XYZ matrices!
 */
  { "Canon EOS-1Ds Mark II", 0,
   { 6517,-602,-867,-8180,15926,2378,-1618,1771,7633 } },
Logged
papa v2.0
Full Member
***
Offline Offline

Posts: 197


« Reply #1 on: January 18, 2008, 09:06:20 AM »
ReplyReply

camera colour space

Yes i would agree with mr bjanes



 In single chip cameras, Raw files tend to be 'sensor referred RGB' after demosaciaing.

Three chip cameras (and scanners) produce a 'sensor referred RGB' file, no demosaicing!


we also tend to refer to 'camera colour space' as  device coordinates in order to differentiate from,  for example CIE  colour spaces,  CIEXYZ or CIELAB etc or from output referred colour spaces such as sRGB , Adobe RGB, ProPhoto.



saves a bit of confusion – well for me at least!
Logged
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #2 on: January 18, 2008, 10:29:41 AM »
ReplyReply

Does a raw file have a color space?

When tryig to answer this (and other) questions, it is the very least useful to quote the opinion of other people. Instead, one needs to analyse the issue.

The first step is always to agree on the definition.

Examples, picked from the internet:

- A mathematical way to describe color information. In different color space schemes all the different formulas used essentially describe the same colors

- A model for representing color in terms of intensity values; a color space specifies how color information is represented. It defines a multidimensional space whose dimensions, or components, represent intensity values

- A system for ordering colors that respects the relationships of similarity among them. There are variety of different color spaces, but they are all three dimensional

now, that's general, isn't it?

- A map or chart of the visible portion of the spectrum

Even more generous.

- A two or three dimensional geometric representation of colors that can be seen or generated. RGB is an example of a two dimensional color space. LAB is a three dimensional color space

Refreshing news.

- A system for describing color numerically

A color model is an abstract mathematical model describing the way colors can be represented as tuples of numbers, typically as three or four values or color components. When this model is associated with a precise description of how the components are to be interpreted (viewing conditions, etc.), the resulting set of colors is called a color space

- A device color space simply describes the range of colors, or gamut, that a camera can see, a printer can print, or a monitor can display. Editing color spaces, on the other hand, such as Adobe RGB or sRGB, are device-independent. They also determine a color range you can work in

The definitions are quite different, and I am afraid there won't be any agreement on which one to use.

However, note that the rubbish "not a color space because it contains greyscale channels" can not be reasoned from any of these definitions.

Anyway, IMO the raw files have color spaces, namely the respective camera's color space.

A side note: DNG has nothing particular to do with ProPhoto RGB.
« Last Edit: January 18, 2008, 10:32:47 AM by Panopeeper » Logged

Gabor
papa v2.0
Full Member
***
Offline Offline

Posts: 197


« Reply #3 on: January 18, 2008, 10:41:27 AM »
ReplyReply

A two or three dimensional geometric representation of colors that can be seen or generated. RGB is an example of a two dimensional color space. LAB is a three dimensional color space

I dont think that bit is right RGB, 2D colour space?
who said that?
Logged
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #4 on: January 18, 2008, 10:56:22 AM »
ReplyReply

Quote
I dont think that bit is right RGB, 2D colour space?
who said that?

I don't remember from where I took it; it's irrelevant, my intention was to demonstrate the multitude of understandings.
Logged

Gabor
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #5 on: January 18, 2008, 11:09:17 AM »
ReplyReply

I add one more definition:

The specific range of colors in a particular model

and the definition of model is

The dimensional coordinate system used to numerically describe colors
Logged

Gabor
papa v2.0
Full Member
***
Offline Offline

Posts: 197


« Reply #6 on: January 18, 2008, 11:11:01 AM »
ReplyReply

thats ok then
Logged
bjanes
Sr. Member
****
Offline Offline

Posts: 2825



« Reply #7 on: January 18, 2008, 01:28:00 PM »
ReplyReply

Quote
Does a raw file have a color space?

However, note that the rubbish "not a color space because it contains greyscale channels" can not be reasoned from any of these definitions.

Anyway, IMO the raw files have color spaces, namely the respective camera's color space.

[{POST_SNAPBACK}][/a]

Some prominent photographers have taken the rubbish route, including [a href=\"http://luminous-landscape.com/forum/index.php?showtopic=21046&view=findpost&p=155674]Jeff Schewe[/url] and Andrew Rodney.

Your caveat about definitions is important. Some make them up as they go along, while others quote selectively. For simplicity, I chose to compare the characteristics of aRGB with the Camera space, noting their similarities.
Logged
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #8 on: January 18, 2008, 01:55:56 PM »
ReplyReply

Quote
Some prominent photographers have taken the rubbish route, including Jeff Schewe and Andrew Rodney

Mistakes happen, This does not take away anything from their being prominent photographers.

Quote
I chose to compare the characteristics of aRGB with the Camera space, noting their similarities.

Those similarities are not greater, than the similarities to sRGB, for example.

Anyway, similarity is not a supplement for the definition. Saying "a color space is something like aRGB, but slightly different" does not cut it.

What definition do you take as basis for the discussion about the "color spacebility" of camera sensors?
Logged

Gabor
Schewe
Sr. Member
****
Offline Offline

Posts: 5500


WWW
« Reply #9 on: January 18, 2008, 02:30:55 PM »
ReplyReply

Quote
Some prominent photographers have taken the rubbish route, including Jeff Schewe and Andrew Rodney.
[a href=\"index.php?act=findpost&pid=168012\"][{POST_SNAPBACK}][/a]

Ok. . .what color space does the Canon 1DsM3 raw file have? What are the camera color's color coordinates? What are the spectral properties at D65 and tungsten? What are the separation filter properties of the Bayer RGGB photo sites?

The raw file has color information that, until somehow represented as color data (either via demosaicing or some other color interpretation) is grayscale values.

So, to me that makes it a grayscale file waiting to be interpreted as color data...but it's still grayscale until somebody somewhere does something to it.

If walks like a duck and quacks like a duck...

And one wonders what _YOUR_ definition of "rubbish" is? Garbage, worthless material or nonsense?

When I use DNG Validate to process a raw file without demosaicing...I get a linear grayscale file.
« Last Edit: January 18, 2008, 02:38:26 PM by Schewe » Logged
bjanes
Sr. Member
****
Offline Offline

Posts: 2825



« Reply #10 on: January 18, 2008, 03:14:47 PM »
ReplyReply

Quote
Ok. . .what color space does the Canon 1DsM3 raw file have? What are the camera color's color coordinates?
[a href=\"index.php?act=findpost&pid=168027\"][{POST_SNAPBACK}][/a]

Had you read my post, you would already know the answers to these questions. I don't have the matrix values for the 1DsMIII, but here are the values for the 1DsMII from DCRaw. I had already listed those for the 1Ds. What more do you want?

 6240,-466,-822
-8180,15825,2500
-1801,1938,8042
"Canon EOS-1D Mark III"

Quote
What are the spectral properties at D65 and tungsten? What are the separation filter properties of the Bayer RGGB photo sites?
[a href=\"index.php?act=findpost&pid=168027\"][{POST_SNAPBACK}][/a]

The spectral characteristics of the RGGB sites are characterized in the matrix.

The spectral properties for D65 and tungsten have been developed by Thomas Knoll for use in ACR, and are included in his profile for that camera. All information does not need to be included in the actual file, but can be given by reference. If I save a file in ProPhotoRGB, without the tag, then I restore the meaning of the numbers when opening in Photoshop with the assign profile command. By analogy, the same principle applies to the raw file. The white point is also encoded in the file, and the dng_matrix & CameraToPCS () function is used to determine the white point x,y values.

Quote
When I use DNG Validate to process a raw file without demosaicing...I get a linear grayscale file.
[a href=\"index.php?act=findpost&pid=168027\"][{POST_SNAPBACK}][/a]

You are working with a file containing tristimulus color information. Whether or not you make use of it is up to you. A true grayscale file has no color information, only luminance values.

When I look at a TIFF image with a binary editor, I get only numbers. Where is the color? When I delete the red and blue channels from a TIFF, there is no color. What happened? Perhaps a different approach is needed. I agree with Panopeepr, you are taking the garbage route in your argument.
Logged
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #11 on: January 18, 2008, 03:19:22 PM »
ReplyReply

Quote
Ok. . .what color space does the Canon 1DsM3 raw file have?
The Canon 1DsM3 color space.

Quote
What are the camera color's color coordinates?
Nonsensical question.

Quote
What are the spectral properties at D65 and tungsten?
The sensor has spectral properties, not the image, nor the color space.

Perhaps you wanted to know, how the color space can be described in CIE terms.

at D65 the camera's color space a the result of the conversion from CIE XYZ using following matrix:

{{0.5859, -0.0211, -0.0930}, {-0.8255, 1.6017, 0.2355}, {-0.1732, 0.1887, 0.7448}}

I don't know it for tungsten.

Quote
What are the separation filter properties of the Bayer RGGB photo sites?

Again, you are mixing up the properties of the sensor with that of a colorspace.

Please note, that the question if the sensor has a color space does not depend on the fact, that you or I or anyone else knows the details of those properties or does not. What you are doing is something like seeing a bird over your head, and if I can't tell you the name of the bird, then you don't accept that it is a bird. Well, if you can't tell a bird and a Boing apart without knowing the model name, that's your problem.

Quote
The raw file has color information that, until somehow represented as color data (either via demosaicing or some other color interpretation) is grayscale values

The same can be stated re the sRGB channels. Going further: is Lab a color space? What color do the "a" and "b" channels have? What is the color of the "L" channel? Is a Lab image a color image at all without knowing, how to interpret the channels?

Quote
When I use DNG Validate to process a raw file without demosaicing...I get a linear grayscale file

You can complain about that for example on the Adobe DNG forum; or you can use Rawnalyze to see non-demosaiced color images.

On the other hand, you can see color RGB images on a black and white monitor. Where did the colors go? Do the characteristics of an image depend on how one looks at them?
« Last Edit: January 18, 2008, 03:19:52 PM by Panopeeper » Logged

Gabor
bjanes
Sr. Member
****
Offline Offline

Posts: 2825



« Reply #12 on: January 18, 2008, 04:54:20 PM »
ReplyReply

Quote
Mistakes happen, This does not take away anything from their being prominent photographers.
[{POST_SNAPBACK}][/a]

That is quite true. But perhaps they should stick to their areas of expertise. In many areas, Jeff is quite accomplished and I rely on his new ACR book and enjoyed his contribution to the From Camera to Print video and will probably get the new ACR video when it comes out.

However, I'm not so naive as to take his every word as the absolute truth.

Quote
Those similarities are not greater, than the similarities to sRGB, for example.

Anyway, similarity is not a supplement for the definition. Saying "a color space is something like aRGB, but slightly different" does not cut it.

What definition do you take as basis for the discussion about the "color spacebility" of camera sensors?
[a href=\"index.php?act=findpost&pid=168020\"][{POST_SNAPBACK}][/a]

There does not seem to be one definition that covers all bases. You have made some good suggestions and perhaps can help supply a working definition.

Quote
When I use DNG Validate to process a raw file without demosaicing...I get a linear grayscale file.
[a href=\"index.php?act=findpost&pid=168027\"][{POST_SNAPBACK}][/a]

If you use switches with DNG_Verify to get gray scale output, it is not surprising that you get gray scale. BTW, there is a nice raw file analyzer program [a href=\"http://www.cryptobola.com/PhotoBola/RawnalyzeGuide.htm]Photobola Raw Analyze[/url] that can show color or gray scale without demosaicing.

With this program you can view in gray scale or color, as well as check clipping and many other characteristics. I think the program was written by Panopeeper, who is participating in this thread.
Logged
digitaldog
Sr. Member
****
Offline Offline

Posts: 9191



WWW
« Reply #13 on: January 18, 2008, 04:59:36 PM »
ReplyReply

Quote
However, I'm not so naive as to take his every word as the absolute truth.
There does not seem to be one definition that covers all bases. You have made some good suggestions and perhaps can help supply a working definition.

I tend to ask other people who I respect as knowing more than I do. So I asked the follow people this morning and here are their replies. This may make you feel better, you seem to wish to prove your point as being correct and I would submit that the language of the past (essentially Grayscale) needs to be altered after asking the questions of these experts. So I should probably thank you for the efforts.

Quote
On 1/18/08 11:09 AM, "RikWalowit@aol.com"  wrote:

<<Does a raw file have a color space?>>

Fundamentally, absolutely YES, but we may not know what that color space is. The image was recorded through a set of camera spectral sensitivities which defines the intrinsic colorimetric characteristics of the image. An simplistic way to think of this (while not purely accurate) is that the image was recorded through a set of "primaries" and these primaries define the color space of the image.

Practically, it makes little difference unless you are interested in accurate scene-referred data. In the context you described, a simple transform is applied to convert from the camera primaries to a new set of primaries (eg CIE or working space) that have more desirable characteristics than the RAW primaries.

Mathematically, of course, you know the foregoing is true. Matrix algebra informs that if the final color image encoding is a "color space" then so must be all of its infinite number of linear transforms (lets ignore the 1 D non-linear transfer functions), including the original RAW encoding.

Eric Walowit
Tahoe

On 1/18/08 9:36 AM, "Thomas Knoll"  wrote:

The fact that a mosaic array is “grayscale” is a red herring in this argument.  An early processing step fills in the missing values, and you have a 3 or 4 channel image as a result.  For most cameras, if you just “assign” a working space RGB profile, you get a recognizable color image as a result, so it certainly seems like a color space.

The camera color space differences from a more common working color space in that it does not have a unique one-to-one transform to and from CIE XYZ space.  This is because the camera has different color filters than the human eye, and thus sees colors differently.  Any translation from camera color space to CIE XYZ space is an approximation because of this.

On 1/18/08 10:20 AM, "Chris Murphy"  wrote:

I think we'd all agree that CIE XYZ, ProPhoto RGB, and sRGB are each a color space. We consider the whole of ProPhoto RGB a color space even though it contains imaginary colors, from a human point of view. And we consider the whole of sRGB a color space even though it contains less than the colors that we humans can see.

A camera has colors it really can capture and encode as unique values compared to others, that are imaginary to us. They don't exist. There are colors we can see, but the camera can't capture that are imaginary to it. So what to do with these captured values that humans can't see? Well they typically get mapped to something, possibly something that another rather different SPD is already mapped to. Two reds that appear the same to us, but have different SPDs, yet the camera sees them as different so it encodes them with two different RGB values. That's fugitive colors. It's nothing new in photography.

So yes a camera (and thus a Raw file) has a color space. The challenge is what to do with the real and "unreal" colors it's encoding. And how to deal with luminance and dynamic range which is still a really big weak area of the ICC workflow. This is not a one size fits all campaign, to get good results. Color appearance models can help us deal with this better, as getting a 1:1 correlation to XYZ is an appropriate starting point prior to color appearance and gamut mapping, but it's not a desirable end result either on-screen or in print.

If we had spectral sensitivities for the camera, that would make the job of mapping to XYZ better and easier, but we'd still have decisions on what to do with the colors the camera encodes, that are imaginary to us.

Chris
Logged

Andrew Rodney
Author “Color Management for Photographers”
http://digitaldog.net/
papa v2.0
Full Member
***
Offline Offline

Posts: 197


« Reply #14 on: January 18, 2008, 05:05:10 PM »
ReplyReply

One should think of the raw file as a data file. (or interleaved rgb composite)

To call it a grayscale is probably not the most perceptual way to describe it.

Of course when it is opened in a imaging application as a composite, it is displayed as if it were a 'grayscale' as the application has no other way of interpreting the data.


I think to call it a grayscale gives it the personification of what most people would think of as a correctly exposed, processed etc  black and white, monotone , greyscale etc picture.

In fact one could open any file  containing numbers (providing it was encoded correctly) and it would display as grayscale.

For example: If asked 1000 random people at Picadilly Circus their age and entered that into an excell worksheet, then import into Matlab, reshape the vector to a 50x20 matrix and wrote the file out to a Tiff format, I could open it in photoshop and it would appear as a grayscale.
What it would look like is anyones guess!

But hey thats given me an Idea for a whole new genre of photography!  Wow.
Logged
John Sheehy
Sr. Member
****
Offline Offline

Posts: 838


« Reply #15 on: January 18, 2008, 05:34:27 PM »
ReplyReply

Quote
One should think of the raw file as a data file. (or interleaved rgb composite)

To call it a grayscale is probably not the most perceptual way to describe it.
[a href=\"index.php?act=findpost&pid=168051\"][{POST_SNAPBACK}][/a]

It has probably happened that RAW became notorious for "not having color" simply because of the geometry of CFAs, alone, since three color receptors can occupy a single channel, flat raster because they never occupy the same locations.  Had all cameras detected full RGB at each pixel from the beginning, no one would be saying that RAW data is colorless; just dark (if linear), desaturated and slightly off-hue.
Logged
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #16 on: January 18, 2008, 05:48:07 PM »
ReplyReply

Andrew,

I appreciate your attitude.
Logged

Gabor
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #17 on: January 18, 2008, 05:49:04 PM »
ReplyReply

There is one point to add, IMO, though I guess this can be debated as well.

Viewing the color space through the conversion matrix is not the ultimate wisdom. It is convenient for sure, and it is ok between orthogonal color spaces. For example sRGB, Adobe RGB, ProPhoto RGB has been defined with reference to CIE, in terms of conversion matrices, so the conversion with matices is natural.

However, cameras' color spaces can not be described in an orthogonal system based on the three (or four) color pixels, because of the overlap between the spectral responses. Therefor the conversion with the matrix (actually, with its inverse) is not satisfactory. I think this is the reason, that no matter how hard one calibrates ACR, one can not get all colors right, not even all colors of the small color checker.
Logged

Gabor
bjanes
Sr. Member
****
Offline Offline

Posts: 2825



« Reply #18 on: January 18, 2008, 05:54:43 PM »
ReplyReply

Quote
I tend to ask other people who I respect as knowing more than I do. So I asked the follow people this morning and here are their replies. This may make you feel better, you seem to wish to prove your point as being correct and I would submit that the language of the past (essentially Grayscale) needs to be altered after asking the questions of these experts. So I should probably thank you for the efforts.
[a href=\"index.php?act=findpost&pid=168050\"][{POST_SNAPBACK}][/a]

Thanks for the info, Andrew. I certainly respect you more for having published comments by TK and CM that do not entirely agree with your previous statements. I don't know the other fellow, but I assume he is an expert also.

I did not want to engage in another flame war over this topic, and had written you privately concerning this matter. On receiving no reply I made the public post. I'm sure you all know more about photography than I do, but in this case I felt I was at least partially correct.

Your thanks are graciously accepted.

Bill
« Last Edit: January 18, 2008, 05:55:10 PM by bjanes » Logged
digitaldog
Sr. Member
****
Offline Offline

Posts: 9191



WWW
« Reply #19 on: January 18, 2008, 06:17:09 PM »
ReplyReply

Quote
Thanks for the info, Andrew. I certainly respect you more for having published comments by TK and CM that do not entirely agree with your previous statements. I don't know the other fellow, but I assume he is an expert also.
[a href=\"index.php?act=findpost&pid=168059\"][{POST_SNAPBACK}][/a]

Eric is up there with color geeks like Thomas. He's got a patent that describes some core functionality within ColorSync and at one time was the fellow who ran ColorSavvy. Now he's a snow board guru. He's the first person I go to when asking about aspects of digital camera color management.

The "Essentially Grayscale" comments date back to another respected color geek and in my case, mentor, Bruce Fraser. I believe I quoted him from his book in the previous debated thread. IF Bruce were alive and had he asked the same questions to this group of experts and received the same answers, I suspect he, (as I will from now on),  would use better language than "essentially Grayscale" to describe this aspect of Raw data.
« Last Edit: January 18, 2008, 06:18:37 PM by digitaldog » Logged

Andrew Rodney
Author “Color Management for Photographers”
http://digitaldog.net/
Pages: [1] 2 3 ... 12 »   Top of Page
Print
Jump to:  

Ad
Ad
Ad