What causes that ugly over saturated yellow "blooming" when shooting certain sunset pictures at certain exposures? It seems to get worse when shooting into the sunset itself, even after the sun as set.

White balance is required beyond the limits of adjustability. Like hitting a wall. That blocks the color up.

Can you link to a RAW file?

Yep, here you go:
http://www.dwdallam.com/shared/

I have my server in list mode for this directory. You'll see it when you get there.

I've seen the same thing on film prints at shows. IS this also a film phenomenon, and I assume it is something we have to live with?

Nothing is clipped in the RAW data. The highlights look linear. Perhaps it is your WB settings? Your "as shot" setting is bad, and ACR's auto looks bad, too. Daylight is what I would use as a starting point for this.

I always play with the white balance. Could this be a damaged sensor?

It's clipping. Setting the color temp to about 4400 minimizes it, but does not get rid of it at 0 EC. Given ACR's internal highlight recovery, if you show clipping at 0 EC, the RAW data is clipped, and other converters will block up the highlights with solid colors. Back off your exposure 1/3-2/3 of a stop, and the problem will go away.

Jon, I think this is the reason I was questioning the histogram conversation we had a while ago, that they don't lie when shooting RAW. I shot that with the camera histogram being a seemingly perfect gram, if my memory is faithful, and it got blown out.

I may be wrong here, if my memory fails me, but if you pop that into your 5D, what does the histogram tell you about it? If it tells you it's a pretty good exposure, then we have a problem. If I'm remembering correctly, then I need to adjsut something that affects the jpg histogram that is being read for the raw file?

Good to know nothing is wrong with my camera though.

Play with the white balance where?

There is nothing wrong with your RAW image.

The RAW data isn't clipped, though. Not a single clipped RAW pixel in the image. This image shows up a weakness in RAW converters.

Would you like me to post my manual conversion from the RAW?

Your problem with my histogram-based exposure recommendation is that you obviously never bothered to do the testing I recommend to determint the exact relationship between what the histogram says and the RAW data. Once you nail down the exposure interval between histogram clipping and RAW clipping, and incorporate that into your exposure adjustments, shooting by histo works just fine, even when exposing to the right. You can't claim the method doesn't work when you're only doing half of it.

Play with it where--in ACR? You know, where is says "white balance," 'temperature," and "tint." I rarely change the WB in camera unless I know I'm going to be shooting in light I know doesn't change every 10 minutes--such as daylight without clouds, street lights at night, or strobes in a studio--which sunsets are not. In other words, if you need to adjust theWB a little after shooting, why do it twice, once in camera and once in your software?

No, you are quite right. I'm not saying it doesn't work though, just that, as you have stated, you must do some extra work to get it to work more accurately. I think a good starting point would probably be to turn the jpg contrast all the way down as low as it will go. I read that was a good practice somewhere.

Probably on my site. But that still doesn't excuse you from doing the testing. If you can't be bothered to do it, you really don't have any right to complain when you have exposure difficulties.

First, I'm not complaining. "Discussing" is not "complaining."

Second, just because I don't take your suggestions and apply them physically doesn't mean I don't learn by them.

Third, I had no idea exposure was directly linked to yellow blooming. So even if I had applied your information, I may still have gotten yellow blooming while bracketing, and we'd still be having this conversation.

Last, although you are extremely well versed in this technology and I greatly respect your time and knowledge, you over-extend that knowledge and diminish that respect when you tell me how and when I have a right to ask questions.

When you ask for the solution to an exposure problem after you've already been given a strategy for obtaining optimal exposure, it's hard to see how you learned from the original discussion about histograms and exposure we had. If you had applied the advice I gave you previously, you wouldn't have had any reason to start this thread.



If you want to be respected by people here and have them continue to take the time to answer your questions, you might want to start applying the advice you're given. Otherwise people will tend to regard answering your questions a waste of time and may stop doing so.

There is no RAW exposure problem within the RAW you linked to. Any less exposure would have resulted in a darker version of the same thing, with a lower signal-to-noise ratio. Converters often boost the saturation of certain colors, causing clipping or near-clipping of one or two color channels in the output. If its a highlight area, it can be even worse. Perhaps you are seeing a combination effect of this fact, and the converter's inability to handle exposure to the right.

My simple manual, step-by-step conversion with IRIS has colors that look normal to me. Should I post a small copy of it here? When I get home, I will post a RAW histogram so you can see that it is not clipped.

I've never seen your render, or a precise description, so I don't know what is wrong with your conversion. Your embedded JPEG is very pale, which is to be expected if you set contrast low and expose to the right.

The RAW exhibits clipping when exposure is set to 0 EV in ACR. It can be gotten rid of by setting the exposure to about -.40EV, but that's using ACR's highlight recovery to deal with clipped RAW data. Not all RAW converters handle clipped highlights well; Canon's converter tends to output ugly blocks of solid, saturated colors under such circumstances, which is why I quit using it.

It's blown, sorry mate, the scene just has too much DR for the exposure you've given it, i.e. you've been opening the rest of the picture too much to hold the brightest portion of the sky. Bracketing and making an HDR image or using a double rendition of the RAW file using the aforementioned ACR recovery trick and using your favorite method to combine the two is the way to go...

There is no clipped RAW data. Here is a RAW RGB histogram of the image; the top is the entire image, the bottom is a selection of a rectangle around the brightest 2/3 of the image. The left edge is RAW 128 (black), and the right edge is 3652 (clipping point). The scale is linear:

You're entitled to your opinions about how you think I should learn, but you're not entitlted to talk to me in a condescending manner for any reason. If that's unaceptable to you, then simply ignore my posts.

Respect is a two way street.

JS,

This is quite interesting. So does this mean that the exposure is in the tolerance of "correct" but the yellow blooming still happened? So what does this tell us then? It seems like Bob's simple response was closer than the rest?

I don't see any WB problem, nor do I see any problem with the image, other than the fact that the embedded JPEG is very pale (I shoot that way, too, though, with JPEG Contrast set to minimum, and ETTR). All I can tell you is that no clipping of the RAW has occured. If you are having a WB issue, it is probably the converter. I didn't see the original sunset, so I don't know what it looked like to you, so I can't see that anything has gone wrong.

If you mean that you saw all oranges and reds in the clouds, and some of them came out yellow instead?

John, what tool did you use to generate that RAW histogram? What it shows is quite at odds with what I'm seeing when I open the RAW in ACR, which is significant clipping when converting to ProPhoto with a 0EV exposure setting at any WB setting. I have to back the exposure off by at least 1/2 stop to avoid clipping if I want to play with WB at all.

I loaded it into IRIS, subtracted 128 from the image, set the clipping points in the threshold dialogue to 0 and 3524, called on the "convert CFA image" menu option, and saved out as a BMP. I loaded the BMP into PSP8, which has a histogram tool that is easy to read in RGB mode.

So, whatever problem people are having with ACR, it would probably not have the problem if then RAW data were simply scaled by 0.7 or so - something that you would think that the "Exposure" slider should do, but doesn't.

The blackpoints and clipping points are different for different cameras models, and for different ISOs in the same camera, so if they are unknown, a deliberately clipped and a black file are needed to determine these values. All current Canons except the Rebels are 128 for black; the Rebels are 256-258 depending on ISO. Nikons are generally 0, which is not as good for binning/stacking, etc.

I see the same thing, but I used that image to show the worst case blooming for that partuicular shoot. I think I have a few that are actually underexposed, according to ACR, but still bloom in the yellow. I've seent ath in other shoots too.

dwdallam, I've asked you twice already if you want me to post my manual RAW conversion, and you haven't replied. I am asking one more time.

I'm sorry, of course. Anything to add to my knowledge. Remeber that this image isn't anything I need or want to use. It's simply a good example of the topic in question.

Here's the process in stages:

1) The upper left is the literal linear RAW image, after interpolating the color channels fully (clipping at black, and at the brightest pixel (a green 3247 out of 3524)).

2) The upper right is after white balance - I guestimated 1.9 for red, and 1.4 for blue scaling (still linear; highest red value is now 4364). I set the new upper clipping limit to 4364 to preserve everything.

3) The lower left is with a gamma of 2.4 applied. 2.2 is standard, but with this high-DR scene, I figured 2.4 would be a little better.

4) The lower right has saturation boosted globally.



Still, what in real life looks reddish orange is a yellowish orange. Applying a hue shift towards red for a selective range of oranges, and applying the Photoshop Shadow/Highlight tool to the shadows, I get this:

I was playing around with the image in ACR again today, and it seems pretty clear that ACR clips it quite easily because of implicit boosted saturation; a very common problem with converters. They try too hard to make certain colors as vibrant as they seem in real life, and they tend to over-do it, or where it isn't applicable. I would venture to say (as I've seen it before), that if the the blown areas in ACR were actually higher in RAW RGB values, but were white or a different hue, the actual converted values in one or two channels would be less. Case in point as I remember - the red and white squares in the Color Checker - the RAW r values are slightly less in the red square than they are in the white square, but in conversion, the red of the output is higher in the red square, IIRC. Or it may have been the red foam ice cream stick I put next to the Color Checker.

Actually, that is one picture I will probably use for display--not that specific one, but one similar. I'd forgotten which one I uploaded. Your conversion is not too bad. Very good explanation and examples too. What converter did you use?

I used IRIS to load the RAW data, and interpolate the color planes. Those are the only things I need it for, although I did use it for most steps, but they could have been done in PS. Only the selective hue shift and shadow/highlight adjustments were done in PS.

Actually, I could do it all in PS (without ACR) if I wanted to, by getting the RAW bitmap from an uncompressed DNG, and using my own simple demosaicing, but the first stages are easier in IRIS.

The bottom line is that there really is no RAW clipping in that capture; only clipping of output in RAW converters. The significance of this is that RAW converters are forcing you to under-expose (ETTL) to avoid clipping in their software (increasing noise), when they could actually just scale the data with a good exposure. This is a major problem with converters, and I think it is important that people know this so that they can complain, instead of drooling over spoon-fed infinitessimal improvements with new versions of converters. Literal scaling of exposure should have been in every RAW converter from day one.

I just saw a screen shot in an article that sugggests that the ACR in CS3 actually allows control of saturation, hue and intensity based on color group. That might help with these overly-saturated colors.

Yeah it always amazed me that ACR didn't have a seprate saturation slider for RGB. CR3 is supposed to be out when? I assume it's 64 bit? Thanks for all the work and information you did.

Sometime this spring or summer, is my guess, since it's in beta now. Download it and test it if you want; except for some printing issues and magically disappearing menus, it's reasonably stable.


If you're thinking about memory limits, no.