HDR Workflow For The Rest Of Us
By Vegard Brenna
Alexandre Buisse in A Plea For HDR said it all. He speaks for me. One of my favourite photographic subjects is church interiors (no castles in this part of the world), and HDR is indispensable. But I don’t want my pictures to look like HDR. And I also want HDR to be just another part of my toolbox. Does anyone remember as far back as 2003? That was when we corrected exposure using multiple layers and blending modes in Photoshop, and when we created “fill flash” using clever masks. Now with Lightroom there are simple sliders for them all and more. Until the “HDR slider” arrives I would like to have something similar and simple.
In the first part of this essay I will look at the problem of consistent colours in Photomatix Pro, and how it can be solved. The solution applies to any HDR program that creates unnatural but consistent colours.
In the second part I will present a workflow “for the rest of us”, where I use what I found in the first part to effortlessly create a consistent suite of HDR images from one shoot. My arsenal consists of
- Lightroom 2.7
- Photomatix Pro 4.1
- Photoshop CS4
- 6 year old Mac G5 with OS X 10.5.8 (Hence no Lightroom 3)
Part 1: The Colour Problem
Have a look at the two images below. The one to the left has been “HDR-ed” of course. The one to the right is one of the three original exposures. Notice the red roses to the left. In the HDR image they are not red, but luminescent crimson.
Fig. 1. St Katariina Church, Turku Finland
Photomatix exaggerates red. But red is not the only colour that Photomatix is particularly fond of. Yellow is also a favourite. And it is not only a question of simple saturation. To get rid of the red cast you have to reduce saturation to a point where the rest of the image is verging on b&w.
But does not this problem sound familiar? It is exactly the same as Adobe ACR, which exaggerates red for Canon, Nikon and Sony alike. The remedy was to calibrate our cameras. Or rather, calibrate ACR and Lightroom to interpret the camera correctly.
But can we calibrate Photomatix? Yes, we can.
The calibration process here can be applied to any HDR program. The trick is to do exactly the same that we did with our cameras. Use a standardized colour chart, GretagMacbeth or QPcard 201. I have used the latter. Now we do not need the physical card itself, but we need a digital version with all the correct colour values. These values are readily available on the internet, and using Qpcard 201, you even get one “for free” at the end of the calibration process. It is used as a target at the final visual comparison.
- I created one such card and saved it as DNG. See image below.
- Then I saved five TIFF versions of it from Photoshop, each 1 EV darker than the previous. Ithen ended up with “exposures” -4/-3/-2/-1/0.
- I tried several variations, but found -4/-2/0 to be closest to actual shooting. Note that this combination has -2 as the middle and presumably “normal” exposure.
- Then I simply ran this triplet through Photomatix.
- I converted the TIFF output to DNG in Lightroom and calibrated that file as if it were from a camera.
- Then I tried another setting and calibrated that one. And another. And another. Hmmm….
Long story short: Some settings in Photomatix produce consistent colour shifts and can be calibrated. Other settings produce dynamic colour shifts depending on the surrounding pixels. No surprise, Detail Enhancer can be one of these. To identify those settings that behave predictably we need to create two versions of the reference chart, namely one with black and one with white background.
Fig. 2. Dual QPcard 201
The colour blotches are identical, but since half of them are surrounded by white and the other half by black, those Photomatix processes that treat colours dynamically (depending on pixel surroundings) will produce different colours for each identical pair.
After that it is only “guess and try”. A simple verification procedure is to position the white half on top of the black half in Photoshop and set the top Layer to Blend mode Difference. That should turn all the colour blotches to black if they are identical in the two layers. Exposure Fusion: Average is one obvious candidate for consistent colours, and yes, it worked.
Unfortunately the images were too dark. Something brighter is needed. Several algorithms produced consistent colours. A good one with bright colours was Tone Compressor with the following settings.
Fig. 3. Tone Compressor settings and profile for consistent colour rendition
Yes, the settings are all zero. You may choose any setting that works for you. This worked for me, andif you use my settings it will also work for you, since there is no sensor or production varianceinvolved. The reason for the –1.70 Exposure setting is that the “reference card” is actually the –2EV version.
Now this may sound theoretical and perhaps slightly boring. But it will come to good use in the nextpart, which presents us with an actual, usable workflow. I use it for my own pictures now.
Part 2: The Workflow
I need a workflow. I cannot afford to spend hours on each individual image. What I really want is the “HDR Slider” that will let me create an HDR image with more or less HDR effect, preferably less. Well here it is. It sits inside Photoshop and we all know it.
Fig. 4. The “HDR Slider”
The reason I spent so much effort on arriving at consistent colours is of course this slider. If I letPhotomatix generate two files at the same time, then joining them together in Photoshop, I have what I wanted.
- File one: Consistent colour, for instance Tone compressor with all settings at zero.
- File two: Wild and wacky HDR, or soft and blurry, whatever you fancy. I use Detail Enhancerwith some extreme settings.
- File one goes into the Background layer.
- File two goes into the next layer, with a Blend mode of Luminosity, and the Opacity slider regulating the effect.
“… there is one important trick that I should mention here: you should never try to obtain the final look of an image in the tone mapping phase, and instead just focus on bringing all the details back from shadows and highlights. You should aim for a low contrast (especially localcontrast), low saturation image at this stage, and only afterward, in Photoshop and with local curves adjustments, bring back the full glory of the scene.”
And that is exactly what this workflow produces: good quality raw material for traditional postprocessing, as quoted above.
Fig. 5. The processing path
While working on this article I quickly discovered that even testing was a chore without some computer assistance. So I wrote two AppleScripts. The first renames the output from Photomatix to a strict naming standard. The second script runs Photoshop in batch, merging the Colour and the Luminance files, producing the TIFF files that can be processed manually at the end.
And to my Windows brethren: These scripts are so simple that I am sure half of you can whip them up in VBS in an hour. The difficult part is to get Photoshop to respond like you want it to. For instance, I found no way to make Photoshop flatten the image as part of the Save operation.
The individual Steps of the Workflow
Here we’ll have a look at each step from start to finish. What works, what doesn’t and why.
All “the other” tutorials go on and on about shooting and bracketing. You can read them. Again, I’m only after what concerns my workflow. Shooting HDR outdoors is usually ok
Outdoors you can often shoot handheld. Indoors you should handheld.
The portal shot at the beginning was shot handheld. We usually consider a tripod.all have cameras with image stabilization now in 2011, and the software successfully aligns the images. My camera (Sony α700) can bracket -2/0/+2, which is enough for most “normal” scenes.
Indoors three exposures are not enough, and I shoot two bracketed series after each other. One is adjusted an additional -3EV, the other +3EV, resulting in a 6-exposure series of -5/-3/-1/+1/+3/+5. Fortunately the sequence does not matter to Photomatix, so the actual sequence may be for instance +3/+1/+5/-3/-5/-1. Since I have to pause between the two series and adjust the camera, it is obvious that a tripod is needed. The other obvious reason for a tripod is that exposure times easily exceed 30s.
You may prefer to always shoot -3/+3 in that sequence or alternate between the over/underexposure settings. Both have advantages. The advantage of alternating is that it is quicker. The next series can retain the settings of the last bracket of the previous series. The advantage of always following the same routine is just that (don’t underestimate the value of routine), plus another we will see in Lightroom later.
Be aware of the camera’s longest exposure time.
Talking about exposure times, you may end up in a situation whereyour camera will sabotage your bracketing attempts. My camera has a longest shutter time of 30s, meaning that if the “0” exposure in the bracketing sequence exceeds 8s then the “+2” exposure (8x2x2 = 32,which is rounded to 30) will not be +2, but perhaps reduced to +1.7 or less. If the “0” exposure is actually 30s then the sequence will be -2/0/0. In cases like this, manual exposure might be called for. Do some test runs with your camera in low light and learn how itbehaves.
If the exposures are not evenly spaced and/or some of them are identical, Photomatix will be confused. However, the result may not be as bad as one might think. As an experiment I took one sequence of 6 exposures and corrupted it afterwards, making two copies of the +3 exposure and two copies of the -3 exposure. I then created two different HDR images with Exposure Fusion: Adjust Default, both based on 8 exposures:
-5/-3/-3/-3/-1/+1/+3/+5 (“dark”) and -5/-3/-1/+1/+3/+3/+3/+5 (“light”)
The results were different. The histogram shows the difference between the two when compared in Photoshop. This also means that if the camera limits your longest exposure this will influence the result.
It is best to use the same number of exposures for all series during one shoot. This is something that you will be grateful for when you return to Lightroom, which I’ll explain later.
Within one shoot, always shoot the same number of exposures.
Do NOT think that this is a scene that requires fewer exposures. Instead do everything in the same way. If you canorganize the images into logical groups, for instance Inside and Outside, you may consider using 6 exposures in one group and 3 in the other. But try to avoid mixing!
Finally, having all series with an identical number of exposures makes Photomatix batch processingpainless!
The only important thing about importing to Lightroom is your file naming scheme. Use filenames that create a sortable sequence.
Use filenames that sort well.
You may either use the image names straight out of the camera, orsome datetime format. I recommend ExifRenamer on the Mac, and there are similar solutions for Windows. My naming standard is PYYYYMMDD_hhmmss_sss, wheresss is a sequence number within the same second, always starting at 001. With a camera that shoots 5fps that number can be from 001 to 005 of course. Lightroom supports generation of a similar filenameexcept for the _sss part.
The reason for a sortable sequence is of course that files from the same series will appear in sequence both within the Lightroom Library browser and the OS file system.
Lightroom Library Organization
Now here’s the reason why I recommend you stick to the same number of exposures within the sameshoot or subject. It is how you view them in Lightroom!
Fig. 7. 6-exposure series as seen in Lightroom
With an identical number of exposures for all series, you can adjust the viewing in Grid so that oneseries will correspond to one row in the Grid. We also see that for the first three series I shot the +3/-3pattern, but then started alternating. It does not matter to Photomatix, but it might definitely matter to you. As we can see, the second exposure from the left is almost correct for the first three, and can be used as is. It is always useful to have a “real” reference shot, and following the routine instead ofalternating gives us these reference shots nicely lined up vertically.
The other Organization advice is to assign separate folders to series with different number ofexposures. Bracketing with 3 goes into the _3 folder, while bracketing with 6 goes into the _6 folder,etc. Viewing becomes so much easier.
Series that do not conform to either go into a separate folder.
It should also be obvious that if you delete a file you will upset the pattern in Grid view.
If you google like I did, you will find statements to the effect that TIFFdoes only support sRGB. Don’t believe everything you find on the ‘net.
TIFF supports the usual colour profiles.
TIFF supports both Adobe RGB and ProPhoto RGB just fine. This is what HDRsoft claims, and this is also what I could confirm with a simpletest using Lightroom and Photoshop.
(Curious about the comparison test? I took one particular image with green grass and red flowers andpushed the saturation mile high. I then exported that image twice, once as sRGB and once as ProPhotoRGB. I then opened them in Photoshop, placed them in layers above each other and set the to layer’s Blend Mode to Difference.)
The other thing about TIFF is what the HDRsoft FAQ for Photomatix says. (I have edited the textslightly.)
Do I need to process bracketed RAW files to TIFF format first?
The answer depends on your needs and preferences. If the quality of rawdemosaicing/debayering or proper white balance adjustments are important to you, it is better to first process your bracketed RAW files in your favorite RAW converter, and then combinethe converted TIFF files in Photomatix.
The RAW conversion of Photomatix Pro and Photomatix Light is not as full-fledged and sophisticated as the RAW conversion done in Photoshop, Lightroom, Aperture, Bibble,DxO or other applications specialized in RAW conversion.
In general then, do your normal processing in Lightroom, then export to TIFF.
For each run of Photomatix, you use one set of TIFFs from Lightroom of course, each set with thesame number of exposures. So for my 12 final images, I started with 72 TIFFs.
Photomatrix can create three files at the same time.
The Batch processing of Photomatix is quite powerful. From top left corner and down:
I choose to save a version of the merged file. By doing that I cango back and try different tone mappings “at no cost”. I also askPhotomatix to create both a Tone map/Details Enhancer file and a Tone map/Tone Compressor file atthe same time, using the same 6 base images.
For Settings, I set Strength, etc to 10 to “get the most out of it”. (Those of you who have purchased the Photomatix Spinal Tap Edition may try to set everything at 11…). Your mileage may vary.
Click Run and go do something else.
Fig. 8. Photomatix options for Batch Processing
After Photomatix has finished there will be triplets of almost identically named files in the ouput folder. The 32-bit EXR file will have no suffix, while the two other ones have one as you can see.
Fig. 9. Photomatix files after Batch Processing
This is how Photoshop is used.
- Open the Enhancer.tif file in Photoshop.
- Open the corresponding Compressor.tif file.
- Holding down the shift key, drag the Background layer from the Compressor file onto the editing (main) window of the Enhancer file. This copies the layer and ensures that it is centred.
- Set Blend Mode to Luminosity.
- Adjust Opacity.
- Save as TIFF.
I have written two scripts. The first script renames the output files from Photomatix to something manageable.
The second script runs Photoshop in batch, doing exactly what was explained above, except it can produce several variations of the same picture. It has one prompt where you can enter whatever Opacity values you want.
After you have entered the values, the script will process all files dropped on it with these same settings.
And Finally - Lightroom
Lightroom can easily post-process HDR files.
You may not be entirely satisfied with the outcome. You are noteven supposed to be! Re-import the final Photoshop image into Lightroom and make whatever changes you might fancy. Remember that these images are really 16-bit, not 12 or 14 like the ones from our cameras.
The following two scenes have been processed “on automatic” and present three variations to choose from, 25%, 50% and 75%. I tend to prefer 50%, but not always. The images are from St Katariina Church. The images may seem slightly bland, which is just what I want. Lightroom takes care of therest. The differences are more clearly visible at normal screen sizes.