An essay by Charles S. Johnson, Jr.

Reality is merely an illusion, albeit a very persistent one.  – Albert Einstein

I don’t mean that the real world doesn’t exist with all its objects, scattered and absorbed radiation, etc., but rather that we only “see” a very small part of what is in front of us, and that part is heavily filtered, selectively enhanced, interpreted, and corrected.  In short we experience a colorized illusion, or better a simulation of the world that is ”visible” from our perspective.  And that simulation is about 100ms out of date by the time we experience it, except for selective parts that have been extrapolated in time to predict the “now.”  This and every other comparison of “reality” with perception is controversial. 1   However, useful predictions or “future seeing” based on experience should come as no surprise.  Evolution works to increase our survivability rather than to accurately record all aspects of our environment.  Our memories are designed the same way.  At best they provide instructions for getting along in the world.  “Our repository of memories is less like a library and more like Wikipedia, where each entry is open to editing anytime it’s pulled up.” 2

In the following sections I will briefly explore the operation of our visual system and then approach the question of where that leaves photography.  In other words, what is left for photography and what can photography achieve?   I believe there are two paths which are not exclusive.  First there is the quest for the ultimate recreation of reality.  The result would be a spherical, 3D, panorama of the world that would be instantaneously corrected and updated for our direction of vision and point of focus.  This imitation of our real world stimuli could perhaps be experienced with the help of virtual reality goggles or their future counterparts.  In this path there would be nothing left for the artist or the hobbyist.  I call the other path the artistic pathway.  In this path there would be self-imposed restraints as in traditional art.  The artist does not attempt to capture the entire world on one canvas, and sculptors limit themselves to one or at most a few figures.  Before I continue along this line we need a few details.

Vision:  The eyes and brain permit the simulation of the real world by the mind.  The simulation is not designed by evolution to show realism, but rather to enhance survivability.  The primary visual cortex is not a correlate of consciousness.  The response of cells in our retinas and our visual cortex have some correlation with our visual experience some of the time, but not always.  In other words, we do not see with our eyes!  Christof Koch in his insightful book, Consciousness, has described three ways we know this to be true. 3   First, the image on the primary visual cortex is jumping every second or so as our eyes dart from point to point (saccades) (Fig. 1).

Figure 1:  Eye movement when examining a figure with instructions (Yarbus, 1967).

At this stage vision would resemble the output of a video camera that is quickly changing direction.  Second, each of our eyes has a significant blind spot in the place where the optic nerve leaves the retina.  We are normally not conscious of this omission because the brain fills in missing parts of the image by experienced guesswork.  And finally, we see colored images with great reality in some dreams even though our eyes are closed.  Processing of the stimuli from the visual cortex occurs in three dozen different regions of the brain by modules that detect edges, vertical, horizontal, and on a slant.  Motion is detected, and so are spatial frequencies.  In addition color is assigned to patches in each image to insure that we distinguish things necessary for survival in a consistent way.  The colors assigned to each patch, e.g. a leaf, exhibit constancy even when the light scattered from that patch has a different wavelength.  Luminance is also manipulated in unexpected ways (Fig. 2).

Figure 2:  Contrary to appearance the squares marked A and B are the same shade of gray.©1995,Edward H. Adelson

This processing has built-in priorities and capabilities such as the learned ability to recognize human faces in the proper orientation and for interpreting scenes with the assumption that light is incident from above (Fig. 3).  It is estimated that all of this processing requires about 100ms before visual information can be sent to the prefrontal cortex and thus appear in our conscious view of the world.

Figure 3:  Sketch of the human brain.  The optic nerves, lateral geniculate nucleus, and connections to the visual cortex are not shown.

Visual information reaches the unconscious mind somewhat earlier, and there is editing in the short term memory and the long term memories.  In addition to this direct picture, we must realize that everything is not strictly linear in time.  Apparently the mind corrects when possible for processing delays by extrapolation so that we can react to instantaneous necessities such as dodging a thrown rock Fig. 4).  Occasionally, the correction is wrong, and the memory must be rewritten.  All of this has been verified by experiment.   Numerous examples of illusions in space and time can be found online at illusionoftheyear.com.  By the way, limitations of vision, visual illusions, and timing corrections are commonly used by magicians to perform their “magic.”   Some of this is described in the delightful book, Slights of Mind, on neuromagic by Macknik and Martinez-Conde. 4

Figure 4:  (a) The ball rolls to the left and the lamp flashes when the ball is directly below it. 
(b)  An observer is aware of this flash 100 ms later and reports that the ball is well to the left of the lamp when the flash occurs. 
We conclude that the mind projects the ball where it is expected to be when it is actually seen.  This is an illusion.

So what does all this have to say about our reaction to and our appreciation of art?  This brings us to recent work on the responses of the conscious and unconscious mind to works of art.  The most complete and up to date account of the status of this work can be found in Eric Kandel’s recent book, The Age of Insight. 5   Art evokes emotions and stimulates the pleasure centers in the brain.  It takes advantage of the brain’s ability to recognize objects and people from sketchy outlines, to respond to exaggerated features in drawings of faces and anatomy, and to respond to colors.  We enjoy the sights and sounds of nature as well as urban scenes and the appearances of some people without conscious thought.  And, of course, we enjoy some forms of art.  Scenes on canvas and in photographs often give pleasure.  Why art and other forms of imaginative fiction give pleasure is controversial.  Kandel suggests that feelings of well-being may arise because art predicts biological rewards, 5 while Gazzaniga thinks that fiction and art help organize the brain, a demanding adaptive task in human development. 6

Photography:  If vision ultimately provides a simulation of reality and not reality itself, what can we say about photography?  Photography provides an imitation of the visual stimuli we experience, but in a very restricted way.   Photographic images are limited by most or all of the following:

1. Two dimensional presentation .  Most of our sense of depth of field in vision results from stereopsis associated with binocularity rather than aperture related depth of field.  Just close one eye and notice the apparent increase in the depth of field.  After all, the focal length of the eye is only about 20mm or close to that in the wide angle lenses on our DSLR’s.

2.  A fixed point of focus .  The focus is fine for objects in one plane and deteriorates as a function of distance for objects not in the focal plane.  The depth of field can be limited by increasing the size of the aperture in order to emphasize some features in an image, but this is poor substitute for stereopsis.

3.  Framed or boxed images .  In the real world our eyes can look in unlimited directions (front facing).  The frame limits our view and focuses our attention for better or worse.

4.  A fixed dynamic range that is limited both by a constant aperture for all parts of an image and by fixed sensitivity of the sensor.  In contrast our visual apertures (pupils) adjust quickly, and our visual pigments adapt over time to different levels of light.

5.  An inaccurate representation of color resulting from simulations using three primary colors or the extreme case of black and white presentations.  It should be acknowledged that no set of three monochromatic colors can match all the colors experienced by the mind in combination with our set of visual receptors (cones).

6.  A fixed presentation that does not respond to our changing point of attention .  In the real world our eyes dart to a point of interest and that point becomes perfectly focused.  Also the aperture adjusts over a wide range to give proper brightness in the selected area.

Can these limitations be addressed and even corrected by technology?  In most cases, yes.  I consider the corrections in turn:

1.  We can capture sets of images from different perspectives in order to simulate 3D vision.  This will not completely capture our visual experience because it does not require lens accommodation and change in relative orientation of the eyes that occurs when objects move relative to our location.

2.  New plenoptic cameras, for example the Lytro, can capture the light field rather than a set of intensities in a plane.  The light field contains information about the direction of the rays reaching each point on the image, and this information can be processed after the exposure to adjust the point of focus to any desired plane.

3.  A spherical panorama can be recorded so that the frame can shift to encompass any direction of view.  This technique is well known and simply requires the collection of overlapping images that cover all directions in space.  A 3D version of the spherical panorama is trickier.

4.  A collection of images taken with different exposures can encompass the full dynamic range of an image.  The images can then be combined to obtain the correct exposure for any point.

5.  In the case of color photography we are close to matching all perceived colors, at lease in principle, by the use of imaginary color primaries that define a color gamut covering most of our visual space.  The imaginaries are simply colors that can be represented in a computer but cannot be created in nature.  A more serious problem is our present inability to display the complete or near complete visual gamut on a screen or in a print.  Of course, the dynamic range of nature will always exceed our displays.

6.  Instruments can accurately determine our point of focus so that, at least in principle, the focus and brightness can be instantaneously corrected by the methods described above.

Suppose that our goal is to stimulate our retinas in a way that closely emulates the stimuli we would experience in nature.  In the future all of the corrections described above can be implemented with the aid of improved sensors, more powerful computers, and improved displays.  From my early 21 st century perspective, I envision state-of-the-art virtual reality goggles that respond to the orientation of the head and the eyes.  The resulting world view could be close to that presented to the camera on a Mars rover, an observer sitting on the edge of the Grand Canyon, or a camera hanging over Dettifoss in Iceland.  This would be is impressive and probably enjoyable; but it would not be art, and there would not be much left for a hobbyist. 

But I do not that believe that future virtual reality presentations will diminish art or artistic photography.  It has been said that art is what artists do with limitations.  I know there are many quotations about exceeding your limitations and admitting no limitations, but history and successful artists, musicians, and writers teach us a different lesson.  For example:

”No matter how gifted a painter, his work is defined by his limitations.” Alan Feltus

“If you have five elements available use only four.  If you have four elements use three.”Pablo Picasso

“What is not constrained is not creative.”Philip Johnson-Laird 

It is interesting to consider the reaction of artists when photography appeared on the scene and more and more co-opted the presentation of “realism” in scenes.  Artists had discovered perspective and had learned how to work with light mixing as well as pigment mixing.  Then photography seemed to have it all for the future.  Rather than being depressed, Henri Matisse said, “The invention of photography released painting from the need to copy nature.”  He saw this as an invitation to appeal directly to the emotions.  Gustav Klimt responded in 1907 by throwing out conventional perspective and painting the glorious portrait of Adel Bloch-Bauer with beautiful decoration and a flattened space (Fig. 5).  In 2006 that painting sold for $135 million.

Figure 5:  Gustav Klimt’s portrait of Adele Bloch-Bauer with enlarged inset of detail in the dress.

  In the near future photographers can look forward to cameras with improved resolution, greater dynamic range, and better color rendition.  There will continue to be advances in the feature sets including enhanced 3D capture, automatic panoramas, high dynamic range capture and processing, time-lapse, wireless, GPS, and so on.  So one can go in many directions with photography, but in order to create art that will last, one would be well advised to find a niche and to concentrate on mastering that limited area.  That might mean working only with a fisheye lens or with long time exposures.  Or who knows, maybe some yet unexplored light field time-lapse technique.  So many ways to focus attention coupled with unlimited possibilities for combination and manipulation of photographic images erase any boundaries remaining between art and photography.

Is there a take home message in these ramblings?  I think there are two.  First, the serious photographic artist should not fear what automated cameras with extensive feature sets will do to photographic art, and second we should be open to any kind of photographic processing to produce images with impact.  Images should be judged on their own as art, and the methods employed should be disclosed only at the photographer’s discretion.  This follows from our realization that we always “see” the world through heavily manipulated images based on assigned colors, enhanced edges, interpreted shapes, extensive guess work, and learned associations.

© 2012 Charles S. Johnson, Jr .
December, 2012 

Acknowledgement:  I thank Sönke Johnsen for comments and suggestions.

Biography

Charles S. Johnson, Jr., is a scientist and educator. He received a Ph.D. degree from M.I.T. in Physical Chemistry (1961); and he has taught at the University of Illinois at Urbana, Yale University, and The University of North Carolina at Chapel Hill where he held the title of Smith Professor of Chemistry. His bibliography contains approximately 150 research papers including review articles on magnetic resonance as well as books on quantum mechanics, laser light scattering, and photography. His research was recognized by fellowships from the Alfred P. Sloan Foundation and the John Simon Guggenheim Foundation. In addition he was elected to the rank of Fellow in the American Physical Society and the American Association for the Advancement of Science.

He continues his professional activities from his office in the Chemistry Department at UNC and through his association with Invitrox, Inc., a biomedical company in RTP. However, he is now an avid nature and travel photographer. He devotes much of his time to reading and writing about the scientific and technological aspects of photography. His book, “Science for the Curious Photographer,” is an example. Also, he maintains a photographic blog and he publishes articles in photographic newsletters. In the future he plans to write review articles and possibly a book for photographers on various aspects of science of the mind, creative photography, and computational photography.  

Links to “Science for the Curious Photographer”

Publisher:  http://www.crcpress.com/product/isbn/9781568815817

Vendors: http://www.amazon.com/Science-Curious-Photographer-Introduction-Photography/dp/1568815816

http://www.barnesandnoble.com/w/science-for-the-curious-photographer-jr-johnson/1111956498

Corrections and updates:  photophys.com

References: 1.  Dale Purves and R. Beau Lotto (2011), Why We See What We Do Redux, A Wholly Empirical Theory of Vision, Sunderland, MA: Sinauer Assoc., Inc.

2.  Greg Miller (2012), “How memories are retrieved?” Science,338, 30-31.

3.  Christof Koch (2012), Consciousness, Confessions of a Romantic Reductionist, Cambridge, MA: MIT Press.

4.  Stephen L. Macknik, Susana Martinez-Conde with Sandra Blakeslee (2010), Slights of Mind, What the Neuroscience of Magic Reveals about Our Everyday Deceptions, New York: Picador.

5.  Eric R. Kandel (2012), The Age of Insight, New York: Random House, Inc.

6.  M. S. Gazzaniga (2008), Human, New York: HarperCollins Books. p 221.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.