Ad
Ad
Ad
Pages: « 1 ... 4 5 [6] 7 »   Bottom of Page
Print
Author Topic: Should Camera Companies adopt non-proprietary raw?  (Read 11728 times)
Schewe
Sr. Member
****
Offline Offline

Posts: 5415


WWW
« Reply #100 on: April 23, 2013, 12:44:26 PM »
ReplyReply

So far, it hasn't been made very clear why DNG would not be a good idea.

Yep...it isn't clear why DNG would not be a good idea because, well, DNG is a good idea thought up by one of the bright minds of digital imaging–Thomas Knoll who basically jump started this whole industry by writing Photoshop, then Camera Raw and DNG. Thomas is kinda an expert on these matters, having decoded over 300 different cameras raw file formats.
Logged
jrsforums
Sr. Member
****
Offline Offline

Posts: 722


« Reply #101 on: April 23, 2013, 02:03:05 PM »
ReplyReply

Yep...it isn't clear why DNG would not be a good idea because, well, DNG is a good idea thought up by one of the bright minds of digital imaging–Thomas Knoll who basically jump started this whole industry by writing Photoshop, then Camera Raw and DNG. Thomas is kinda an expert on these matters, having decoded over 300 different cameras raw file formats.

Why do you keep repeating points that no one has an issue with?

DNG could/would be good...for many.  Is it critical?  Is it best for the manufacturers?  who covers the cost for some to implement, that which is a cost savings for others?  How does it really effect the average photographer vs. status quo?  Those are the points we have been discussing.

Also, no one questions Thomas Knoll's knowledge or contributions to this industry.  

If nothing else, DNG seems to build internal efficiencies in ACR/LR processing, currently and with LR Smart previews in LR5.  The good news is that this is under the covers, not requiring any end user to implement or convert, which might cause users to seek out a different solution.
« Last Edit: April 23, 2013, 02:14:07 PM by jrsforums » Logged

John
peterv
Jr. Member
**
Offline Offline

Posts: 60


« Reply #102 on: April 23, 2013, 03:09:26 PM »
ReplyReply

who covers the cost for some to implement, that which is a cost savings for others?

I think this is the second time you bring up the aspect of the costs of reverse engineering the proprietary raw files. In one of their videos Michael and Jeff talk about the fact that the ACR team is actually quite small. I can't believe Adobe makes a case for a universal DNG in order to save on the salaries of a few good man.

The same goes for the camera makers. The costs of the man-hours for making DNG available are not relevant to the total costs of marketing a new model.
Logged
fredjeang2
Sr. Member
****
Offline Offline

Posts: 786



« Reply #103 on: April 23, 2013, 03:39:51 PM »
ReplyReply

It's very possible that softwares like Lightroom, not in a so long future (could be very short), are going to be a raw dev plus a file manager for both still and motion imagery.
DNG is the only Raw possibility to date that can cover the needs to come in imagery very very soon. (that are already there but still in infancy)

They will have to do it yes or yes. The only question is when.
« Last Edit: April 23, 2013, 03:50:10 PM by fredjeang2 » Logged
Schewe
Sr. Member
****
Offline Offline

Posts: 5415


WWW
« Reply #104 on: April 23, 2013, 03:46:50 PM »
ReplyReply

The same goes for the camera makers. The costs of the man-hours for making DNG available are not relevant to the total costs of marketing a new model.

Yep, the issue is not related to costs, nor to technical limitations of standardizations, not really–any short term technical issues can be dealt with and the cost of adding a standard raw file format would be a drop in the bucket for the development of the chips and cameras.

It's all about the politics...Nikon and Canon simply don't want to adopt any standards at this time. However, I hope that over time and with grassroots support, one of the two main players might blink and either adopt a standard format or make a standardized raw file format an option like JPEG already is–in fact, the Lossy DNG is a potential replacement for JPEG that would offer the potential file savings of JPEG while making editing of the scene referred image data far more flexible.

Once that happens, it will be a tipping point for the industry. And while they are a niche "rich man's camera" one very successful camera maker is doing just that. Leica has successfully adopted DNG as their native raw file format. And Leica is doing great. They literally can sell avery camera/lens they make and have backorders for lots of their stuff. They enjoy the benefits of DNG which includes automatic support for new cameras on current and older software platforms as well as pushing the image quality as far as possible by including lens profiles to address aberrations. Leica is a shining example of how DNG can benefit a camera maker. So, if Leica can afford to adopt DNG (since they are such a small company) one presumes there should be no economic roadblock to Nikon or Canon doing so. No, it ain't about the money...
Logged
davidgp
Jr. Member
**
Offline Offline

Posts: 65



WWW
« Reply #105 on: April 23, 2013, 04:55:27 PM »
ReplyReply

So, I have been reading this topic, maybe a bit heated, but I think I got some pieces of information about how Adobe or other software companies works... nice information to know (thanks!!!).

My two cents. My background is more of a software programer than a photographer, basically an amateur photographer that earns its living working 8 hours a day designing a programming software (no Image processing software of any short to leave that clear).

As a programmer, for me that the camera companies adopt or not DNG is not a matter of cost, for them it must be less of 1% of the work they employ to write the software, and as difficult, again for them, than to add the new RAW format the camera engeniers decide to adopt for the new Canon 1D Marl XX or the new Nikon D900e, I mean, they just get a full specification of a file that need to read, translate to the internal format the program uses (like Adobe does to DNG), and the rest of the code of their RAW software its the same. They don't write an entire pipeline of RAW processing for each new RAW format their company produces. Well, at least in my mind, that will be a very inefficient way to work.

I'm more of the opinion that this is a political reason as Jeff Schewe says than a software problem. The real cost is to convert the data coming from the sensor into RAW data, and that happens in the camera, then if the camera stores it in DNG, NEF, or whatever... it is just a very little piece of the puzzle.

Other topic that I read in this thread and in the previous one in the Lightroom sub-forum is the argument that the camera vendors give their software for free. Really? I thought I paid for that when I bought the camera, in the price of the camera you are not paying only for the little pieces of metal, plastic and so on... and the people that worked in the factory building it. You are paying for the design of the camera, paying for the people that wrote the warranty information, paying for the people that wrote the manual of the camera, and paying for the software that comes with it, everything adds... after all, they are giving you software that only works with their cameras, it is not useful for anybody RAW files.

(well, there are some exceptions, some camera makers from time to time add software that can work with other cameras, if memory does not fail, I think Leica used to bundle a copy of Lightroom with some of its cameras, not sure if Phase One gives you a copy of Capture One with their backs (they are over my pay-rate so I don't look to much to them)... but anyway, the price of that software it is included in the Leica camera or in the Phase One back...).
« Last Edit: April 24, 2013, 01:45:32 AM by davidgp » Logged

Schewe
Sr. Member
****
Offline Offline

Posts: 5415


WWW
« Reply #106 on: April 23, 2013, 05:59:43 PM »
ReplyReply

(well, there are some exceptions, some camera makers from time to time add software that can work with other cameras, if memory does not file, I think Leica used to bundle a copy of Lightroom with some of its cameras, not sure if Phase One gives you a copy of Capture One with their backs (they are over my pay-rate so I don't look to much to them)... but anyway, the price of that software it is included in the Leica camera or in the Phase One back...).

Yes, when you buy that expensive Leica, you get a copy of the then shipping Lightroom (upgrades are the responsibility of the user). And Phase One (and Hasselbad) "gives you" the software to drive the camera and process raw images from the back. In the case of Capture One, you get the digital back (DB) version of Capture One, not Capture One Pro which also processes other supported cameras like DSLRs. Not sure about Hasselbad's Focus software, but I know their software will process any raw capture from their backs.
Logged
Tony Jay
Sr. Member
****
Offline Offline

Posts: 2044


« Reply #107 on: April 23, 2013, 11:02:15 PM »
ReplyReply

Yep, the issue is not related to costs, nor to technical limitations of standardizations, not really– any short term technical issues can be dealt with and the cost of adding a standard raw file format would be a drop in the bucket for the development of the chips and cameras.

It's all about the politics...Nikon and Canon simply don't want to adopt any standards at this time. However, I hope that over time and with grassroots support, one of the two main players might blink and either adopt a standard format or make a standardized raw file format an option like JPEG already is–in fact, the Lossy DNG is a potential replacement for JPEG that would offer the potential file savings of JPEG while making editing of the scene referred image data far more flexible.

Once that happens, it will be a tipping point for the industry. And while they are a niche "rich man's camera" one very successful camera maker is doing just that. Leica has successfully adopted DNG as their native raw file format. And Leica is doing great. They literally can sell avery camera/lens they make and have backorders for lots of their stuff. They enjoy the benefits of DNG which includes automatic support for new cameras on current and older software platforms as well as pushing the image quality as far as possible by including lens profiles to address aberrations. Leica is a shining example of how DNG can benefit a camera maker. So, if Leica can afford to adopt DNG (since they are such a small company) one presumes there should be no economic roadblock to Nikon or Canon doing so. No, it ain't about the money...

This is true.
A simple reading of the DNG specification, last updated in December 2012, will rebut many of the fallacies currently been pushed.

Claiming that adopting DNG, or perhaps some other similar format, as a universal RAW standard somehow 'limits' technical advances on the hand of the camera-makers simply does not stack up.
As a principle it is a bit like claiming that the CD or Blue-ray disc that one buys limits the creativity of musicians, mixing specialists and music producers in the studio.
No-one even thinks about that as an issue per se.
Is it possible to change how that music sounds when we listen to it - sure.
Both different players (and the technicalities of the amplifiers and speakers may make adifference) as well your tastes with regard to treble and bass etc.
In principle, a/d conversion of light in the camera and the subsequent number crunching is what happens in the 'studio'.
The finished product on the CD or Blue-ray disc is the RAW file written to a particular format.
Finally, the player and the sound we eventually hear is analogous to the combination of our RAW conversion and our tastes.

Truly understanding what a RAW file format actually does as well as the TRUE similarities as well as differences between proprietary RAW formats and DNG will immediately squash any claims that DNG stifles the creativity of camera-makes in order to give us better cameras (read sensors and processors in the immediate context).
The DNG specification is published and freely available from Adobe's website.
It is an interesting read.
My reading of the specification, anyway, most emphatically confirms that DNG has space in the metadata component for proprietary information should it be required. (At least one post tried to take me to task on that one.) The key is that propurietary information is not placed in other areas of metadata not designed for the purpose (a bad habit especially prevalent in, now, more dated proprietary formats).
The entire concept of DNG is designed to be extensible, not only in volume, but also in technology.
Also, if one is prepared to acknowledge this, the very fact that the specification has been updated several times to take into consideration potential, or actual, technological advances, shows that not only the attitude of the designers, but the actual format design - it is there to be changed and updated as required.

Additionally, Adobe has always, as far as their published intent goes anyway, expressed a desire to make sure that the format be kept current, but in reality the plan has always been for the International Standards Organisation to take on the format, in which case the prime responsibility would not be Adobe's even if their involvement did persist.

There are no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file.
In fact most proprietary RAW formats follow the fundamental outline of the DNG design (which uses the TIFF-EP specification as its basis).
Most proprietary RAW formats have also adopted TIFF-EP as the basis of the format - and this appears to have only occurred since the release of the first iteration of the DNG specification.

Mentioning the fact that DNG has already been adopted by several manufacturers has stimulated comments along the line that images files from these models are crap and the implication is that DNG somehow is the limiting factor.
I was pleased that Jeff mentioned Leica.
Are there any Leica users out there using models where DNG is the RAW format generated who feel that the image files are crap, and are they prepared to finger DNG as the culprit, and what is the evidence then that DNG might be the problem?
Lets see if anyone is prepared to stick their hand up on this one.
Somebody mentioned that at least one Pentax model using DNG has crap RAW files. Maybe you are correct in your assessment of the file quality. What evidence do you have that DNG is to blame?

Currently, RAW file conversion, includes a demosaicing process, omongst others, that requires information about the sensor and A/D conversion. The demosaicing process is there to give us the colour that we see, sensel for sensel, that is not present in the actual recorded information, sensel for sensel, since sensels only record grey tones.
Most sensors use a Bayer array placed over the sensor the filters incoming light using three colour filters - red, green, and blue.
Generally the pattern is a two-by-two repeating pattern with two green and a single red and a single blue filter.
The demosaicing process is an estimation process of what the actual colour, sensel for sensel, that each sensel should be based on how much light has struck that sensel and the surrounding sensels with due regard for the colour filter governing light transmission for each sensel.
So, the decisions made by the RAW converter are governed by what is known about the sensor and some of the A/D processing.
There are two ways this information can be obtained.
Either the information is provided by the manufacturer/designer or the information is obtained by reverse engineering.
Almost all the required information, for the third-party RAW converters, has, historically anyway, been got via reverse engineering.
I have some recent information that I have no personal way of verifying that Adobe may currently be getting their data from the camera-makers directly. Apparently this courtesy has not been extended to other software companies. As I have said I cannot verify this personally.
However, part of the design of the DNG format is to make possible a camera-model independent RAW processing experience where should any camera model use DNG as its RAW format, that no RAW converter would need the information currently required to accurately demosaic the RAW file as well as make other decisions. The process does not risk any proprietary secrets.

Some, but not all of the information in this post, is from the latest DNG specification that I accessed straight off Adobe's website. All the other information has been published on sites well known to the photographic community, and last, but not least LuLa itself.

There has been a hugh amount of heat generated based on complete furfies.
Perhaps you may not want, or feel the need, for a universal RAW format.
You may not want DNG adopted.
These are valid positions per se.
What is not so tenable though is the 'don't confuse me with facts approach' that has been evident.

My advice is NOT to take at face value anything written above - go and research this stuff yourself.
Draw your own conclusions based on knowledge, not ignorance.

BTW Any RAW converting process based on DCRAW has limitations based on the lack of documented format information. All the conversion information required is reverse engineered, ingeniously, but still a hack. What we really need are RAW files that do not need model-specific information for conversion. DNG does offer this.

Remember clarify this all for yourself - nonetheless my reading of the situation is not so rosy especially looking forward without a universal RAW format.

Tony Jay
Logged
Bryan Conner
Sr. Member
****
Offline Offline

Posts: 514


WWW
« Reply #108 on: April 23, 2013, 11:34:58 PM »
ReplyReply

Thanks Tony.
Logged

Schewe
Sr. Member
****
Offline Offline

Posts: 5415


WWW
« Reply #109 on: April 23, 2013, 11:56:02 PM »
ReplyReply

There are no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file.

Well, to be accurate, there are almost no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file...sadly, the recent Sigma cameras that use Foveon chips can't...the couple of cameras that can be converted to DNG end up as Linear DNG, not raw DNG. And the recent cameras by Sigma aren't supported at all although Adobe is working with Sigma to change that. Sadly, those people who have bought those cameras have no DNG/ACR/LR support and users are forced to use the Sigma supplied raw processing software :~(
Logged
Tony Jay
Sr. Member
****
Offline Offline

Posts: 2044


« Reply #110 on: April 24, 2013, 01:01:04 AM »
ReplyReply

Well, to be accurate, there are almost no proprietary RAW formats currently in production that cannot be accurately and fully documented in a DNG file...sadly, the recent Sigma cameras that use Foveon chips can't...the couple of cameras that can be converted to DNG end up as Linear DNG, not raw DNG. And the recent cameras by Sigma aren't supported at all although Adobe is working with Sigma to change that. Sadly, those people who have bought those cameras have no DNG/ACR/LR support and users are forced to use the Sigma supplied raw processing software :~(
Thanks for the clarification Jeff.
I will follow that up and read all about it.

Tony Jay
Logged
Tony Jay
Sr. Member
****
Offline Offline

Posts: 2044


« Reply #111 on: April 24, 2013, 02:55:36 AM »
ReplyReply

Well, memory jogged.

I did read Michael's review of the Sigma SD1 but some of that detail did not really penetrate at the time.

More in depth research of the sensor and how it works clearly indicates why neither Adobe products or any DCRAW application can read the file.
Paradoxically though, my reading of this situation indicates that this is a situation that DNG was born for, not as an intermediate, but as the native out-of-the-camera RAW file.
Here we have a very interesting new type of sensor that is potentially a good step forward in sensor technology. Potentially better colour fidelity because no Bayer-type arrays are required so no demosaicing (and all its attendant assumptions) and no anti-aliasing filter is required so real resolution/detail may be improved. Nice!

But, the only bit of software that can read this baby's files is provided by Sigma.
Given Michael's brief dismissal of that software obviously it is not up to scratch.

However, if Sigma took on DNG as its native RAW format the result could be a RAW file of rare quality that is also simultaneously accessible by any RAW converter that knows DNG (and there is no reason that, that should not be possible).
If Sigma did this I would suggest that any Sigma camera model using Foveon-type sensors would really become hot-ticket items.
Potentially a win for Sigma.

I doubt anything I have written here in the absolute sense is original.
This must have occurred to others.
Nonetheless, this is an excellent potential use-case demonstating the principle for which the DNG format was designed.
Many cameras using very diverse technology can output a camera-independent RAW format that can be read and processed by any application that knows DNG without that application needing to know anything about the camera that originally produced the image.

Whats not to like?

Tony Jay
Logged
fredjeang2
Sr. Member
****
Offline Offline

Posts: 786



« Reply #112 on: April 24, 2013, 03:46:02 AM »
ReplyReply

What would be a gas is that devs
Could provide a separate file (like the RMD)
Where are stored all the metadatas.
The RAW file remaining always untouched.

This system is incredibly flexible.

Red adopted it and it really is taking full advantage
Of the RAW.

You can create a look from the dev, put the file into
The card and shoot with the created look. You film
With the parameters that have been created in their
RAW dev. But the camera settings aren't lost either.

You can mail the file. Batch apply any look in the editor,
In a second, rebuild the looks at any point, name those
Metadatas files so they have a meaning within the
Prod, assure that every system reads the same metadatas
Accuratly.

To simplify, the file structure is
001.r3d. The RAW
001.rmd. the metadatas of the 001, at the same
Directory, plus,
All the created rmd. That could be located anywhere and
Named. Ex: folder name: night-shots-day-1, with inside
Nameofyourchoice.rmd etc...

They can be loaded in-camera or in a proper software
That is not Red proprietary as LUTs.

For the ones who are not familiar with this type of workflow,
I did a short video tutorial recently on the advantage of having
separate standardized metadatas files.
http://vimeo.com/63008611

(ps: very sorry for my english, this microphone experience has been a nightmare
for me as I was looking for my words all the time and, well, it's clumpsy speach
but you got the idea.)
« Last Edit: April 24, 2013, 04:41:32 AM by fredjeang2 » Logged
AlfSollund
Full Member
***
Offline Offline

Posts: 128


« Reply #113 on: May 01, 2013, 11:04:21 AM »
ReplyReply

For what its worth, I believe it is in our best interests to support the adoption of a non-proprietary raw format from the camera manufactures. This would benefit us with lower manufacture costs and improved future-prof interoperability.

I thank for all the information given, in particular to Tony.

What really eludes me is the reasoning behind what I consider as personal resistance against a (de-facto) standardized open RAW format. Is it some kind of hidden fan-boy resistance due to the understanding that Canikon and others have made the wrong choices? By wrong I mean long-term both internally as a cost-driwer as well as a part of a photographic community. Sorry if tis offends, its not a personal attack on anyone.
Logged

-------
- If your're not telling a story with photo you're only adding noise -
http://alfsollund.com/
Vladimirovich
Sr. Member
****
Offline Offline

Posts: 1264


« Reply #114 on: May 01, 2013, 05:36:38 PM »
ReplyReply

But, the only bit of software that can read this baby's files is provided by Sigma.

or Iridient Rawdeveloper for example... which is a one man shop really... which shows how much effort really needed if there is a desire to support...

Many cameras using very diverse technology can output a camera-independent RAW format that can be read and processed by any application that knows DNG without that application needing to know anything about the camera that originally produced the image.

once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard... shall Panasonic disclose in advance their move to competition ? shall Panasonic seek approvement from competition for what they want ? no, decided Panasonic... so they did what they wanted, when they wanted and how they wanted... and half a year later Adobe had to update the DNG standard... right  Roll Eyes ?
Logged
Bryan Conner
Sr. Member
****
Offline Offline

Posts: 514


WWW
« Reply #115 on: May 01, 2013, 11:38:28 PM »
ReplyReply


once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard... shall Panasonic disclose in advance their move to competition ? shall Panasonic seek approvement from competition for what they want ? no, decided Panasonic... so they did what they wanted, when they wanted and how they wanted... and half a year later Adobe had to update the DNG standard... right  Roll Eyes ?

I did not know that Adobe in competition with Panasonic.  How is this?  I am by no means an expert on dng, or any raw file format, but does a camera manufacturer have to reveal what they are doing to Adobe in order to make a change to the parameters in the dng file format?  In other words, does Adobe have to know what is going into the container before making changes to the container?
Logged

Ellis Vener
Sr. Member
****
Offline Offline

Posts: 1726



WWW
« Reply #116 on: May 02, 2013, 07:55:23 PM »
ReplyReply

Tony Jay: That's a well and dispassionately stated debunking of the F-U-D spreading about DNG.
Logged

Ellis Vener
http://www.ellisvener.com
Creating photographs for advertising, corporate and industrial clients since 1984.
Tony Jay
Sr. Member
****
Offline Offline

Posts: 2044


« Reply #117 on: May 02, 2013, 11:36:12 PM »
ReplyReply

Thank you Ellis.

Tony Jay
Logged
fredjeang2
Sr. Member
****
Offline Offline

Posts: 786



« Reply #118 on: May 03, 2013, 06:25:32 AM »
ReplyReply

We really need standardized international
And independant formats and workflows,
Like the efforts in the as11, with capabilities
To add shims.

The counsiousness of broadcasters in order
To solve the not-tolerable-anymore proprietary
Mess and move from "common practises" to
Real common file format is not a luxury.

We still are in the wild west in digital imagery,
And it needs to be civilized.

This, also concerns still imagery and specially
Raw formats.
When RAWvideo will be the norn, it will not be
Possible to handle with the nonsense current panorama.

It's not a question if standart will occur, it will,
It's more a question of when.

And the very important issue is separate metadatas files
from the viewing datas. Really really key. We need proper versioning
within platforms and proper mastering-archiving etc...
« Last Edit: May 03, 2013, 11:18:37 AM by fredjeang2 » Logged
NikoJorj
Sr. Member
****
Offline Offline

Posts: 1063


WWW
« Reply #119 on: May 06, 2013, 02:54:57 AM »
ReplyReply

Just to cast another "yes" vote...
Beyond the standardisation issue, which is a tad more about principles than pragmatism - even if those are good principles in my book, because for an end-user like me, camera support in "any" (as "any that supports DNG") software would be a appreciable bonus.

once upon a time Panasonic decided to introduce a certain flavor of software correction parameters to be stored in raw files... do you remember how much time it took Adobe to support that in DNG ? imagine DNG being an ISO standard...
DNG supports proprietary metadata, so it would certainly not block innovation, and from what I understand of DNG structure, the file should still be processed in other software not aware of the innovation, only without this particular innovation - the relevant metadata just waiting to be read to add the plus.
From what I can guess and imagine, DNG adoption would really be a win-win situation in such a case.
Logged

Nicolas from Grenoble
A small gallery
Pages: « 1 ... 4 5 [6] 7 »   Top of Page
Print
Jump to:  

Ad
Ad
Ad