Ad
Ad
Ad
Pages: « 1 2 [3] 4 »   Bottom of Page
Print
Author Topic: Lloyd Chambers compares D3x - 5DKII shadow performance  (Read 22551 times)
Daniel Browning
Full Member
***
Offline Offline

Posts: 142


« Reply #40 on: January 26, 2009, 03:33:34 PM »
ReplyReply

Quote from: LEPING
Lloyd just reported that D3x is also much cleaner (no hot pixels or streaking) at long exposures (30 seconds and over) WITHOUT dark frame subtraction ("Long Exposure NR") than the 5DII, which he says come with a "dirty" sensor.

Even when NR is disabled, Nikon applies noise reduction to long-exposure raws designed specifically to remove hot pixels. (Incidentally, it also removes stars, which is frustrating for us astrophotographers).

If he wanted to see what Nikon raw is like without noise reduction, he should *enable* dark frame subtraction in the camera (enabling this causes the camera to disable the other hot pixel noise reduction), take an exposure, then turn off the camera after the light frame (but before the dark frame can finish). This will cause the dark frame subtraction *and* the hot pixel noise reduction to both be skipped. (The cost, of course, is a bunch of manual labor every time you take a light frame.)
Logged

--Daniel
Guillermo Luijk
Sr. Member
****
Offline Offline

Posts: 1274



WWW
« Reply #41 on: January 26, 2009, 03:47:08 PM »
ReplyReply

Quote from: Daniel Browning
take an exposure, then turn off the camera after the light frame (but before the dark frame can finish). This will cause the dark frame subtraction *and* the hot pixel noise reduction to both be skipped. (The cost, of course, is a bunch of manual labor every time you take a light frame.)
And is the NEF file properly saved into the memory card when doing this 'operation'? that sounds a bit strange.

Logged

Leping
Jr. Member
**
Offline Offline

Posts: 87



WWW
« Reply #42 on: January 26, 2009, 04:10:59 PM »
ReplyReply

I understand and appreciate your elaboration (your long post on the next page where many if not all the links are not working).  However to me you did not study and really understand the two 3D charts I posted which made your results largely irrelevant.

On the DxO exact measurements: if you were patient enough and mouse over the DxO charts (on their web site not my extracted images) you will see the actual data values pop out in small yallow squares.

Read their technical documents and understand what their numbers really mean.

Quote from: Panopeeper
The problem is, that they do not publish the images and the exact measurements they have taken; thus one can not reconstruct their results and critiques can not be qualified.
« Last Edit: January 26, 2009, 05:17:21 PM by LEPING » Logged

BernardLanguillier
Sr. Member
****
Offline Offline

Posts: 7776



WWW
« Reply #43 on: January 26, 2009, 04:44:21 PM »
ReplyReply

Quote from: Panopeeper
I created a demonstration for the support of my statements re the D3X and 5D2 evaluations.

a. I used the raw files from Imaging Resources, because they are suitable (more or less), and I don't have any others suitable. They were recorded in 12bit mode (why on earth); this would pose a problem *if* the DR of the D3X were really so large; however, it is not.

With all due respect.... I stopped reading here... how on earth are you hoping to be able to convince anyone that testing with 12 bits file is going to give representative results for a camera that claims to be the first one able to make useful use of 14 bits raw files?...

Is that approach more "scientific" than those proposed by DxO and Lloyd?

Cheers,
Bernard
Logged

A few images online here!
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #44 on: January 26, 2009, 06:00:13 PM »
ReplyReply

Quote from: BernardLanguillier
With all due respect.... I stopped reading here...
Well...my post is not meant as an introductory course in the understanding of raw image data.
Logged

Gabor
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #45 on: January 26, 2009, 06:06:37 PM »
ReplyReply

Quote from: LEPING
However to me you did not study and really understand the two 3D charts I posted which made your results largely irrelevant
To me you did not study and really understand my demonstration, which make your reasult largely irrelevant (sorry for the wrong URL's; an internet savy person would have noted the strange semicolon at the end of the URLs).

Now, that I repaired the links, you should have no difficulty to relate my statements to actual measurements. Note: these are measurements reproducable for anyone from the used images - in contrast to DxO's and Loyd's numbers, which I can type in in any program as "proof".
Logged

Gabor
jjj
Sr. Member
****
Online Online

Posts: 3249



WWW
« Reply #46 on: January 26, 2009, 06:14:19 PM »
ReplyReply

Quote from: LEPING
What you have said is that basically Lloyd was right and there are solid recommendations in his Mac performance site,
No I said he talked some rubbish and some good stuff, but as he started with the rubbish.....

Quote
but you need more hard drives beyond the Mac Pro's capacity of 7,
It's ostensibly 4 unless you want to start doing hacking cables and digging around in your Mac, removing optical drives and then it's 6 SATA drives.
Quote
...and you know things like the tricks to run Win64 on a Mac Pro he didn't.
Uh yeah, run Bootcamp and insert W64 DVD into Mac DVD drive!! But shush don't tell anyone else.

Quote
Congratulations for being such a elite computer user requiring at least two 4-HD raws inside your desktop machine,
I'm elite now! That's good to know, I shall go away and practice my sneering in a downwards manner.
Nothing elitist about accumulating data. 3 HDS for storage is pretty damn small these days particularly if you mirror for safety and archive. Which is a 3 drives already. But your archive should be elsewhere really, so only 3 in machine. Which leaves  1 bay for a scratch disc, 1 bay for OS and Progs and 2 for storage. Not a lot really. Certainly not elitist to need more.

Quote
but isn't that true for the most of the rest (needing less than 7 drives, etc.) there are still things to learn from the site, such as how to separate data and applications, and how to set up a Photoshop specific scratch volume or a RAM disk to speed things up?
How do you know what is good advice, accurate advice or fanboi wittering?
Quote
And is it true for them the Mac Pro is still quiet and tidy?
Not when the DVD spins up and one reason why I took it out.

Quote
I do competitive photography, but I do not happen to know that much about computer.  Is this makes me disqualified to express my findings here in a photography forum, or my own blogs?
Nope, never said that was the case.
Logged

Tradition is the Backbone of the Spineless.   Futt Futt Futt Photography
BernardLanguillier
Sr. Member
****
Offline Offline

Posts: 7776



WWW
« Reply #47 on: January 26, 2009, 06:16:54 PM »
ReplyReply

Quote from: Panopeeper
Well...my post is not meant as an introductory course in the understanding of raw image data.

Well... I am not sure why you even bother commenting on this issue without having proper testing material (14 bits images). I'll read your stuff once you have done your homework and tested (14 bit files) what has to be tested and not something else (12 bits files).

In case you respect Thom Hogan more than the Field medal winners/MIT professors belonging to the DxO scientific board, he is claiming that he sees one extra stop DR in the D3x compared to any other Nikon DSLR at base ISO (see this updated review just published - http://www.bythom.com/nikond3xreview.htm). Considering that the D3 is also considered to have more DR at base ISO compared to any Canon DSLR you get claims that are pretty close to those made by both DxO and Lloyd.

So again, these people have tested the camera with suitable 14 bits raw files, you have not, yet you keep making sweeping statements on the D3x vs 5DII.

The most reasonnable thing to do would be to try to find suitable data, test again and report back to us. I will then consider your inputs as one of the information sources and weight it fairly relative to the others... final comment from me on this matter.

Cheers,
Bernard
Logged

A few images online here!
Daniel Browning
Full Member
***
Offline Offline

Posts: 142


« Reply #48 on: January 26, 2009, 06:17:57 PM »
ReplyReply

Quote from: GLuijk
And is the NEF file properly saved into the memory card when doing this 'operation'? that sounds a bit strange.

That is how it works on previous Nikon DSLR; I don't have a D3X to test for myself. It is strange and laborious, just like the many other  workarounds photographers are foced to use if they want to get the highest quality images (e.g. UniWB, negative EC instead of ISO 1600+, MLU in liveview, etc.).
Logged

--Daniel
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #49 on: January 26, 2009, 07:01:51 PM »
ReplyReply

Quote from: BernardLanguillier
Well... I am not sure why you even bother commenting on this issue without having proper testing material (14 bits images)
[
I see I have to reveal this to you: Nikon is offering not only 12bit, but LOSSY 14 bit, even worse, LOSSY 12 bit as well.

If you think that the 12bit recording is inferor to the 14bit, you should create a thread about that subject, and report your factual findings to Nikon. Isn't it a bande of suckers, selling a camera fo $6000-8000 or whatever,  and fooling the customers into using seriously inferior options?

I for myself would not use 12bit, nor lossy compression, but I am not a Nikon owner.

Back to the subject: the measurements on 12bit do not indicate, that any big change is expectable with greater bit depth.

Quote
In case you respect Thom Hogan more than the Field medal winners/MIT professors belonging to the DxO scientific board, he is claiming
I don't give a fig for his or others' credentials; I am writing about sensors, not about testers. I wonder, why you are not questioning their measurements: you are accepting numbers presented in nice charts without any proof.

I know almost nothing of LLoyd, but what I do know is more than enough: if he is testing these cameras at ISO 50 and ISO 6400, then he knows a LOT less about these cameras than I do without ever having held them in my hands.
Logged

Gabor
Ray
Sr. Member
****
Offline Offline

Posts: 8854


« Reply #50 on: January 26, 2009, 07:03:50 PM »
ReplyReply

When there are doubts as to the relevance of scientific tests or reviews of camera performance, one simply has to carry out one's own tests. This is what I did after initial reports on the performance of the D3 claimed it had over a stop  (and maybe up to 2 stops) lower noise at high ISO than any other camera on the market at the time.

I was not able to hire a D3 (they were in such short supply at the time) so instead made arrangements with the Nikon agent in Bangkok to compare their demonstration copy of the D3 with my own 5D, in the store. Unfortunately, they wouldn't let me borrow their D3 and take it outside for thorough testing because they had only one copy for demonstration purposes.

Nevertheless, I was able to satisfy myself that at ISO 3200 and above, the D3 had approximately 1/3rd stop to 1/2 stop noise/DR advantage over the 5D.

However, I didn't bother and indeed didn't have the time to compare both cameras at base ISO. If I had done so, I would no doubt have discovered that the main advantage of the D3 with respect to noise and DR was not at high ISO but at base ISO where I now believe it's at least one stop better than the 5D.
Logged
BernardLanguillier
Sr. Member
****
Offline Offline

Posts: 7776



WWW
« Reply #51 on: January 26, 2009, 07:26:04 PM »
ReplyReply

Quote from: Panopeeper
I see I have to reveal this to you: Nikon is offering not only 12bit, but LOSSY 14 bit, even worse, LOSSY 12 bit as well.

If you think that the 12bit recording is inferor to the 14bit, you should create a thread about that subject, and report your factual findings to Nikon. Isn't it a bande of suckers, selling a camera fo $6000-8000 or whatever,  and fooling the customers into using seriously inferior options?

I for myself would not use 12bit, nor lossy compression, but I am not a Nikon owner.

This isn't you Gabor, is it? Tell me that somebody hacked your login/password and is trying to damage your reputation with this non sense?

How is the availability of lower image quality options in a camera relevant when discussing the best output it can produce? Why did you stop there and didn't you comment on the stupidity of Nikon providing the option to shoot in medium resolution jpgs? Using the same logic you are using above, you could have written "If Nikon provides medium resolution jpgs, it must mean that the quality of a medium resolution jpg file is as good as that of a 14 bits raw".

You know full well that 12 bits/lossy options with the D3x can be useful sometimes because of the smaller file sizes and higher frame rate they boast. The availability of these options says nothing about the quality gap between them and the best available 14 bits mode.

Quote from: Panopeeper
Back to the subject: the measurements on 12bit do not indicate, that any big change is expectable with greater bit depth.

How on earth are you able to make assumptions on the behaviour of a 14bits file from a 12 bits file? How rigorous and scientific is this approach?

Quote from: Panopeeper
I don't give a fig for his or others' credentials; I am writing about sensors, not about testers. I wonder, why you are not questioning their measurements: you are accepting numbers presented in nice charts without any proof.

I stopped believing in conspiracies a few years back. I have no reason to doubt the credibility of these information sources, and you have so far not brought any evidence to the table indicating that they could be wrong. You are measuring something different.

All I am saying is, please do an apple to apple comparison by measuring the same thing they are measuring.

If you are not interested in doing this, please avoid commenting on the quality of their results.

Cheers,
Bernard
Logged

A few images online here!
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #52 on: January 26, 2009, 08:18:57 PM »
ReplyReply

Quote from: BernardLanguillier
How is the availability of lower image quality options in a camera relevant when discussing the best output it can produce? Why did you stop there and didn't you comment on the stupidity of Nikon providing the option to shoot in medium resolution jpgs?
Bernard, you too are posting and reading on other forums as well, for example DPReview. Thre is a thread in the D3 forum called "NEF Compression". Several members were expressing their firm conviction, that compression means always loss; like if you intend on editing your best bet is not to compress. I read many other threads stating, that the lossy compression does not cause any visible loss (ask Emil Martinec about that).

In short: expecting the customers making such decisions is far too much.

However, I reiterate: I am a proponent of lossless compressing in 14bit depth.

Quote
How on earth are you able to make assumptions on the behaviour of a 14bits file from a 12 bits file? How rigorous and scientific is this approach?
This is quite simple: look at the involved pixel values and the standard deviations.

It IS possible, that the D3X shows somne extra advantage in the very-very deep shadows, but

1. do you find it useful, when the camera is more noisy in the "normal shadows" but less noisy in the "very deep shadows"?

2. only a few of us know, thay all Nikon cameras are cutting off the worse part of the very dark regions; that is the reason, that you see the horizontal or cross-hatch pattern with the 5D2 bu not with any Nikon. This is the question of raw processing.

Anyway, there is something I don't understand here. You do have the D3X; you developed a template for the DR measurement. Why don't YOU make suitable shots and publish the raw files instead of this fruitless discussion? Then we could see the difference between the noise in 12bit and 14bit, as well as the very deep shadows, instead of speculating about that.
Logged

Gabor
BernardLanguillier
Sr. Member
****
Offline Offline

Posts: 7776



WWW
« Reply #53 on: January 26, 2009, 09:38:49 PM »
ReplyReply

Quote from: Panopeeper
Anyway, there is something I don't understand here. You do have the D3X; you developed a template for the DR measurement. Why don't YOU make suitable shots and publish the raw files instead of this fruitless discussion? Then we could see the difference between the noise in 12bit and 14bit, as well as the very deep shadows, instead of speculating about that.

Gabor,

Yes, I do own a D3x, but I have not developped any template for DR measurement.

I do personnaly not doubt the excellence of the D3x DR at low ISO and frankly speaking, my life is already complex enough those days that I am not willing to embark on such a project. Sorry about that.

What I could perhaps do is send you a few 14 bits nefs using yousendit if you provide me with a suitable email address through PM.

Cheers,
Bernard
Logged

A few images online here!
Panopeeper
Sr. Member
****
Offline Offline

Posts: 1805


« Reply #54 on: January 26, 2009, 10:22:15 PM »
ReplyReply

Quote from: BernardLanguillier
Yes, I do own a D3x, but I have not developped any template for DR measurement

Then I don't remember who made the template for Ray. Sorry for the mixup.

Quote
I do personnaly not doubt the excellence of the D3x DR at low ISO and frankly speaking, my life is already complex enough those days that I am not willing to embark on such a project. Sorry about that
This is perfectly all right. In fact, your happiness with the camera should not depend on any evaluation but on what it is doing for you. Evaluations/reviews are meant for before purchasing, not afterwards.

I purchased my 40D after many customers reported their findings, but before DPReview reviewed it. Then I have never read the review; what for?

Quote
What I could perhaps do is send you a few 14 bits nefs using yousendit if you provide me with a suitable email address through PM
Thanks for the offer, but

1. I can use only particular images: containing some smooth, non-textured, unicolored, dark, not curving surfaces of very even illumination in the very dark shadows, in different shades. Otherwise measuring the noise is not much worth,

2. I have no personal interest in this question, I have been doing this for those, who might be interested and were misled (IMO) by the mentioned reviews;

3. my email box quota is not enough for even a single file (max. 15 MB); yousendit is the way to go anyway.

However, if you have some "spare" raw files with 1/3 stop ISOs, particularly if clipping occured, only in 14bits, that would be nice to have, so that I can determine the saturation levels, and if those ISOs are fake (like the 5D2's) or associated with some analog gain. Though I don't know why you would be using these ISO steps.

Finally, for the case you don't know this yet: ISO 50 is non-existent and ISO 3200 is fake, so there is no reason to use them with raw.
Logged

Gabor
Leping
Jr. Member
**
Offline Offline

Posts: 87



WWW
« Reply #55 on: January 26, 2009, 10:39:55 PM »
ReplyReply

This is totally untrue.  Nikon clamps the DC level at the center of the noise distribution is a well known fact, not only a few know, but this is does not mask or bury any pattern noise in the shadows and much above that level.  Plus, the extraordinarily clean deep shadows is a brand new phenomenon to the DSLR sensors, as the Nikon's own D3/D700, which also cut the true black noise by half, do not match the cleanness, as well documented in both DxO measures and visual test results.

Take close look at the crops in the Lloyd's open article again.  The Canon's pattern noise does not only happen at nearly the pure dark noise level but also way above it, far above where Nikon biased down to zero.

http://www.diglloyd.com/diglloyd/free/Push...acks/index.html

Actually, today, Lloyd just posted new comparisons in the DAP (pushing underexposed foreground shadow details by two stops in RAW conversion by setting two stops of exposure compensation, in real landscape situation at ISO 100).  There is this free mouse over comparison, which demonstrates the outcomes, but the paid section offers much more details including the full size images.  If, as you claimed, Nikon just buried all the deep shadow details, there would be nothing to push to be compared with the Canon.

http://www.diglloyd.com/diglloyd/2009-01-b...0126D3x_vs_5DM2

I just wish I can have Lloyd's permission to copy his conclusion words here.

Quote from: Panopeeper
2. only a few of us know, thay all Nikon cameras are cutting off the worse part of the very dark regions; that is the reason, that you see the horizontal or cross-hatch pattern with the 5D2 bu not with any Nikon. This is the question of raw processing.
« Last Edit: January 26, 2009, 11:27:22 PM by LEPING » Logged

NikosR
Sr. Member
****
Offline Offline

Posts: 622


WWW
« Reply #56 on: January 26, 2009, 11:52:43 PM »
ReplyReply

Gabor,

Iliah Borg warned you not to use imaging resource images for a noise comparison due to reasons of target quality and shooting methodology
http://forums.dpreview.com/forums/read.asp...essage=30792855
http://forums.dpreview.com/forums/read.asp...essage=30795654

Target quality (noise inherent in the target) was mentioned also by THom Hogan in the same thread.

Additionally, DxO state that according to their findings 14bits do make a difference in the case of the D3X
http://www.dxomark.com/index.php/eng/DxOMa...r-the-Nikon-D3X

There's no doubts in my mind that imaging resource targets were not shot with you analysis in mind. I would tend to trust more someone who shoots for the test (DxO) rather than an analysis a posteriori of the resuts of two independent shoots.

However you chose to ignore all of the above and use imaging resource images shot at 12-bit, to try and demonstrate that DxO are wrong.

Do you honestly believe you have demonstrated this? Your rant about compression (how on earth did you manage to get this in the discussion?) tells me that you don't.

(I want to be clear that I believe you might well be right. However any self respecting scientist would not go public in the aggressive way you have done so, based on the quality of your data. If I was feeling so strongly about proving somebody wrong, I would beg borrow or steal to be able to test under a controlled environment. Otherwise I would just express my doubts and shut up).

 I will give you this though. DxO publish numbers and findings without revealing their testing methodology. Thus it is difficult to argue about the way they are testing).


(PS. For anyone inclined to do this, the above dpreview thread contains an interesting exchange between Gabor and member bobn2 which, leaving test image validity aside, boils down to how and if Gabor considers in his measurements the difference in read noise (as opposed to shot noise) which by many accounts is the factor which differentiates the D3X at very low EV levels and seems to differ substantially between 12bit and 14bit captures. For reference one can use DxO's logarithmic FULL SNR diagrams here:

http://www.dxomark.com/index.php/eng/Image...abase/Nikon/D3X
http://www.dxomark.com/index.php/eng/Image.../EOS-5D-Mark-II

Just select the Full SNR diagrams and press the Logarithmic tab
)

PS2. One final word from me in this thread. Quoting Michael, when your eyes disagree with the numbers trust you eyes... In my case, I can only use my eyes indirectly since I have no access to a D3X. Because of this, and until shown differently, I will choose to trust Lloyd's eyes as evidenced in his visual tests which, in any case, seem to be quite in line with at least one set of measurements.
« Last Edit: January 27, 2009, 01:28:36 AM by NikosR » Logged

Nikos
Leping
Jr. Member
**
Offline Offline

Posts: 87



WWW
« Reply #57 on: January 27, 2009, 01:03:46 AM »
ReplyReply

Hi NikosR,

Thank you for your contributions -- I really enjoyed reading them.

To me DxO had actually quite adequately documented their testing methodology, in the more than 20 pages in their Technologies section:

http://www.dxomark.com/index.php/eng/Technologies

For instance, their noise measurement protocols.  They are the only tester claimed to follow the ISO standards, as far as I know.  Some of the descriptions and formulas are not very inviting for quick reading there, but they did try to clarify what they measure and how the numbers are defined and derived.  I am sure there are rooms for clarification, organization, and comprehensiveness though.

http://www.dxomark.com/index.php/eng/Techn.../Noise-protocol

Also, as I did, you will find your links (to the dxomark site) did not lead to where you wanted.  They do not provide direct links to most of their charts (but you can link to the two charts I extracted and posted in this thread).

Best regards,
Leping

Quote from: NikosR
I will give you this though. DxO publish numbers and findings without revealing their testing methodology. Thus it is difficult to argue about the way they are testing.
« Last Edit: January 27, 2009, 01:43:27 AM by LEPING » Logged

NikosR
Sr. Member
****
Offline Offline

Posts: 622


WWW
« Reply #58 on: January 27, 2009, 01:25:59 AM »
ReplyReply

Quote from: LEPING
Also, as I did, you will find your links (to the dxomark site) did not lead to where you wanted.  They do not provide direct links to most of their charts (but you can link to the two charts I extracted and posted in this thread).

Indeed. Just press on the Full SNR tab in the links provided and choose Logarithmic.

It's obvious that there is a unique characteristic in the D3X curves (as compared to other cameras) which, not knowing better, I can only attribute to very low read noise.

Leping, thanks for pointing me to their noise protocol write up. I had missed that although I had gone over some of their other write ups in that section.To a novice like me, their methods sound much more controlled than measuring off files produced by imaging resource.

(PS. One interesting thing I noted is that they are measuring all channels for noise. Gabor argues, not unconvincingly, in the aforementioned dpreview thread, that it should not matter which channel you're measuring in the raw data. I'm wondering and the only explanation I can give, if indeed the channel makes a difference, is the potential of metamerism effects or non-linear wavelength response in sensors leading to difference in shot noise depending on wavelength  )


PS2. I've just gone over Lloyd's latest comparison in DAP in which he extracts details out of the shadows in a very high contrast image by pushing in the raw converters by 2 stops. I don't care if somewhat different results might have been achieved by different selection of raw converters. The difference between the 2 cameras can only be descibed by 'Wow!'. Since for a digital capture to be a photograph it has to go through the demosaicing phase, what does it matter if the difference in noise in the raw files can be measured as 0.5EV or 1.7EV? Just looking at Lloyd's comparison does not leave anyone with a doubt about the GREAT difference in low level detail that can be extracted in the two cameras and the quality differences resulting from the 2 stop push.The difference is well beyond pixel-peeping territory. It's about the difference I see between my D70 and my D700. It jumps up and smacks you in the face. I hope I'm not infringing any copyright here since the same can be seen in his free overview.
« Last Edit: January 27, 2009, 03:35:44 AM by NikosR » Logged

Nikos
Ray
Sr. Member
****
Offline Offline

Posts: 8854


« Reply #59 on: January 27, 2009, 04:58:30 AM »
ReplyReply

Quote from: Panopeeper
Then I don't remember who made the template for Ray. Sorry for the mixup.

Ray's template is very basic, but accurate enough for him. If image of very contrasty scene from camera A, with ETTR exposure, appears to have the same noise in the deepest shadows as image from camera B, when image from camera B is overexposed by one stop, then camera A has has 1 EV greater DR than camera B.

Nothing could be simpler. I don't know what all the fuss is about.
Logged
Pages: « 1 2 [3] 4 »   Top of Page
Print
Jump to:  

Ad
Ad
Ad