Well I have "broken" a few lenses and it's always been damn obvious, I wouldn't have needed to do a "test" to tell. The differences between good and bad were much larger then what I ever see in the sample variations in internet tests. (E.x. 50mm lens being as soft at f/8 as it was at f/1.
If the lens is obviously broken then there's no need to carry out a series of tests to determine if it's broken, Ben. We're not completely silly here, you know. However, if you perhaps dropped a lens and it seemed
to be still okay but you are not sure, you might want to do a few controlled tests to confirm this.
Bottom line is I can't trust 99% of what I see on the internet, especially when it comes to tests which have assigned quantifiable #s based on analyzing a picture.
Nor can I, but I don't ascribe a number to the quantity I can't trust because that would be imprecise. The reason I don't trust many of the lens tests I see on the net is because it's often clear the people carrying out the tests are (1) not using best practices, ie. tripod, remote release, MLU, consistent focussing and exposure, (2) not using RAW mode with consistent conversion settings and no sharpening, (3) presenting the images in highly compressed jpeg form instead of minimally compressed jpeg form (ie., maximum quality).
Michaels tests are a little easier for me to trust, as he posts the original images and just writes about his feelings about the two. Since the images he is basing his opinion on are right there in the article we can all look at them and make our own decisions. He also doesn't assign numbers, which eliminates the need to worry about accuracy and margins of error.
You're not making much sense, Ben. The results of most of Michael's equipment tests are based on hard, visual evidence rather than feelings. Numbers are something you can't get away from whether it's a number describing an f stop, a shutter speed, an ISO setting or a lens focal length. Some of Michael's tests are replete with numbers; his DXO Analyzer tests of the Canon 28-300 and 70-300 DO for example.
The only equipment reviews by Michael I see on this site that are based primarily on feelings and impressions are the field reports and the like where Michael has not tested the equipment under controlled conditions, such as the tilt & shift Hartblei Super Rotator, for example.
I just don't see the point in debating so massively or worrying so much about the tests, as even if you decide based on the internet tests you could still get a bad sample, or the lens could work perfectly and you just decide it doesn't have the right look for your pictures.
No need to worry at all. If you are concerned about getting the biggest resolution bang for your buck, then test the lens yourself before buying, or better still, find a store with a 'no questions asked' return policy in the event you are not satisfied. You can then test the lens more thoroughly than would be possible in the store and return it a week later if you think it's below par.