Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
next PZ Lens Test Report - Zeiss ZA T* 24mm f/2 SSM
#11
[quote name='Klaus' timestamp='1292451337' post='5022']

So how do you rate the performance at f/1.4 then ? This cannot be ignored. If it offers f/1.4 (and priced higher) it should deliver something more than that. Just arguing that the marks aren't much worse at comparable apertures doesn't seem to be valid to me.

[/quote]

That does not really make that much sense though, Klaus...



So on one side we have lens A, doing everything well (Zeiss 24mm f2). On the other side, we have 2 lenses, B and C, doing exactly the same things equally well (Canon 24mm f1.4 L USM II and Nikon AF-S 24mm f1.4).



Lens A gets a higher score. Why? It is not about that the f1.4 lenses, B and C, are more expensive, as there is another rating for that.



No, It is because lens B and C offer something lens A does not offer.



So... we have lens A, B and C doing more or less equally well at f2, f2.8, f4, f8 and so on. Distortion, sharpness, vignetting, bokeh, they are all close.

B and C do more or less equally well at f1.4, where lens A totally fails.



No, it is not right to rate a lens less, just because it offers f1.4 extra, where the other lens stops at f2. If two lenses perform equally, at the same settings, they should get the same rating. Price gets rated in "Price/Performance".



I do understand that then the rating will not specifically be about f1.4. And that at f1.4 one would hesitate to give it 4 stars. But rating a lens better when its f1.4 performance is just... non-existent makes little sense. And no, I do not know how to solve this in your rating system, right now. But it is food for thought.
#12
[quote name='Brightcolours' timestamp='1292453774' post='5024']

That does not really make that much sense though, Klaus...



So on one side we have lens A, doing everything well (Zeiss 24mm f2). On the other side, we have 2 lenses, B and C, doing exactly the same things equally well (Canon 24mm f1.4 L USM II and Nikon AF-S 24mm f1.4).



Lens A gets a higher score. Why? It is not about that the f1.4 lenses, B and C, are more expensive, as there is another rating for that.



No, It is because lens B and C offer something lens A does not offer.



So... we have lens A, B and C doing more or less equally well at f2, f2.8, f4, f8 and so on. Distortion, sharpness, vignetting, bokeh, they are all close.

B and C do more or less equally well at f1.4, where lens A totally fails.



No, it is not right to rate a lens less, just because it offers f1.4 extra, where the other lens stops at f2. If two lenses perform equally, at the same settings, they should get the same rating. Price gets rated in "Price/Performance".



I do understand that then the rating will not specifically be about f1.4. And that at f1.4 one would hesitate to give it 4 stars. But rating a lens better when its f1.4 performance is just... non-existent makes little sense. And no, I do not know how to solve this in your rating system, right now. But it is food for thought.

[/quote]

I dont agree often with BC but this time he is right.(Logic wise, I didnt compare the test results and assume it is as BC says.)



Anyhow, we should not waste too much time and effort discussion star ratings here. In the end no sensible person is going to base a buying decision on them. You need f/1.4, you dont buy an f/2 lens. You shoot Canon /Nikon, you dont give a shit about this sony lens, because you wont carry around another camera incompatible to all your lenses just to shoot this wonder. In any event your Canon /Nikon 24mm gives fully usable results at f/1.4 so why would you worry about another lensmakers slower lens? You shoot sony, you are happy and dont care about canon, or Nikon as long as your sony lens is bright enough.
#13
The Canon received 3pts optical & 4pts field, the Zeiss got 3.75pts total.

The field rating does take your argumentation into account actually.

Frankly I don't see anything wrong here.



[quote name='Brightcolours' timestamp='1292453774' post='5024']

That does not really make that much sense though, Klaus...



So on one side we have lens A, doing everything well (Zeiss 24mm f2). On the other side, we have 2 lenses, B and C, doing exactly the same things equally well (Canon 24mm f1.4 L USM II and Nikon AF-S 24mm f1.4).



Lens A gets a higher score. Why? It is not about that the f1.4 lenses, B and C, are more expensive, as there is another rating for that.



No, It is because lens B and C offer something lens A does not offer.



So... we have lens A, B and C doing more or less equally well at f2, f2.8, f4, f8 and so on. Distortion, sharpness, vignetting, bokeh, they are all close.

B and C do more or less equally well at f1.4, where lens A totally fails.



No, it is not right to rate a lens less, just because it offers f1.4 extra, where the other lens stops at f2. If two lenses perform equally, at the same settings, they should get the same rating. Price gets rated in "Price/Performance".



I do understand that then the rating will not specifically be about f1.4. And that at f1.4 one would hesitate to give it 4 stars. But rating a lens better when its f1.4 performance is just... non-existent makes little sense. And no, I do not know how to solve this in your rating system, right now. But it is food for thought.

[/quote]
#14
[quote name='jenbenn' timestamp='1292457528' post='5025']

I dont agree often with BC but this time he is right.



Anyhow, we should not waste too much time and effort discussion star ratings here. In the end no sensible person is going to base a buying decision on them. You need f/1.4, you dont buy an f/2 lens. You shoot Canon /Nikon, you dont give a shit about this sony lens, because you wont carry around another camera incompatible to all your lenses just to shoot this wonder. You shoot sony, you are happy and dont care about canon, or Nikon as long as your sony lens is bright enough.

[/quote]



Have you noticed that there's a field rating for the Canon - 4pts.

We've introduced this rating for ultra-large aperture lenses specifically to balance between the max. aperture performance and the "normal" range.
#15
[quote name='Klaus' timestamp='1292457581' post='5027']

Have you noticed that there's a field rating for the Canon - 4pts.

We've introduced this rating for ultra-large aperture lenses specifically to balance between the max. aperture performance and the "normal" range.

[/quote]

No I didnt check, sorry for being a bit ignorant. I was just follwing BC'S argument and found it logic in itself. Anyhow, Klaus rate as you like, since I recognize that a final grading always involves some subjective opinion on how to weigh the measured results. So frankly I cant and dont want to argue with you on that. It is your personal opinion, which is nice to know but can always be only an argument in my buying decision, not the decsion itself ( I believe you are very much aware of this ). Anyway you are doing a tremendous job testing all these lenses and every person spendning 2000 bucks on a 24mm 1.4 lens should know the reasons for which he buys the lens regardless of any stars awarded by photzone ( He will find your graphs much more useful).
#16
[quote name='Klaus' timestamp='1292402041' post='4997']

You're, of course, right that the difference is not overly significant at comparable apertures.

[/quote]

Umm... not sure what you mean by this in the conclusion then:



The Zeiss Distagon T* 24mm f/2 SSM may not be as fast as its Canon and Nikon counterparts but it's capable of delivering a substantially better image quality at comparable aperture settings.



<img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/smile.gif' class='bbc_emoticon' alt='Smile' />



My only gripe is with how you guys do your corner crops. In my experience, the 24LII is sharper in the corners than the 24 1.4G:



http://www.flickr.com/photos/genotypewriter/4698284415

[Image: 4698284415_53ecb51f01_m.jpg]



But apparently the way you choose your "extreme borders" more extreme than mine accounts for the photozone results. But as I explained about... what is the meaning of the [url="http://www.opticallimits.com/lens-test-faq"]photozone borders[/url] if they account only for a very small area of the image frame?



http://www.flickr.com/photos/genotypewriter/5207301499

[Image: 5207301499_ed6d40dd74_m.jpg]







[quote name='Klaus' timestamp='1292457531' post='5026']

The Canon received 3pts optical & 4pts field, the Zeiss got 3.75pts total.

The field rating does take your argumentation into account actually.

Frankly I don't see anything wrong here.

[/quote]

In regards to that, I understand where you're coming from... people want reviews to give hard conclusions when the results are shown right in front of their eyes. If there's no rating or ranking, many are unhappy because there are a lot of people out there who much rather rely on someone else telling them what's good instead of using their judgment and standing behind it. But if you're really going to give numbers for the ratings, you should have a proper definition of what they mean.



Honestly I can't see how a single number ("optical quality") can describe distortions at different focal lengths, vignetting at different apertures, absolute sharpness and its distribution at different apertures and focal lengths, LoCA and CA at different apertures and focal lengths, bokeh, etc. in a way that's fully consistent and comparable across lenses. So why give it at all?



GTW
#17
[quote name='Klaus' timestamp='1292451337' post='5022']

So how do you rate the performance at f/1.4 then ? This cannot be ignored. If it offers f/1.4 (and priced higher) it should deliver something more than that. Just arguing that the marks aren't much worse at comparable apertures doesn't seem to be valid to me.

[/quote]



As I wrote in my closing sentence, I don't know how to reflect that consistently in the rating system. After all, the summarized ratings are just for those people looking for... well... a quick summary, so they inevitably have to abstract from the numbers. In you user lens This is most likely spam content DB I believe you have different ratings for wide-open and stopped-down performance. But even this would only be a compromise. While "wide open" is perfectly clear, the rating is of course again relative to the lens' spec. And what does "stopped down" mean? Make that "one or two stops below the lens' maximum aperture" and it is still a rating relative to the lens' spec (and might push slower lenses well beyond the system's DLA). Make it a "common aperture shared by all lenses" it becomes f5.6 (or f6.3 for certain 3rd party superzooms on their long end), making the rating meaningless for fast lenses and making it fall together with the wide open rating for f5.6 lenses.

Perhaps you could make one rating "overall optical quality" (could be basically the current rating) and another one relative to the lens' competition. So the Canon would fall into the "ultra-fast UWA (for FF, make that WA in the APS-C scope) prime" for full frame, while the Zeiss would only be "fast". But even here I don't know if it makes sense to separate the two lenses into two categories... f2 is still quite fast, after all. But at least with a relative rating you could express that while "superzoom X stinks optically", it is actually "as good as superzooms get these days". With the relatively new rating for "field quality" which you use for some lenses (as it seems, you use it especially for fast lenses that are not totally flawless but have some unique capabilities) you have something that goes into that direction.





So, sorry, no, I don't have a conclusive answer, and I wonder if there is one that doesn't involve significantly extending the current rating system, which I understand is not desirable.

Since I guess anyone having a real interest in a lens will not make his purchasing decision based solely on the star ratings, but will have a look at the numbers, anyway.

The sole purpose of my posting was that I find the rating for the 24s not to be fully fair. And I don't even own that lens (nor a full-frame body), so I am not a over-defensive owner that tries to justify and confirm his purchase. But I am a Canonist, if you didn't already guess that. <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />



Anyway, to make this not sound too negative, I really like your site, it is one of the best lens review sites on the web, and I based many lens purchasing decisions on your measurements (and I always look at the star ratings as well, but do not use them alone <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/wink.gif' class='bbc_emoticon' alt='Smile' />). So, please keep up the good work.



Grummbeerbauer



PS: Oh, and if I ever have an inspiration for a simple-yet-more-precise rating system that solves the dilemma, I will tell you.
#18
[quote name='Brightcolours' timestamp='1292453774' post='5024']

So... we have lens A, B and C doing more or less equally well at f2, f2.8, f4, f8 and so on. Distortion, sharpness, vignetting, bokeh, they are all close.

B and C do more or less equally well at f1.4, where lens A totally fails.



No, it is not right to rate a lens less, just because it offers f1.4 extra, where the other lens stops at f2.

If two lenses perform equally, at the same settings, they should get the same rating. Price gets rated in "Price/Performance".

[/quote]



Exactly my point, Brightcolours. Of course one cannot overlook the problems at f1.4 (mushy corners), however, the Zeiss gives no image at all at f1.4, so it should get a lower rating, shouldn't it? <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/wink.gif' class='bbc_emoticon' alt='Smile' />
#19
Well, I reckon we've no solution here. Internally we discussed whether we should divide the optical rating into sub-components including resolution @ max. aperture and resolution @ 2 stops down. However, this wouldn't have solved the discussed issue either - the Zeiss would still be better (well, it is anyway).

If it helps I will delete the "substantially" in the verdict. The stars will not be changed though.
#20
[quote name='genotypewriter' timestamp='1292470691' post='5035']

But apparently the way you choose your "extreme borders" more extreme than mine accounts for the photozone results. But as I explained about... what is the meaning of the [url="http://www.opticallimits.com/lens-test-faq"]photozone borders[/url] if they account only for a very small area of the image frame?



http://www.flickr.com/photos/genotypewriter/5207301499

[Image: 5207301499_ed6d40dd74_m.jpg]

[/quote]



The point made there is of course perfectly valid, but I think many people are concerned that soft corners will ruin the apparent sharpness of an image overall. While this might be true when shooting brick walls and test charts, it is of course not so much of a problem in real world shooting, especially at wide apertures.

However, even while being aware of this fact, I can't help but admit that still a lens with soft corners puts me off a bit.





[quote name='genotypewriter' timestamp='1292470691' post='5035']

Honestly I can't see how a single number ("optical quality") can describe distortions at different focal lengths, vignetting at different apertures, absolute sharpness and its distribution at different apertures and focal lengths, LoCA and CA at different apertures and focal lengths, bokeh, etc. in a way that's fully consistent and comparable across lenses. So why give it at all?

[/quote]



As I wrote in my other post above, I think that any reduction of the properties of something as complex as a camera lens to only a few numbers can only inevitably be a lossy abstraction from the details. And yes, I don't see too much value in that "optical quality" rating -- it suggest that you can compare a superzoom with a fast prime, which you really can't, at least not with a single number.



Perhaps the rating should be transformed into an "optical quality for the primary purpose of the lens" or "relative quality to the competition". So a superzoom could receive five stars in the category of "flexible, bright-sunshine-only outdoor lenses for people that hate swapping lenses and should have bought a P&S in the first place" <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/wink.gif' class='bbc_emoticon' alt='Smile' />.

Seriously, for the purpose available-light UWA/WA (FF/APS-C) photography, the Canon is probably much better than the Zeiss.

So this is pretty much the "field quality" rating that recently appeared in some reviews. However, for many lenses, several "purposes" come into question, and the rating is often less dependent on the actual optical quality than on the general specs of the lens.



Of course we could have dedicated ratings for sharpness, optical defects (distortion, CA) etc. for those people that only look at the bottom of the last page of a PZ review, but this still leaves the question on how to judge zooms that rock at one end and and suck really hard at the other end of their range?
  


Forum Jump:


Users browsing this thread:
1 Guest(s)