Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
next PZ lens test report: Carl Zeiss Vario Tessar T* FE 16-35mm f/4 OSS ZA
#21
Quote:I'm no expert in this but could you really bump resolution figures without knowing if the lens is the limiting factor? And what does Photozone's "excellent line"mean (which the Canon lens never passes outside the center).

 

It sounds like you're really want the Canon to be better...
There is another factor to take into account. Photozone uses sharpening in the MTF images workflow, which skews results somewhat. The A7r is AA-filterless, which will mean it will introduce some fake sharpness which the MTF software can't discern. To, don't put too much weight in small differences in he results, there. 

 

Don't put too much weight on vignetting figures either, they are measures from JPEGs I believe, and tend to differ from different cameras. This is what PZ says about that in case of Nikon (D3x) vs Canon tests:

"We're performing our vignetting analysis based on (uncorrected) JPEGs straight from the camera. The JPG engine of the Nikon D3x features a rather flat gradation curve, thus has a moderate contrast characteristic, resulting in comparatively low vignetting figures - the corresponding Canon figures are roughly 40% higher due to the more aggressive default contrast setting."

http://www.opticallimits.com/nikon_ff/44...ff?start=1

Most probably, the Sony lens indeed has more vignetting. But without comparing them on the same body, in PZ's testing methods, we can't know for sure.

 

CA is significantly better, but the imatest software is not really very good at testing the CA (sometimes it gives clearly wrong results).
#22
I wouldn't be so quick to dismiss imatest.  Even Optikos' MTF bench has software that goes awry every 20-30 minutes, yet it gives some of if not the best MTF measurements you can get.  And it's about $250,000-$300,000 =)

 

1-2px max in the edge of the frame is pretty consistent with the sample images.

#23
Quote:Scythels IS an expert and has given a very detailed and rational reply to your question to which your mind-boggling reaction is to snidely accuse him of an irrational bias.  


You're certainly not an expert and the newbie here.  Pay attention on the site and RTF:

 

http://www.opticallimits.com/Reviews/overview

http://www.opticallimits.com/Reviews/canon-eos-full

 

At the very top:

<p style="margin-left:40px;">Please note that the tests results are not comparable across the different systems!

 

and

 

http://www.opticallimits.com/Reviews/lens-test-faq

Q: Are the figures comparable between cameras or different systems ?

<p style="margin-left:40px;">It depends on the similarities between the image sensor system. A sensor <i>SYSTEM</i><span style="color:rgb(51,51,51);font-family:Verdana, Helvetica, Arial, sans-serif;text-align:justify;"> contains the image sensor with or without micro-lenses, an IR filter, a low-pass (Moire) filter and the signal processing. As you can imagine the output quality is largely dependent on the whole chain on not just on the amount of megapixels. The different output quality between the Canon EOS 350D and the Olympus E-300 is a good example (despite a 8MP sensor). The tests are a good guidance for the lens quality as long as you compare the results WITHIN a test group (e.g. Canon).</span>

Q: Why are the quality ratings different from system to system ?

<p style="color:rgb(51,51,51);font-family:Verdana, Helvetica, Arial, sans-serif;margin-left:40px;">As mentioned above the lens quality is affected by the sensor "system". Every additional step in the pipeline decreases the output quality, specifically the low-pass filter in front of the sensor. Assuming you mount the same lens on different system its maximum resolution will vary according to the max. quality of the sensor system. There're also evolutions regarding the RAW converter quality so more recent system tests starts can benefit from this - e.g. Canon/Olympus RAWs are/were converted using ACR 3.2 whereas Pentax/Sony RAWs are/were converted via ACR 3.7 and there was an increase in converter quality with ACR 3.4). This must all be taken into account regarding the rating system.

 

It's inherent that the tests here are system based and therefore difficult to compare.  The accepted way to compare across tests is to compare the height of the bars on the graphs ex/very-good/etc, for example:

[Image: rating_eos5d.jpg]

 

A visible difference is when about the difference between each level in rating on this scale.  e.g., the scale about a difference of 500 LW/PH is visible.

 

Based on this, the Sony compared to the Canon is:

- Weaker wide open @ 35mm

- Corners don't catch up with the center (2 to 3 levels of difference), where the Canon is pretty even across the frame).  That said, the super high res sensors do tend to exaggerate the differences between the center and edges, but conversely the short flange difference on the E-mount tends to lead to weak corners.

- Not bad on the CA but the Canon is suberb

 

They are both very fine lenses and the differences are pretty small but the Canon edges ahead in a few places.
 

And you accuse me of being a newbie..

 

You say:

1) You really can't compare system results "number by number"

2) Trust protozone's scaling with poor-ex rating

 

I just see the Sony spend more "time" in the ex range than the Canon lens. So, either PZ or Scythels is wrong then?

And a lens is worse because it's sharp in the center and not as even?

CA (if kept reasonably low) is not relevant with todays SW correction. The same goes for distorsion.
#24
Quote:The change to resolution figures are to attempt to remove some of the sensor - the accuracy I would say is +/-10% as there really are no terrible sensors these days. 

 

One cannot compare the resolution of two lenses when the data is borne from a lens+sensor test combination.  If you put the 16-35/4L in front of the A7r and they were well matched to each other (meaning the cover glass on top of the A7r sensor is about 2mm thick) then you could directly compare the two. 

 

I did a bunch of rounding but changing things by +/-50lw/ph is pretty insignificant.  One would do the same compensations to upscale the 14-24 on the D3X to 36mp-equivalent figures.  The true bottom line is that in terms of resolution the performance is pretty similar, the canon is just much better at controlling CA and has less vignetting.  The sony has better distortion performance but neither is particularly offensive.
Thanks for the clarification.

But my question remains; how can you be sure that scaling actually holds and that the system is not limited by the lack of "resolution" of the lens?
#25
Quote: 

You say:

1) You really can't compare system results "number by number"

2) Trust protozone's scaling with poor-ex rating
Yes.

 

Photozone results include three large-scale variables:

 

* test chart

 

* imaging sensor

 

* lens

 

You are trying to extract the lens but the raw results include other variables, so you cannot tell anything from raw results. 

 

Tests exist which extract pure lens performance but they are extremely expensive and much more time consuming than IMATEST which is used by photozone (honestly, imatest is certainly good enough for any photographer's purposes). 


These tests fall into two categories that I am aware of but more may exist:

 

* interferometer (most accurate, +/-1.6nm precision)

 

* MTF bench testing

 

The former requires about $500,000 in equipment to begin testing and a further $20k or so for each focal length "class" you wish to add.  The latter has many variants, each having their own merits. 

 

The MTF bench I have experience with is Optikos' Lenscheck VIS system.  This puppy:

 

http://i.imgur.com/unzdZFK.jpg

 

http://i.imgur.com/BK2uvbd.jpg

 

This setup is really shitty, the lens needs to be better coupled to the bench... something like this: http://i.imgur.com/ni1Sm4B.jpg

 

The machine applies six compensations to its results in order to extract "pure lens performance".  The basic premise of the machine is that it takes a fiber optic light source and shoots it into a collimator and beam expander.  In between the two are two wheels, the test wheel and the filter wheel.  The filter wheel has a couple of options:

 

* open/clear

 

* 546nm bandpass - very narrow range green light

 

* green additive

 

* blue additive

 

* red additive

 

* IR Cutoff

 

+ two empty slots for user requested filters

 

The test wheel has a bunch of options, off the top of my head:

 

* crosshair

 

* 11um pinhole

 

* 29.5um pinhole

 

* 101um pinhole

 

* 300um pinhole

 

* 1mm aperture

 

* 3mm aperture

 

* 1951USAF chart

 

After the collimator you insert the Lens Under Test (LUT).  It images the light coming out of the collimator, behind it lies a microscope objective which images the spot formed by the lens.  Inside the machine a fourier transform of the spot is taken to break it into its various frequency components.  This allows the machine to measure resolution down to extremely fine spacial resolutions, far far finer than any photographic lens has any real resolution (over 1000lp/mm)

 

The machine applies compensation curves for:

 

* the microscope objective (user inputs microscope objective data, machine computes it out)

 

* the sensor behind the microscope (fixed, one of two sensors)

 

* the filter

 

* the test object/thing (typically the 11um pinhole)

 

* tilt of the image plane

 

* the optics of the collimator and beam expander.

 

Here is a test I did today with this machine of a customer's lens.  I can't give you any info on the lens because it is a prototype other than the fact that it is extremely high resolution, nearly diffraction limited in the center.  The diameter of the spot is about 1 micron, or 1/6th the width of the average full-frame sensor pixel.  This lens can handle the sensor in your phone flawlessly and far "outresolves" a FF or APS-C or 4/3 camera sensor.

 

https://gyazo.com/ce6f6bf560e3613cb30681ee3b9115a6

 

MTF bench tests are very far removed from results obtained with a camera.  All factors other than the lens are computed out and importantly there is no coverglass so results are not super correlated to what happens with a real camera.  Not only is such a tool fiscally unwise for PZ to use, it gets away from what the aim of this website is to do (provide lens tests "grounded" to photography). 

 

The poor-ex scale used by PZ is correlated to what the best lenses are capable of on that particular test camera.  The results loosely (+/- 10%) correlate to similar pixel count camera in similar situations (i.e long vs short flange) but must be adjusted to compare between systems.

 

This is how you would convert from MTF bench numbers to a camera sensor very loosely:

 

* calculate nyquist frequency of the sensor (5D2 - 5600/36/2 = 77.7lp/mm)

 

* discount by some "sensor loss" factor.  Anywhere from 60-80% is normal.  66 is a decent starting point (now at 51.33lp/mm)

 

* compare lens MTF to nyquist.  All lenses > 50% MTF at nyquist are approximately equal because that is the threshold where two airy disks may be distinguished (Rayleigh criterion)

 

Using this customer's lens for example, MTF is > 75% at 50lp/mm so the lens would resolve exceptionally well on the 5D2.  This is also in the corner and wide open... The lens in the center could handle about 135lp/mm of resolution so we are looking at a would-be 136*1.5*2*36 = 14688px long full-frame sensor, give or take.  This is the axial MTF data... http://gyazo.com/352613adf684f8ccc5cbec31e7552fa9

 

----

 

Quote: 

CA (if kept reasonably low) is not relevant with todays SW correction. The same goes for distorsion.
You lose resolution to correct CA and distortion, they're still very relevant. 

 

Quote: 

But my question remains; how can you be sure that scaling actually holds and that the system is not limited by the lack of "resolution" of the lens?
You can't, that's why you use margins.  +/-10% is arbitrary but is pretty safe.  The system is lens-limited whenever the image doesn't look "perfectly clear."  This is a situation where the system is absolutely sensor-limited (300/2.8L II on 1Ds3) http://www.the-digital-picture.com/Revie...&APIComp=0

 

This is a border case: http://www.the-digital-picture.com/Revie...&APIComp=0

 

There is some astigmatism in the corner (vertical resolution a bit weaker than horizontal).  This is a more common lens-limited case: http://www.the-digital-picture.com/Revie...&APIComp=0

 

This is a severely lens-limited case: http://www.the-digital-picture.com/Revie...&APIComp=0

 

 

 

The 16-35L is more or less neck-and-neck with the detector so scaling is reasonable - http://www.the-digital-picture.com/Revie...&APIComp=0
#26
Quote:I wouldn't be so quick to dismiss imatest.  Even Optikos' MTF bench has software that goes awry every 20-30 minutes, yet it gives some of if not the best MTF measurements you can get.  And it's about $250,000-$300,000 =)

 

1-2px max in the edge of the frame is pretty consistent with the sample images.
I am not talking about this specific test, in the past there have been results on PZ with much too low imatest CA "measurements" (I do not know the reason for why the test shows that, obviously. Perhaps it can be focus distance related?), where the images show way worse CA performance.
  


Forum Jump:


Users browsing this thread:
1 Guest(s)