1) You really can't compare system results "number by number"
2) Trust protozone's scaling with poor-ex rating
Photozone results include three large-scale variables:
* test chart
* imaging sensor
You are trying to extract the lens but the raw results include other variables, so you cannot tell anything from raw results.
Tests exist which extract pure lens performance but they are extremely expensive and much more time consuming than IMATEST which is used by photozone (honestly, imatest is certainly good enough for any photographer's purposes).
These tests fall into two categories that I am aware of but more may exist:
* interferometer (most accurate, +/-1.6nm precision)
* MTF bench testing
The former requires about $500,000 in equipment to begin testing and a further $20k or so for each focal length "class" you wish to add. The latter has many variants, each having their own merits.
The MTF bench I have experience with is Optikos' Lenscheck VIS system. This puppy:
This setup is really shitty, the lens needs to be better coupled to the bench... something like this: http://i.imgur.com/ni1Sm4B.jpg
The machine applies six
compensations to its results in order to extract "pure lens performance". The basic premise of the machine is that it takes a fiber optic light source and shoots it into a collimator and beam expander. In between the two are two wheels, the test wheel and the filter wheel. The filter wheel has a couple of options:
* 546nm bandpass - very narrow range green light
* green additive
* blue additive
* red additive
* IR Cutoff
+ two empty slots for user requested filters
The test wheel has a bunch of options, off the top of my head:
* 11um pinhole
* 29.5um pinhole
* 101um pinhole
* 300um pinhole
* 1mm aperture
* 3mm aperture
* 1951USAF chart
After the collimator you insert the Lens Under Test (LUT). It images the light coming out of the collimator, behind it lies a microscope objective which images the spot formed by the lens. Inside the machine a fourier transform of the spot is taken to break it into its various frequency components. This allows the machine to measure resolution down to extremely fine spacial resolutions, far far finer than any photographic lens has any real resolution (over 1000lp/mm)
The machine applies compensation curves for:
* the microscope objective (user inputs microscope objective data, machine computes it out)
* the sensor behind the microscope (fixed, one of two sensors)
* the filter
* the test object/thing (typically the 11um pinhole)
* tilt of the image plane
* the optics of the collimator and beam expander.
Here is a test I did today
with this machine of a customer's lens. I can't give you any info on the lens because it is a prototype other than the fact that it is extremely
high resolution, nearly diffraction limited in the center. The diameter of the spot is about 1 micron, or 1/6th the width of the average full-frame sensor pixel. This lens can handle the sensor in your phone flawlessly and far "outresolves" a FF or APS-C or 4/3 camera sensor.
MTF bench tests are very far removed from results obtained with a camera. All factors other than the lens are computed out and importantly there is no coverglass so results are not super correlated to what happens with a real camera. Not only is such a tool fiscally unwise for PZ to use, it gets away from what the aim of this website is to do (provide lens tests "grounded" to photography).
The poor-ex scale used by PZ is correlated to what the best lenses are capable of on that particular test camera. The results loosely (+/- 10%) correlate to similar pixel count camera in similar situations (i.e long vs short flange) but must be adjusted to compare between systems.
This is how you would convert from MTF bench numbers to a camera sensor very loosely:
* calculate nyquist frequency of the sensor (5D2 - 5600/36/2 = 77.7lp/mm)
* discount by some "sensor loss" factor. Anywhere from 60-80% is normal. 66 is a decent starting point (now at 51.33lp/mm)
* compare lens MTF to nyquist. All lenses > 50% MTF at nyquist are approximately equal because that is the threshold where two airy disks may be distinguished (Rayleigh criterion)
Using this customer's lens for example, MTF is > 75% at 50lp/mm so the lens would resolve exceptionally well on the 5D2. This is also in the corner and wide open... The lens in the center could handle about 135lp/mm of resolution so we are looking at a would-be 136*1.5*2*36 = 14688px long full-frame sensor, give or take. This is the axial MTF data... http://gyazo.com/352613adf684f8ccc5cbec31e7552fa9
CA (if kept reasonably low) is not relevant with todays SW correction. The same goes for distorsion.
You lose resolution to correct CA and distortion, they're still very relevant.
But my question remains; how can you be sure that scaling actually holds and that the system is not limited by the lack of "resolution" of the lens?
You can't, that's why you use margins. +/-10% is arbitrary but is pretty safe. The system is lens-limited whenever the image doesn't look "perfectly clear." This is a situation where the system is absolutely sensor-limited (300/2.8L II on 1Ds3) http://www.the-digital-picture.com/Revie...&APIComp=0
This is a border case: http://www.the-digital-picture.com/Revie...&APIComp=0
There is some astigmatism in the corner (vertical resolution a bit weaker than horizontal). This is a more common lens-limited case: http://www.the-digital-picture.com/Revie...&APIComp=0
This is a severely lens-limited case: http://www.the-digital-picture.com/Revie...&APIComp=0
The 16-35L is more or less neck-and-neck with the detector so scaling is reasonable - http://www.the-digital-picture.com/Revie...&APIComp=0