Evaluation of the proficiency testing

Cochran outlier test

The dispersion of the individual values of laboratories are analysed with the outlier test according to Cochran:

Total mean value

The total mean value Ck is then calculated, which generally serves as the reference value during the evaluation. At PT schemes with low number of participants the mean concen¬tration found while analysing quality control samples can be defined as target concentration. This value is used for further statistical analyses such as the total standard deviation Sk, Grubbs test and z-score. Where certified materials are employed, the concentration stated on the certificate is defined as the reference value. The total mean value and total standard deviation are calculated from the data.

Grubbs test for outliers

It is generally assumed that the body of data is subject to normal distribution. The outlier test is then performed at the 95% level (both sides α = 2.5%). The elimination of outliers ulti¬mately produces a mean value which closely approximates to the "true value" of the sample.

In the Grubbs outlier test, the procedure is for all laboratory mean values to be calculated in the first instance and the total mean value and total standard deviation then to be determined from them. The difference is then formed between the individual mean values and the refer-ence value/total mean value, the individual mean value with the greatest difference to the reference value/total mean value being substituted marked * in the formula of the Grubbs outlier test. The test value t is compared to the tabular value for a significance level of α = 2.5% when considered from both sides (see above).

If it is confirmed as an outlier, it is removed from the body of data, and the total mean value and total standard deviation recalculated. The difference between the new total mean value/ reference value and the individual mean values is calculated and the greatest difference substituted in the Grubbs formula. If this value is also confirmed as an outlier, it is likewise removed. The procedure is repeated until no more outliers can be eliminated.

Should the data exhibit such strong variation that no outliers can be identified by means of the Grubbs test, they are defined by the deviation from the reference value. An outlier is recognized as such in this case when the deviation > ± 36%. Where the data material is heterogeneous, the limit can be increased to 50%, or identification of the outliers not be performed.

z-Score analysis

All individual mean values are considered in the z-score analysis, including the values identified as outliers in the Grubbs test.

The z-score of all participating laboratories, including those eliminated as outliers, is calculated with the aid of the outlier-free mean value and the outlier-free standard deviation.

The z-score can be regarded as the quality characteristic of the mean value of the individual laboratories (see below).

The z-score analysis is based upon the following formula:

A permissible deviation of 10 % is assumed for s during analysis of the PT scheme. Where data material exhibits strong scatter, the permissible deviation may be increased to up to 15 %. The individual results are then evaluated as shown below:

• |z| ≤ 1 Good result
• 1 ≤ |z| ≤ 2 Satisfactory result
• 2 ≤ |z| ≤ 3 Questionable result
• 3 ≤ |z| Extremely questionable result

A result for which |z| ≤ 2 is deemed satisfactory, i.e. the PT is deemed passed. Should the |z| score exceed 2, review of the analysis method employed is advisable.

The z-score can be used to determine further statistically important parameters, which shall not however be considered in any greater detail at this point.

Evaluation Report, Confirmation of Participation and Certificate

After the evaluation has been completed, each participating measuring station receives a notification of the results. In the online portal, the participants are provided with an evaluation report with the confirmation of participation and an individual certificate for download.

The evaluation report contains the following information:

• the confirmation of participation
• tabular and graphical presentation of the entire analysis results and the outliers
• tabular and graphical presentation of the z-score analysis
• list of the analysis methods employed by the participants
• alphabetical list of all participants

The integrated confirmation of its participation contains the following information:

• laboratory number
• own laboratory result and z-score
• mean value of the samples following elimination of outliers (Grubbs test)
• reproducibility standard deviation
• reference value (mean value of the control samples for quality control)

The certificate (PDF, 656 kB, non-accessible) is produced individually from each participant's results. The certificate is produced in accordance with the following rules:

• All substances are stated on the certificate for which, averaged over all samples, a z-score ≤ 2 was attained.
  Calculation for the individual substance:
  Of three samples, at least two must yield individual z-scores < 2; a z-score distribution such as "2.9, 2.8, 0", which purely mathematically would yield a mean value of 1.9, results in participation for the substance being deemed unsuccessful.
  No single sample may yield a z-score > 3.
• Participation in the PT is deemed successful when a z-score ≤ 2 (see above) is obtained for over 50% of the analysed substances.
• Should z-scores ≤ 2 be obtained for fewer than 50% of the analysed substances, the certifi¬cate merely confirms participation in the PT.