PROFICIENCY TESTING IN CLINICAL LABORATORY

PROFICIENCY TESTING IN CLINICAL LABORATORY

What Is Proficiency Testing?

Proficiency testing (comparative testing) is an important way of meeting the requirements of ISO/IEC 17025 in the area of quality assurance of laboratory results. It is also mandated by accreditation bodies that laboratories participate in proficiency testing programs for all types of analyses undertaken in that laboratory, when suitable programs exist.

Proficiency testing involves a group of laboratories or analysts performing the same analyses on the same samples and comparing results. The key requirements of such comparisons are that the samples are homogenous and stable, and also that the set of samples analysed are appropriate to test and display similarities and differences in results.

One of the best ways for a laboratory is to monitor its performance it to participate in proficiency testing scheme. Proficiency testing is a type of inter-laboratory comparison exercise in which samples are circulated to the participating laboratories, results are then collated and evaluated centrally. Each laboratory is then informed of its performance relative to the other laboratories in the scheme and relative to either true or consensus result for the sample. Performance is usually quoted in terms of the number of standard deviations between the achieved value and the consensus or true value. In some schemes participants are given a pass /fail rating.

Because of the requirement of the laboratory accreditation bodies from all over the world more and more laboratories participate in proficiency testing schemes. Since in all fields of testing and calibration, the satisfactory performance by an applicant laboratory is the pre-requisite from the accreditation bodies, participation in PT is increasingly being used by laboratories. Moreover PT is being increasingly used by laboratories to demonstrate their competence and thus it is becoming more popular and acceptable to laboratories. Laboratories do face a problem as there are not adequate numbers of PT schemes to meet their need. While many laboratory accreditation bodies organize their own PT schemes many accreditation bodies do use PT schemes or other inter-laboratory comparison schemes being operated by others to evaluate the competence of their accredited laboratories.

To cope up with the increasing requirement for PT schemes or inter-laboratory comparison many commercial third party PT and inter-laboratory comparison providers are entering the market to fill the gap and thus meet the growing need. This has led to the need for maintaining a common standard for operating the inter-laboratory comparison programs. This need is met by operating the inter-laboratory comparison schemes as per the guidance given in ISO/IEC Guide 43 part I. which is currently being revised as a standard, ISO 17043.

Some accreditation bodies have started giving accreditation to PT/ Inter-laboratory comparison program providers based on Guide 43.

Why is PT important?

PT is important because it is a tool the laboratory can use to verify the accuracy and reliability of its testing. Routine reviews of PT reports by the laboratory staff and director will alert them to areas of testing that are not performing as expected and also indicate subtle shifts and trends that, over time, would affect their patient results.

The following text describes an overview of the usefulness of externally provided proficiency testing programs with respect to meeting the objectives described above:

Objective 1: Ensuring an analyst can reproduce the same result on the same sample.

Examples of ways to meet that objective:

Analyst can perform duplicate tests (from sample preparation stage) on the same sample at periodic intervals.

The results from duplicate tests can be assessed. (A clause in the laboratory quality manual indicates the acceptance criteria of duplicate test results.) If the sample is known to be stable, the analyst can re-test the sample after a period of time to ensure they get a result within the same range.

Can this be done with an external proficiency testing program?

Yes, but it is just as important to check analyst reproducibility on a day to day basis.

Requirements:

Define a schedule where these things will be assessed and reviewed.

Define acceptance criteria for the results of observations.

Act / investigate any non-conformances.

Difficulties:

The analyst may have an expectation of the result they should obtain, and may be difficult to avoid performing a test to 'favour' the desired result.

Objective 2: Ensure that analysts within a laboratory would report the same result for the same test on the same sample.

Examples of ways to meet that objective:

Have the group of analysts test a larger quantity of the same sample at the same time and follow the test through to its conclusion. Such samples can be external PT samples, batches of material purchased from an external supplier with a certificate of analysis providing details of homogeneity testing and results, or samples made (or saved from testing work) in-house.

Can this be done with an external proficiency testing program?

Yes, but the sample volume available may not be sufficient to have every participant test the same sample from their own sample preparation step (usually a weigh out). It is possible to check consistency of diluting and identification from external PT program samples. It may be necessary to break up the testing to into separate tasks (for example sample preparation and weight-out, diluting the sample and plating, confirming the results). If this is done then, weigh-out and sample preparation can be compared on samples where sufficient volume of sample exists, and the counting/confirming steps can be performed on most if not all external proficiency testing program samples. If the sample volume is sufficient, however, it is not a problem to use samples from external programs for the whole task.

Requirements:

Define a schedule where these things will be assessed and reviewed.

Define acceptance criteria for the results and observations. There should be criteria for comparisons of internally generated data, and it is useful to look at consistency of internally generated data separate to an external PT report. If internally generated data is consistent, but does not agree with external PT results, it could be a sign that there is a systematic error/difficulty within the laboratory that is quite separate from the abilities of the staff. (This is also one of the reasons it is necessary to compare results internally.)

Act / investigate and non-conformances.

Note: It is not necessary to compare all of the analysts in a laboratory at the same time. It is important to ensure, however, that every analyst has been compared with at least one other analyst. If using external PT programs for this part of the requirement, it will be necessary to roster inter-staff comparisons to ensure everyone is included for every test over a period of time.

Difficulties:

If using samples made or selected in-house, there may be an expectation of the result.

Samples made in-house may yield inconsistent results for reasons other than operator practices. (For example, perhaps the distribution of the analyte is uneven or 'didn't take' when samples were spiked.) It may end up being necessary to perform large amounts of additional test work to establish the reasons for differences in results.

Staff may also feel 'under-pressure' to report the same result as their co-workers, and this may lead to collusion. Staff must be encouraged to provide a very objective opinion of their analysis, and this may be more successful if the atmosphere within the laboratory is nurturing.

Objective 3: Ensuring a laboratory report would reflect the results obtained by most other laboratories.

Examples of ways to meet that objective:

Participate in external proficiency testing schemes.

Organise inter-laboratory trials with other laboratories.

Can this be done with an external proficiency testing program?

Yes.

Requirements:

Define a schedule where these things will be assessed and reviewed.

Define acceptance criteria for the results of observations.

Act / investigate any non-conformances.

Difficulties:

It is difficult to find program providers that cover all the tests in the range performed by the laboratory

Proficiency Testing Guidelines

Each section should ensure that proficiency sample testing is performed in the same manner as patient sample testing. The laboratory integrates all proficiency testing samples within the routine laboratory workload, and those samples are analyzed by personnel who routinely test patient/client samples, using the same primary method systems as for patient/client/donor samples. Primary records related to PT and alternative testing should be retained for two years unless longer retention is required for specific analytes. If the laboratory uses multiple methods for an analyte, proficiency samples should be analyzed by the primary method. The educational purposes of proficiency testing are best served by a rotation that allows all technologists to be involved in the proficiency testing program.

Proficiency testing records should be retained and can be an important part of the competency and continuing education documentation in the personnel files of the individuals. Inter-laboratory communication about proficiency testing samples before submission of data to the proficiency-testing provider is prohibited. Proficiency survey sample will not be sent to a reference laboratory for additional testing or confirmation of results. This prohibition takes precedence over the requirement that proficiency samples be handled as patient specimens if referral lab confirmation is the laboratory’s policy.

Primary records related to PT and alternative assessment testing are retained for two years (unless a longer retention period is required elsewhere in this checklist for specific analytes or disciplines). These include all instrument tapes, work cards, computer printouts, evaluation reports, evidence of review, and documentation of follow-up/corrective action.

Key Elements in Developing a Proficiency Testing Plan

When developing a Proficiency Testing Plan for your laboratory, the first element to consider is the length of the proficiency testing cycle for your laboratory. The laboratory should plan to participate in enough proficiency testing activities to cover the technical scope of the laboratory. If your laboratory is not accredited, you can select any length of time for the cycle that suits you, but if you are applying for accreditation, it is advisable to align your proficiency test plan to the requirements of the accreditation body to which you are applying. If you are an accredited laboratory, the maximum cycle time that you can select is four years. It is important to note that the laboratory may select a smaller period of time for the proficiency testing cycle. A shorter cycle may be necessary for laboratories with a small technical scope of accreditation to meet additional accreditation body requirements, or may be desirable if the laboratory is interested in a more comprehensive quality program.

For accredited laboratories, beginning with selecting the maximum cycle time of four years minimizes cost to the laboratory while still ensuring compliance to applicable requirements The next element to consider is the definition of “Major Discipline” and “Major Sub-Area” as it applies to your technical scope of accreditation. Again, if you are not an accredited laboratory, you can define these terms yourself. One possible source of guidance for defining these terms (which can be used for both laboratories and accreditation bodies) can be found at the Bureau International des Pois et Mesures (BIPM) Key Comparison Database (KCDB). This information can be located at http://kcdb.bipm.org//. This database (in appendix C, Calibration and Measurement Capabilities) defines Major Disciplines and Major-Sub Areas used in the support of the Mutual Recognition Arrangement (MRA). If your accreditation body has defined major disciplines and major sub-areas, you must align your Proficiency Testing Plan to these definitions. If you accreditation body has not defined major disciplines and major sub-areas, please consult with your accreditation body for further guidance, preferably in writing. Lastly, some accreditation bodies require that the proficiency test plan must:

l  include information such as an explanation of why proficiency testing may not be possible for certain major sub-areas

l  address the laboratory’s process for timely submission of proficiency testing results to the accreditation body in the required timeframe

l  Complete and implement corrective actions for measurement areas that have received unsatisfactory proficiency testing results and submit the information to the accreditation body in a timely manner.

Documenting the Proficiency Test Plan

Once the laboratory technical scope has been divided into major disciplines and major sub-areas and minimum annual proficiency testing requirements are understood, the laboratory may begin to schedule proficiency testing activities. The proficiency test may be either from a commercial provider, developed internally, or completed through a cooperative effort such as those provided by NCSL International, but the proficiency test must meet the applicable requirements of the accreditation body, such as the requirement that the proficiency test be designed in accordance with ISO Guide 43-1 (1997), Proficiency Testing by Interlaboratory Comparisons – Part 1: Development and operation of proficiency testing schemes.

The scheduling of proficiency tests to be accomplished during the proficiency testing cycle, proficiency tests already accomplished, and other previously mentioned elements should be contained in a single, easy to use document. There are no formatting requirements for the document, so the laboratory may develop a document that meets the requirements of the accreditation body (or their own requirements if the laboratory is not accredited). It is also permissible for the laboratory to work with consultants or proficiency test providers (some of which will develop a Proficiency Testing Plan at no charge and include price discounts to the laboratory) in the development of their Proficiency Test Plan. It is important to understand that despite whoever develops the Proficiency Testing Plan, it is the ultimate responsibility of the laboratory to ensure that the data is correct and the plan is carried out.

General Proficiency Testing Procedures
Clinical Sections that are required to perform proficiency testing should establish procedures for testing survey samples / cases. The following general procedures should be followed:

Proficiency Survey samples are to be handled in the same manner as patient samples as much as possible. If accessioning survey cases in the Laboratory Information System (LIS) is possible and is important for analysis on instrumentation, cases should be accessioned in Softpath. Proficiency samples should be tested in accordance with the associated instructions provided for each survey set. The laboratory should adhere to the CAP-, or other survey vendor, provided testing instructions for each survey submitted for analysis. These instructions address proper specimen handling, storage, testing limitations or special instructions, reporting criteria, and instructions for submitting results. Testing should be completed by the provider’s due date. This is to ensure meeting grading criteria, but also to ensure specimen integrity. 

Unsatisfactory Proficiency Testing (PT) Performance

Unsatisfactory performance for 1 or more analytes on an event will receive a PT Exception Summary (PTES) report. Laboratory must:

Ø  Investigate problem

Ø  Determine cause

Ø  Implement corrective action

Response to CAP Accreditation Program is not required. Unsatisfactory performance on 2 of 3 PT events, laboratory will receive a PT Exception Summary (PTES) report. Laboratory should:Suspend testing or implement plan of correction. Laboratory Accreditation Program must approve the corrective action prior to restarting testing

Investigation of failed proficiency survey

v  Review for Clerical Errors

v  Review for Technical Issues

u  Reconstitution, pipetting, dilution errors

u  Specimen mix-up

u  Improper specimen handling

u  Incorrect instrument set-up

u  Failure to follow test kit or instrument instructions

v  Potential Methodologic Issues

u  Mechanical difficulties (check maintenance and quality control records)

u  Instrument software issues

u  Calibration issues

u  Inadequate reagent performance

u  Instrument failure

Correct failure and repeat testing once issue is identified and resolved. All corrective actions and retesting should be reviewed by the section’s Medical Director or designee.

Ungraded Proficiency Challenges

Ungraded PT Surveys require internal assessment of performance, documented evaluation and approval by the Medical Director or designee. Reasons for ungraded PT challenges include:

v  the laboratory submitted its results after the cut-off date,

v  the laboratory did not submit results,

v  the laboratory did not complete the result form correctly (for example, submitting

v  the wrong method code or recording the result in the wrong place)

v  lack of consensus.

An ungraded PT challenge will be handled as a failed survey and require internal analysis against survey results utilizing CLIA limits. Results should be reviewed by the section’s Medical Director or designee.

Conclusion

In order to develop a successful PT plan for your laboratory, it is important to understand the documented PT requirements from your accreditation body. It is also helpful to review the quality system requirements that your accreditation bodies must comply to in order to have a better understanding of what drives the PT policies of your accreditation provider. A well developed and documented PT schedule will allow better oversight and management of PT requirements, spreading tests out over the entire technical scope of the laboratory instead of over-concentrating in one particular area. The time and resources associated with proficiency testing is generally associated with overhead costs of operating a laboratory, so it is usually desirable to minimize these costs. A well constructed PT plan will allow the laboratory to meet quality and accreditation requirements and verify the measurement processes associated with the laboratory technical scope at a minimum of expense.