Developments of interlaboratory comparisons on pressure measurements in the Philippines

The National Metrology Laboratory of the Philippines, Industrial Technology Development Institute under the Department of Science and Technology (ITDI-DOST) offered first of its kind interlaboratory comparison in pressure measurement in the country following a demand survey identifying the gaps in proving the competency of local calibration laboratories. This paper presents the development in implementing the two interlaboratory comparisons in the Philippines with six years interval using the same pressure artefact. With its many objectives, the program also aimed to provide these laboratories access to proficiency testing (PT) to comply with the ISO/IEC 17025 requirements. Comparatively, while both are considered successfully implemented, the improved awareness and commitment to quality of the participants and the enhanced competency of the pilot laboratory in implementing such activity are a few of the enumerated factors instigating the increase in the number of participants as well as those obtaining satisfactory results in the latter PT. The interlaboratory comparison schemes offered aimed at sustaining the demands of metrology stakeholders and continuously develop this service to support further progress in the calibration and measurement capabilities of local laboratories in the country.


INTRODUCTION
The National Metrology Laboratory (NML) of the Industrial Technology Development Institute (ITDI) under the Department of Science and Technology (DOST) conducted interlaboratory comparisons in the field of hydraulic pressure measurement among the local calibration laboratories in the Philippines. The provision of interlaboratory comparison, otherwise known as proficiency testing (PT), is a program to strengthen the NML relationship with the said laboratories in establishing scientific metrology in the country.
This PT program aims to [1], [2]: (a) determine the technical capabilities and performance of the laboratories; (b) assess the reliability of their measurement results and validate their calibration measurement capabilities; (c) disseminate a harmonized and validated calibration procedure; (d) demonstrate metrological equivalence to the NML and most importantly, (e) provide access to interlaboratory comparisons for their compliance to ISO/IEC 17025 [ [3]] requirements. The Pressure Standards Section (PSS) of the NML acted as the program coordinator and reference laboratory, accredited under the terms of ISO/IEC 17025:2005 (2017 at present). The PSS was responsible for providing the artefact, the reference value and its measurement uncertainty, the monitoring of the program as a whole, and preparation of written reports for the two intercomparisons being compared; one was conducted in 2010 while the other was in 2016 The following will be the outline of this paper. In Section 2, the comparison process will be discussed followed by the measurement results of the participants. Discussion of these results commences in Section 4, and the evaluation of both the PTs is compared in Section 5. Finally, the concluding section summarizes the major accomplishment and development based on these experiences.

ABSTRACT
The National Metrology Laboratory of the Philippines, Industrial Technology Development Institute under the Department of Science and Technology (ITDI-DOST) offered first of its kind interlaboratory comparison in pressure measurement in the country following a demand survey identifying the gaps in proving the competency of local calibration laboratories. This paper presents the development in implementing the two interlaboratory comparisons in the Philippines with six years interval using the same pressure artefact. With its many objectives, the program also aimed to provide these laboratories access to proficiency testing (PT) to comply with the ISO/IEC 17025 requirements. Comparatively, while both are considered successfully implemented, the improved awareness and commitment to quality of the participants and the enhanced competency of the pilot laboratory in implementing such activity are a few of the enumerated factors instigating the increase in the number of participants as well as those obtaining satisfactory results in the latter PT. The interlaboratory comparison schemes offered aimed at sustaining the demands of metrology stakeholders and continuously develop this service to support further progress in the calibration and measurement capabilities of local laboratories in the country.

COMPARISON PROCESS
The two intercomparisons followed an almost similar process; differences are, however, emphasized in this paper, specifically the contributing factors that affected the results of the two schemes. It is herein referred to the first PT scheme as 2010 and the second PT scheme as 2016 throughout the discussion.
The interlaboratory comparison program was designed as a cycle where the NML, as a reference or pilot laboratory, calibrates the artefact at the program's beginning, middle, and end.
One main difference between 2010 and 2016 is the conduct of a preparatory workshop before the start of the PT program. This workshop proved essential in getting to know each laboratory's capabilities, standards, and procedures they follow in calibrating pressure gauges. Also, in this workshop, an agreement between the participants and the NML was reached, fulfilling the objective of disseminating a harmonized and validated calibration procedure and the identification of limitations of the participants and the PT program in general.

Participants
Participants in the two PTs are local calibration laboratories with NML as the reference lab. In 2010, five (5) participants were all private laboratories in Metro Manila, the capital of the Philippines. In 2016, the 16 participants comprised private and government laboratories, including some DOST regional metrology laboratories from different provinces in the Philippines.

Artefact
The artefact (Figure 1and Table 1) used in 2010 and 2016 is a Bourdon-tube-type pressure gauge. It was noted that in 2010, two artefacts of different ranges were measured by the participants, but in this paper, comparisons will be based on only one artefact used on both PTs, and the description is as follows: The artefact was subjected to initial and subsequent characterization before the PTs and maintained a regular interval of calibration and intermediate checks when not used as an artefact.

Calibration method
The participants were asked to calibrate the artefact through direct comparison to their standard. In 2010, each participant used the typical calibration procedure in their laboratory, which means their laboratory-developed method. Meanwhile, in 2016, after the earlier mentioned preparatory workshop, it was agreed among the participants to follow an international guideline, the DKD-R 6-1 Calibration of Pressure Gauges [4]. which not only guided the calibration procedure but the computation of measurement uncertainty as well. The NML uses the said guideline.

Measurement scheme
The NML chose a measurement scheme to monitor the artefact's metrological quality throughout the PT process.
In 2010, the artefact was calibrated before and after a trip to a participant, as shown in Figure 2. It was hand-carried to and from each participating lab by its representative.
In 2016 however, since there were more participants than in 2010 and some were outside Metro Manila, the NML calibrated the artefact before and after a group of participants, usually 2 to 3 labs, strategically chosen based on location so the sending back and forth to the NML of the artefact are made most efficiently. Figure 3 shows this scheme.

Report of the participants
In 2010, the participants were only asked to submit the filledout NML-provided measurement datasheet and the calibration certificate they usually issue to their customers. Other information, such as the uncertainty budget, was only known when required by the NML. This exchange of information could have been more efficient, consequently causing a delay in data evaluation and publishing of the final report. Hence, the NML changed this practice in 2016. The participants should submit documents such as the measurement datasheets, a copy of the calibration certificate of their standard proving valid traceability, their usual calibration report, and the uncertainty budget. This change improved the data evaluation process for the NML, making it easier and faster since information transparency was present from the beginning.

Reference values
In both PTs, the reference values used in evaluating the normalized error ( n ) for each participant were based on the values nearest the participant's reported results, either before or after the calibration of the reference laboratory. In 2016, this result was given to participants in an Interim Report, showing only the specific participant's results compared to NML's. This interim report was beneficial for participants having to prove their competence to technical peers during their assessment while the intercomparison is not yet completed. It should be noted, however, that in the final report, the reference values reflected are the weighted average of all the measurement results of the NML.

MEASUREMENT RESULTS
The measurement results of participating laboratories are evaluated using the earlier mentioned normalized error or the n ratio [5], calculated using the equation: where lab is the measured value of the participating laboratory,  In 2010, four out of the five participants were able to calibrate the artefact in its full measuring range, while one participant did not submit a result in the last two highest points. Meanwhile, in 2016, all the participating laboratories were able to calibrate the artefact as a whole. Table 2 and Table 3 show the value of the participants in 2010 and 2016 respectively.

DISCUSSION OF RESULTS
The laboratory's satisfactory performance was determined when | n | ≤ 1 in all the prescribed calibration points. In 2010, only 95 % of the total calibration points with | | ≤ 1 were required to be considered satisfactory laboratory performance, but this was later corrected to 100 % of the calibration points in 2016. Some participants interpreted that the 95 % confidence level in the uncertainty budget estimate may also be applied in the inter-laboratory comparison result, thus, assuming that they don't need to perform well in all the measurement points since there is a 5 % margin of error. The NML had to explain the rationalization that the 5 % margin of error cannot be tolerated in the measurement procedure since the basic requirement of the PT was to calibrate the artefact as a whole and any doubt in their procedure must well be accounted for in their uncertainty budget and not on their measurement value. Also, a 95 % satisfactory performance will not be possible in the calibration points prescribed since any one point was already 10 % of the artefact's ten calibration points.
It is, however, emphasized that the laboratory's performance is considered satisfactory only through the n score.
Accordingly, so as not to abuse the said acceptance criteria, it was later suggested that a limiting value for uncertainty is to be set, and most of the participants followed it in 2016.
For illustration purposes, Figure 5 and Figure 6 show the participant's deviation from the reference value with its corresponding uncertainties in the non-zero minimum and the maximum calibration points, respectively. The 2010 participants were represented by blue markers and labeled alphabetically (LAB A to LAB E), while participants in 2016 were of green markers and were labeled with numbers (LAB1 to LAB16).
Comparing the participants' performance in the two PTs, all the participants performed satisfactorily in the minimum calibration point in 2010 as opposed to those in 2016, with 1 participant whose value already lies beyond the limit if not with its uncertainty. Differences in the computed uncertainty values depended mainly on the standard they used, primarily a digital pressure gauge; only a few used a pressure calibrator and a deadweight tester. Moreover, in 2010, while the NML prescribed a guideline for measurement uncertainty calculations, most participating laboratories estimated the expanded uncertainties using their laboratory procedure and technique with the notion that declaring a low uncertainty means better performance.  Contrastingly, in 2016, all the participating laboratories followed the agreed guideline on procedure and uncertainty estimates, with the lower limit as the resolution of the artefact. This agreement ensured that measurement uncertainties were not over or under-estimated. A sample calculation of the recommended minimum components of measurement uncertainty of the pilot laboratory is shown in Table 4.
The minimum components of measurement uncertainty were enumerated as the factors coming from the standard, the height difference between the standard and the artefact or the so-called hydrostatic pressure effect, the artefacts' resolution, zero deviation, hysteresis, and repeatability. The expanded uncertainty, as earlier mentioned, was evaluated with a 95 % level of confidence with the coverage factor, k = 2.
The uncertainty from the standard was taken from the most recent calibration certificate of the artefact before the start of the interlaboratory comparison. The uncertainty due to the hydrostatic pressure effect was evaluated but was found negligible due to minimizing, if not eliminating, the height difference of the reference pressure points of the standard and the artefact during calibration. Similarly, the uncertainty due to zero deviation was also found negligible due to the behaviour of the artefact. The major contributor to uncertainty in the reference value was the resolution of the artefact, which was agreed to be the limiting factor for uncertainty evaluation for all the participants, including the pilot laboratory. This decision accommodated the limited accuracy capability of some participants, which caters mainly to industrial calibrations. The uncertainty due to hysteresis and repeatability are the maximum values among the measurement performed by the NML. The NML used these factors in both the 2010 and 2016 PT.

EVALUATION OF COMPARISONS
Comparing the two PTs as a whole, defined factors affecting the performance of participating laboratories are summarized in Figure 7.
The earlier mentioned preparatory workshop, not done in 2010 but conducted in 2016, proved to be one key factor that led to the increased satisfactory performance of participants. In 2016, four laboratories could not attend this workshop; however, two laboratories asked for details after the event and strictly followed the agreed procedure, 1 lab with a satisfactory result but did not follow the agreed uncertainty limit, and one did not perform satisfactorily. In both PTs, the main objective of the laboratories' participation was to fulfil the ISO/IEC 17025 requirement for PT participation. However, the urgency of this requirement was only partially realized by the 2010 participants since the accreditation to ISO/IEC 17025 was reasonably new in the country during that time.
Moreover, most 2010 participants have inexperienced or untrained personnel and need to familiarize themselves with the calibration method used by the NML. Contrastingly, in 2016, most laboratories had trained personnel and had proven competencies in scopes other than the pressure field. Their submitted results showed that knowledge of estimating uncertainty budgets also significantly improved through training. The latter PT also showed an enhanced selection of standard and upgraded facilities by the laboratories.
It is most probable that the 2010 PT was considered as a test run by the participating laboratories, with two labs participating satisfactorily on both, 1 lab that improved in 2016, 1 that still unsatisfactorily performed, and one that did not continue to be a calibration laboratory, this is also the participant that did not complete the measurement due to inappropriate standard. The 2016 participants, on the other hand, are mostly maintaining their  ISO/IEC 17025 accreditation already or are in the process of acquiring their certification in the pressure scope, supported by this intercomparison. In both PTs, the NML recommended that all the laboratories with unsatisfactory performance review their calibration method and uncertainty budget analysis, investigate sources of error leading to unsatisfactory results, and initiate corrective actions. The NML, on the other hand, as the reference laboratory, continuously improves as a PT provider learning from experience, from the handling of the artefact to the analysis of data that is most appropriate to all participants. Consequently, the NML extended its PT offering regularly, with different pressure ranges and other fields of measurement; also, the conduct of the concluding workshop was planned for the succeeding PTs. Furthermore, coordination with the local accreditation body as the channel to know the demands for PT in the country for NML's plan on PT provision, and in return, the laboratories are made aware of the NML PT offerings. Availability of artefact is still the most significant limitation of the PT provision but was hoping to be resolved to cope-up with ongoing demands on PT.
Currently, the laboratories' commitment to quality, supported by courses and training which are not only technical but also in the quality management systems, are contributors to the laboratories' handling of intercomparison prominent to satisfactory performance. The participants have become more aware of good laboratory practices and are encouraged to continuously improve through refresher and new Metrology awareness courses.

SUMMARY AND CONCLUSION
The intercomparisons, 2010 and 2016, are two independent PTs and are generally considered successful in terms of results, coordination, and the experience gained by the participants and the reference laboratory. Measurement results revealed the calibration and measurement capabilities of each participating laboratory. Based on the requirements of ISO/IEC 17043:2010, the performances are mostly satisfactory, in terms of the n values, especially in the 2016 PT. This also indicates that the measurement practices of these participants significantly improved and are aligned and complying with an internationally validated method. The PT schemes offered by the NML will be of continuous improvement to support further progress of the local calibration laboratories in the Philippines.

ACKNOWLEDGEMENT
Funding support for DOST and the Laboratory Proficiency Evaluation Program (LPEP) of the NML, ITDI-DOST is gratefully acknowledged. The LPEP was initially funded by the Philippines Council for Industry, Energy and Emerging Technology for Research and Development (PCIEERD) making it possible to acquire the artefact used in these intercomparisons.