A method to consider a maximum admissible risk in decision-making procedures based on measurement results

correctness


INTRODUCTION
Measurement results are very often used as input elements in decision-making procedures, which represent the core element of conformity assessment.This is a very critical task in many fields, from the industrial one, where conformity of a product's feature to given specifications must be assessed, to environment protection, health, legal and forensic ones, where decisions are generally related to checking that the presence of a substance (a pollutant, a drug, etc.) or the error of an instrument does not exceed a given threshold or maximum admissible limit.
Most decisions -if not all of them -are taken by comparing a measurement result with a threshold or a range of admissible values, where the threshold, or the upper and lower limits of the range, are given as simple quantity values [1].Then, according to where the measurement result is located with respect to the threshold or the range, a decision is taken on whether conformity can be declared or not.
If measurement uncertainty is not considered, or if it can be assumed to be negligible, this decision can be easily taken by comparing two numerical values: the measured value with the threshold (as shown in Figure 1) or the measured value with the upper and lower limits of the range.Figure 1 shows that, in such a situation, the decision is apparently taken with no risk of being wrong.
However, even if measurement uncertainty has been evaluated and found to be negligible, a risk of wrong decision still exists, because it is widely recognized [2] that "when all of the known or suspected components of error have been evaluated and the appropriate corrections have been applied, there still remains an uncertainty about the correctness of the stated result, that is, a doubt about how well the result of the measurement represents the value of the quantity being measured".It is also well-known, according to the GUM [2], that in many applications "it is often necessary to provide an interval about the measurement result that may be expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the quantity subject to measurement.Thus, the ideal method for evaluating and expressing uncertainty in measurement should be capable of readily providing such an interval, in particular, one with a coverage probability or level of confidence that corresponds in a realistic way with that required".
When measurement uncertainty is taken into account, again a decision about conformity can be readily taken if the coverage interval is completely below or above the threshold (Figure 2).On the other hand, the situation represented in Figure 3 appears to be quite critical, since the threshold falls inside the coverage interval representing the fraction of the distribution of values that could reasonably be attributed to the quantity subject to measurement (the measurand).
This means that there is a probability that some of the values that could reasonably be attributed to the measurand might be greater than threshold t, even if the measured value x ̅  is lower than the threshold.This also means that, if conformity shall be assessed when the measurand is lower than the threshold, a risk exists of declaring the measurand conforming while it is not, and this risk can be evaluated starting from measurement uncertainty [3].
Conformity assessment involves, therefore, a decisionmaking process affected by uncertainty.Such a problem has been widely covered in the literature [4]- [6], mostly by taking epistemic uncertainty into account [7].However when the input elements to a decision-making process are measurement results, uncertainty takes a well-defined meaning, defined by the VIM [1] and the GUM [2], and such a definition and the related evaluation methods cannot be disregarded when evaluating the risk of wrong conformity assessment, as clearly shown in [8]- [13].
This problem is covered by the BIPM document JCGM 106:2012 [14], in a very extensive way under a strict metrological perspective, and treating uncertainty according to the GUM recommendations [2].In particular, it covers the problem of stating whether a measured quantity falls inside a given tolerance interval, which is defined in [14] as the "interval of permissible values of a property".
According to the above definition, the tolerance interval can be both a closed interval and a one-sided interval.Furthermore, document [14] defines acceptance limits in such a way that, given a measurement uncertainty value, the measurand is declared conforming if the measured value falls inside the acceptance limits and non-conforming when it falls outside these limits.The document considers different decision rules and the way to evaluate the associated risk of incorrect assessment starting from measurement uncertainty.Hence, it represents a very useful guide in evaluating the probability of declaring as conforming an item that is not, and vice versa.
Although this problem is well discussed in [14] from a theoretical perspective, little guidance is provided, from a more practical point of view, on how to set the numerical value of the acceptance limit not to exceed the maximum admissible risk of making a wrong decision (once the measurement uncertainty and the maximum admissible risk are given).
This is an important issue when dealing with critical measurements, such as those performed to protect health and environment.This paper, after having quickly reviewed the most used decision rules, proposes a method that, given a threshold (or more in general a tolerance limit), provides the acceptance limit as a function of measurement uncertainty and a predefined maximum admissible risk of exceeding the given threshold.Example are given for some of the most used probability distributions.

THE MOST COMMON DECISION RULES
To correctly evaluate the risk associated with decision rules, it is necessary to identify or assume the probability density function (PDF) representing the distribution of values that could reasonably be attributed to the measurand [2], since this risk can be evaluated only after integrating such PDF from - to the threshold [2], [3].
It is well known that, according to the GUM [2], the standard uncertainty u(x) associated with a measurement result x is the standard deviation of the PDF representing the distribution of values that could reasonably be attributed to the measurand.
On the other hand, the expanded uncertainty U(x) = ku(x) identifies a coverage interval [x -U(x); x + U(x)], built about the numerical value x of the measurement result, whose coverage probability depends on the assumed probability density function and the considered coverage factor k.
It is also worth reminding that the PDF representing the distribution of values that could reasonably be attributed to the measurand depends on the available information.It is generally -and wrongly -considered that the available information comes only from the employed measuring equipment [15], while document JCGM 106:2012 [14] states that such information always has two components: the one available before performing the measurement (called prior information) and the additional information supplied by the measurement.The resulting, or posterior PDF, can be obtained by applying Bayes' theorem [14].
Keeping in mind the above considerations, it is possible to consider and discuss the two most common and employed decision rules in conformity assessment.It is assumed that the PDFs considered in the following sections are always the posterior PDFs.

Decision rule based on simple acceptance
This rule, also known as shared risk, considers accepting as conforming (and reject otherwise) an item whose property has a   measured value inside the tolerance interval.In this case, uncertainty is not explicitly considered.
In mathematical terms, and assuming that the tolerance interval is given by all values lower or equal than threshold t, an item is accepted as conforming if the measured value xm of property x satisfies to condition  m ≤ .
Let's make a few considerations about this decision rule.It can be readily checked that, assuming a symmetrical PDF about xm for the values that could reasonably be attributed to the measurand, the highest probability of exceeding the threshold is obtained in the limit case of xm = t and is 50%, no matter on the evaluated uncertainty value and the PDF.Therefore, when this decision rule is applied, measurement uncertainty does not affect the risk: reducing uncertainty only decreases the width of the interval of non-conforming values  nc that are considered as conforming, but does not reduce the risk of misidentifying nonconforming items as conforming, which still remains 50% (when xm = t).To define a maximum width of the interval of nonconforming values that are considered as conforming, a mutually agreed maximum acceptable expanded uncertainty Umax is generally set and it is therefore suggested that the expanded uncertainty U associated to the measured value, for a coverage factor k = 2, must satisfy U  Umax [14].

Decision rule based on guarded acceptance / rejection
The simple acceptance rule reported in Sec 2.1 shows that the closer the measured value to the threshold, the higher is the probability (up to 50%) of accepting an item as conforming that is not, and vice versa [14].This probability can be reduced by setting an acceptance limit inside the tolerance interval, as suggested by [14] and as shown in Figure 4, when, respectively, the measured value is required to be lower or equal a given threshold ( m ≤   ) -as in Figure 4a -and the measured value is required to be within a closed interval (  ≤  m ≤   ) -as in Figure 4b.
Figure 4 represents the case of guarded acceptance [14], that is the decision rule for which the risk of accepting a nonconforming item is reduced by setting an acceptance limit AU inside the tolerance interval (see Sec. 8.3.2 in [14]).
According to this rule [14], if the tolerance interval is a onesided interval, upper limited by TU (Figure 4a), an acceptance limit AU is set inside the tolerance interval.The interval between AU and TU (highlighted in yellow in Figure 4a) is called the guard band and its width (with sign) is defined as [14]: (1) In the case of Figure 4a, it is w > 0.
On the other hand, in the case of a two-sided tolerance interval, two acceptance limits AL and AU are set, as shown in Figure 4b.In this case, two guard bands are obtained, whose widths are defined as wU= TU -AU> 0 and wL= TL -AL< 0 respectively.
From Figure 4, it can be conclusded that, when a guarded acceptance decision rule is considered, an acceptance interval smaller than the tolerance interval is obtained.This decision rule is hence in favour of increasing the probability that an accepted item is truly conforming.
For the sake of completeness, let's consider that, with respect to the two cases shown in Figure 4 ( m ≤   and   ≤  m ≤   ), other two cases exist, that is: •  m ≥   : in this case, considering guarded acceptance, AL is on the right of TL and w= TL -AL< 0; •  m ≤   ∪  m ≥   : in this case, considering guarded acceptance, AU is on the right of TU and AL is on the left of TL, so that wU < 0 and wL> 0; so that the obtained acceptance interval is smaller than the tolerance interval.
A similar, though opposite situation is obtained in the case of guarded rejection [14].In fact, this decision rule is in favour of increasing the probability that a rejected item is truly nonconforming [14].In this case, acceptance limits are set, providing acceptance intervals greater than the tolerance interval.Without entering the details, as an example, by considering again Figure 4a, if the guarded rejection decision rule were applied, then the acceptance limit would be at the right of TU, thus providing a wider acceptance interval.
In general, |w| is set as a multiple of the expanded uncertainty: |w|= rU [14].If the PDF representing the distribution of values that could reasonably be attributed to the measurand is known or assumed, it is also possible to evaluate the risk of declaring a non-conforming value as conforming (or vice versa), as shown in Figure 5 in the case  m ≤   , when a normal PDF is considered and |w| = U = 2u is taken, as suggested by [16].In particular, Figure 5a and Figure 5b represent, respectively, the decisions of guarded acceptance and guarded rejection.

Guard band
To understand the relationship between w and the risk of wrong decision, let us consider the numerical example in Figure 5, where the maximum admissible limit (MAL, or, employing the same notation as the one in [14], TU) for a pollutant in water is assumed to be 50 mg/l.The pollutant concentration is assumed to be measured with a standard uncertainty of 5 mg/l, and the PDF representing the distribution of values that could reasonably be attributed to the measurand is assumed to be normal, as summarized in Table 1.
According to these values, the expanded uncertainty with k = 2 is U = 10 mg/l.A guard band w = U = 10 mg/l is considered, so that an acceptance limit AU = TU -w = 40 mg/l is set, when guarded acceptance is considered.
Therefore, the concentration of the considered pollutant in water is considered as conforming for every measured value xm  AU.If xm = AU, the situation shown in Figure 5a is obtained, in which the red line is located on TU.
Since a coverage factor k = 2 has been considered, the coverage probability of interval [xm -U; xm + U] is p = 95.45%.Therefore, the risk of exceeding TU, that is the probability pw of taking the wrong decision, when this decision rule is adopted, is On the other hand, if the aim of the measurement procedure is to assess, with high probability, that the pollutant concentration is higher than TU, the acceptance limit should be set, according to [14], at AU = TU + w.With the same numerical values and assumptions as before, this means that AU = 60 mg/l.Therefore, the concentration of the considered pollutant in water is considered as non-conforming for every measured value xm  AU.If xm = AU, the situation shown in Figure 5b is obtained, in which the red line is again located on TU.
In this case, the risk of declaring that the pollutant exceeds the tolerance limit TU while it does not is again pw = 2.28%.On the other hand, the risk that the pollutant is above the limit is, obviously, 97.7%,

THE RELATIONSHIP AMONG UNCERTAINTY, ACCEPTANCE LIMIT AND THE MAXIMUM ADMISSIBLE RISK
The example shown in the previous section relates uncertainty, acceptance limit and the risk of wrong conformity assessment in an implicit way, since it assumes that the distribution of the values that could reasonably be attributed to the measurand obeys to a normal PDF and the generally used coverage factor k = 2 is considered.These assumptions lead to the well-known 2.28 % risk of wrong decision.
However, different situations with different PDFs and different values for the acceptance limits may occur in practical cases, where also different values might be desired for the maximum admissible risk (MAR) of wrong decisions.
Therefore, a general formulation relating uncertainty, acceptance limit and MAR would be very useful to obtain one of them, given the other two ones.Document JCGM 106:2012 [14] provides some very general indications on how to do this, mostly referring to a normal PDF.Attempts were made in the past, especially in the legal metrology domain [17], to set the acceptance limits in such a way that, given the measurement uncertainty, a pre-defined risk could be granted.However, to the Authors' knowledge, no practical indications are available to relate acceptance limits, measurement uncertainty, and risk in such a way that, having set two of them, the third one could be found.
Such a relationship can be obtained starting from the PDF p(x) of the distribution of values that could reasonably be attributed to measurand x.Having defined such a PDF, the pertaining cumulative probability distribution function (CDF) can be obtained as: It can be readily checked that   () represents the probability that variable X is lower than x.Similarly, 1 −   () represents the probability that variable X is greater than x.
Therefore, using the same notation as the one used in [14], in the general case shown in Figure 4b, given a CDF   (), a tolerance limit TUL, and a MAR, if, for the considered measurable property, the measured value must be below the given tolerance limit TU, the following inequality must be satisfied: while, if the measured value must be above the given tolerance limit TL, the following inequality must be satisfied: Therefore, one of the following two equations must be solved to get the value of the acceptance limit AU (or AL) that ensures that the probability that the tolerance limit TU (or TL) is exceeded is exactly equal to the MAR: when the measured value must be below the threshold, or when the measured value must be above the threshold.
The acceptance limit values ensure that, respectively, if   ≤  U (  ≥  L ), then pw  MAR, where xm is the measured value and pw is the probability of exceeding the tolerance limit, that is, the probability of wrong decision.
Of course, solving these equations is strictly related to the shape of the PDF associated with the measurement result, and a solution cannot always be found in closed form.This does not prevent, however, the application of this method, because a numerical solution can be obtained by means of a Monte Carlo simulation, following the recommendations provided by Supplement 1 to the GUM [18].
On the other hand, the vast majority of the practical cases consider normal, uniform, triangular or trapezoidal PDFs1 .In such cases, a closed-form solution can be readily obtained for AU (or AL), and, hence, the normal, uniform, triangular and trapezoidal PDFs are considered in the following.
trapezoidal PDFs are generally obtained when two uniform PDFs are linearly combined, as in many practical measurement applications.Table 1.Numerical example.

Standard uncertainty
u pdf type 50 mg/l 5 mg/l Normal

The measurement results distribute according to a normal posterior PDF
When a normal PDF is considered: where  is the mean value and  is the standard deviation.Then, the corresponding CDF is given by: where: is the error function and can be well approximated with no more than 10 terms in ( 9).  ( U ) can, therefore, be written as: while   ( L ) can be similarly obtained.
In the above equation, , the mean value of the normal PDF, represents the measured value  m of the measurand.Therefore, if we want to find AU (AL), that is the maximum (minimum) value of the measured value such that  w ≤ MAR,  =  U ( =  L ) must be considered in (10).Therefore, according to ( 5) and ( 6) When  m <  U is required, solving equation ( 11) yields: where erfinv is the inverse error function, which is given by: where  0 = 1 and Similarly to the error function, also the inverse error function is well approximated with no more than 10 terms in (13).
On the other hand, when  m >  L is required, solving (11) yields: Since the inverse error function is an anti-symmetric function, that is: equations ( 12) and ( 14) can be grouped into a single equation: Therefore, to have a risk lower than MAR to exceed the tolerance limit TU (or TL), an acceptance limit AU (or AL) should be evaluated, obtained by shifting the tolerance limit to the left (or right) by quantity √2 ⋅ σ ⋅ erfinv(1 − 2 • ).In particular: • the limit is shifted to the left when   ≤  U is required and guarded acceptance is applied; • the limit is shifted to the right when   ≥  L is required and guarded acceptance is applied; • the limit is shifted to the right when   ≤  U is required and guarded rejection is applied; • the limit is shifted to the left when   ≥  L is required and guarded rejection is applied.To provide a numerical example, let us consider again the example considered in Section 2.2 and the values in Table 1.Let us remember that   <  U is required and suppose the MAR is set to 5 % and guarded acceptance is considered.By applying equation ( 16), it follows  U = 41.8 mg/l.
Figure 6 shows the normal PDF with a mean value equal to the obtained AU and standard uncertainty given in Table 1.In this figure, the coloured area represents the probability of being above TU, as also reported in Table 2.This probability is exactly the set MAR.This means that, for every value   <  U , the probability of exceeding  U will be lower than 5%.
It is therefore possible to set the acceptance limit, given the PDF associated with the estimated measurement uncertainty and the desired MAR.

The measurement results distribute according to a uniform posterior PDF
When a uniform PDF is considered: where  is the mean value and 2a is the support of the PDF, which is related to the PDF standard deviation by  =  ⋅ √3.
The corresponding CDF is given by: Table 2. Probability of being below or above TU in the case of Figure 6.

𝑷(𝒙 < MAL)
( > MAL) 0.95 0.05 and therefore: From ( 5) and ( 6), and considering  =  U,L in (19): By solving the above equations, the value for the acceptance limit is found: which shows that, to have a risk below MAR, the acceptance limit must be set to the left or right of the tolerance limit (as detailed in the previou section) by quantity  ⋅ (1 − 2 ⋅ MAR).To provide a numerical example, let us consider again the example of the pollutant in water considered in Section 2.2.Let us consider again that TU = 50 mg/l as in Table 1, but let us now suppose that the PDF associated with the estimated measurement uncertainty is uniform.Let us also suppose that the half-width of this uniform PDF is a = 10 mg/l, that the MAR is set to 5 % and that guarded acceptance is applied.By applying equation ( 21), it follows AU = 41 mg/l.
Figure 7 shows the uniform PDF with a mean value equal to the obtained AU.The coloured area represents the probability of being above TU, which is exactly 5 %.This means that every measured value of the pollutant in water lower than AU will provide a risk of exceeding TU lower than 5 %.

The measurement results distribute according to a triangular posterior PDF
When a symmetric triangular PDF is considered, its equation is the following: where: and: and where  and 2a are, respectively, the mean value and the support of the PDF.Furthermore,  =  √6 holds, where  is the standard deviation of the PDF.
To evaluate the corresponding CDF, two situations should be considered, that is the case when  ≤ μ and the case when  > .
If  ≤ μ, then CDF is given by: while, if  > μ, the CDF is given by: Now, ( 5) and ( 6) should be solved for both  ,1 and  ,2 , thus leading to four equations.However, only the most likely situations are here reported.In fact, when  ≤  U is required and the MAR is supposed to be small (as it should be when environmental, legal or health situations are considered), the situation shown in Figure 8a will occur, so that equation ( 27) must be solved.
On the other hand, when   ≥  L is required and again the MAR is supposed to be small, the situation in Figure 9b will occur, so that equation (28) must be solved.
(28) Equation ( 27) yields: By solving this simple equation with respect to AU, the following second-order equation is found:  which provides the two solutions: Among the two above solutions, the one with the minus sign can be discarded.In fact, if we considered a PDF with width 2a and  = ( U − ) −  • √2 • MAR, this PDF would not cross TU and therefore it would provide a risk of exceeding TU equal to zero.Of course, this should be a very lucky situation, but here the limit not to exceed the MAR needs to be found and, therefore, the following equation holds, under the assumption that  ≤  U is required: When, on the other hand,  ≥  L is required, (28) yields: Solving this simple equation with respect to AL, the following second-order equation is found: which provides the two solutions: Among the two above solutions, the one with the plus sign can be discarded.In fact, if we considered a PDF with width 2a and  = ( L + ) +  • √2 • MAR, this PDF would not cross TL and therefore it would provide a risk of exceeding TL equal to zero.Of course, this should be a very lucky situation, but here the limit not to exceed the MAR needs to be found and, therefore, the following equation holds, under the assumption that  ≥  L is required: Finally, by considering (32) and (36) together, it is: that is, the acceptance limit must be shifted to the left or right of the tolerance limit (as detailed in Sec.3.1) by quantity  ⋅ (1 − √2 • MAR) to have a risk lower than the MAR.As a numerical example, let us consider again the example of the pollutant in water considered in Section 2.2.Let us consider again that TU = 50 mg/l as in Table 1, but let us now assume a triangular PDF associated to the estimated uncertainty.Furthermore, the half-width of the triangular PDF is supposed to be a = 10 mg/l, the MAR is set to 5% and guarded acceptance is applied.Since  ≤  U is desired, by applying (37), AU = 43.2mg/l is obtained.Figure 9 shows the obtained PDF (centered on the obtained AU value), where the coloured area represents the probability of being above the tolerance limit TU, which is exactly equal to the pre-set MAR (5%).This means that every measured value of the pollutant in water lower than the obtained AU value will provide a risk lower than 5 %.

The measurement results distribute according to a trapezoidal posterior PDF
When a symmetric trapezoidal PDF is considered, then the PDF is described by the following equations: where: is the mean value of the PDF, 2a is its width and  is the ratio between the two basis.
To evaluate the corresponding CDF, three situations should be considered, that is the case when  −  ≤  ≤  −  , the case when  −   <  ≤  +   and the case when  +   <  ≤  + .However, similar considerations as the ones made for the triangular PDF apply, so that only the two situations shown in Figure 10a and Figure 10b are considered.
Let us call  ,3 () the CDF for the case in Figure 10a.The following equation must then be solved:   | ,3 ( U ) = 1 -MAR . (41) On the other hand, let us call  ,4 () the CDF for the case in Figure 10b.The following equation must then be solved:   | ,4 (  ) = MAR .
(42) Figure 10.Example when a trapezoidal PDF is assumed.The coloured area represents the probability of exceeding the tolerance limit when: a) the measured value is required to be below the tolerance limit (  ≤  U ); b) the measured value is required to be above the tolerance limit (  ≥   ).

Figure 1 .
Figure 1.Comparison of a measured value with a threshold, when the measured value  is lower (a) and greater (b) than threshold t and measurement uncertainty is not taken into account.

Figure 2 .
Figure 2. Comparison of an uncertainty interval with a threshold, when the uncertainty interval is completely below (a) and above (b) threshold t.

Figure 3 .
Figure 3.Comparison of an uncertainty interval with a threshold, when the threshold falls within the interval.

Figure 4 .
Figure 4. Decision rule based on guarded acceptance.In Figure 4a a one-sided tolerance interval upper limited by TU is considered, while in Figure 4b a twosided tolerance interval is considered between a lower and an upper limit TL and TU.

Figure 5 .
Figure 5. Example when  m ≤   and a normal PDF is supposed.The standard uncertainty () and the maximum admissible limit TU (red line) are given in Table 1, and || = () = 2 () is supposed.The coloured area represents the probability of exceeding TU, when the measured value corresponds to AU, in the cases of guarded acceptance (a) and guarded rejection (b).

Figure 6 .
Figure 6.Example when the normal PDF is centered on AU = 41.8 mg/l.The coloured area represents the probability of being above TU.

Figure 7 .
Figure 7. Example when the uniform PDF is centered on AU = 41 mg/l.The coloured area represents the probability of being above TU.

Figure 8 .
Figure 8. Example when a triangular PDF is assumed.The coloured area represents the probability of exceeding TU (or TL) when: a) the measured value is required to be below the tolerance limit (  ≤  U ); b) the measured value is required to be above the tolerance limit (  ≥  L ).

Figure 9 .
Figure 9. Example when the triangular PDF is centered on AU = 43.2mg/l.The coloured area represents the probability of being above TU.

Figure 11 .
Figure 11.Example when the trapezoidal PDF is centered on AU = 42.7 mg/l..The coloured area represents the probability of being above TU.