Sample volume length and registration accuracy assessment in quality controls of PW Doppler diagnostic systems: a comparative study

In clinical diagnostics, Pulsed Wave (PW) Doppler is one of the most used spectral Doppler techniques since it provides quantitative information about the severity of several cardiac disorders. Therefore, routine quality control tests should be scheduled to check whether a proper level of performance is maintained over time. Despite continuous research in the field, performance evaluation of Doppler equipment is still an open issue. Therefore, the present study is focused on the comparative investigation based on a test parameter for the automatic analysis of faults in sample volume length and range gate registration accuracy. The Velocity Profile Discrepancy Index (VPDI) provides a quantitative estimation according to the agreement between the theoretical parabolic velocity profile and the measured one. The index was assessed through an automatic method that post-processes PW spectrogram images acquired at six sample volume depths with respect to the vessel radius of a Doppler reference device. Tests were repeated for three brand-new ultrasound diagnostic systems, equipped with convex and phased array probes, in two working conditions. From the analysis of the results, a lower discrepancy between the measured and the theoretical velocity profile was found for the convex array probes as well as a lower uncertainty contribution.


INTRODUCTION
Pulsed Wave (PW) Doppler is extensively used in diagnostic imaging since it allows displaying and quantifying the velocity of both arterial and venous flow through time. Ultrasound (US) diagnostic systems provide a real-time spectral Doppler display by performing the spectral analysis of the Doppler signal from the blood vessel [1], [2]. This signal is detected within a sensitive region, known as Sample Volume (SV), whose length and depth (from the interface between the patient skin and the US probe) can be adjusted by the operator on the B-mode anatomical image [2], [3]. In the literature, two main parameters were used to characterize the sample volume, i.e., size and range gate registration accuracy [4], [5]. In particular, SV size depends on both the effective sample duration and the width of the US beam [6], and it affects the PW Doppler spatial resolution. In turn, sample volume size can be split into two main contributions [7]: lateral and axial resolution. Nowadays, SV axial resolution is included by the UK's Institute of Physics and Engineering in Medicine (IPEM) among the basic functional checks for Spectral Doppler designed for control functioning assessment and faults detection [8]. As regards the range gate registration accuracy, it quantifies the error due to the difference between the SV position displayed on the grayscale image and the actual one. Such parameter is one of the Quality Control (QC) Doppler tests recommended by the American Institute for Ultrasound in Medicine (AIUM) [9].
It is worth noting that the majority of the scientific literature that focused on sample volume size [7], [10]- [12] and range gate registration accuracy [12], [13] dates back to the last years of the XX century. Moreover, the QC test procedures recommended by the two international organizations are affected by operator subjectivity. In this regard, performance evaluation of Doppler

ABSTRACT
In clinical diagnostics, Pulsed Wave (PW) Doppler is one of the most used spectral Doppler techniques since it provides quantitative information about the severity of several cardiac disorders. Therefore, routine quality control tests should be scheduled to check whether a proper level of performance is maintained over time. Despite continuous research in the field, performance evaluation of Doppler equipment is still an open issue. Therefore, the present study is focused on the comparative investigation based on a test parameter for the automatic analysis of faults in sample volume length and range gate registration accuracy. The Velocity Profile Discrepancy Index (VPDI) provides a quantitative estimation according to the agreement between the theoretical parabolic velocity profile and the measured one. The index was assessed through an automatic method that post-processes PW spectrogram images acquired at six sample volume depths with respect to the vessel radius of a Doppler reference device. Tests were repeated for three brand-new ultrasound diagnostic systems, equipped with convex and phased array probes, in two working conditions. From the analysis of the results, a lower discrepancy between the measured and the theoretical velocity profile was found for the convex array probes as well as a lower uncertainty contribution.
systems is still an open issue in the scientific research field [4], [14]- [17], despite the increasing demand for proper QC programs. Nonetheless, the measurement of blood flow peak velocity is of particular relevance in clinical applications since several clinical parameters are evaluated from the peak velocity envelope. PW Doppler provides quantitative information about the severity of many cardiac disorders [18], including the degree of stenosis [19], [20], arterial wall shear stress [21], [22], and pressure drops across cardiac valves [23]. Therefore, inaccurate measurements can impact diagnosis and clinical therapy: overestimations ranging from 12 % to 50 % have been estimated as regards the measurement of blood peak velocities [24], [25].
From the abovementioned considerations, the present study would give a contribution to the field of QCs for Doppler equipment testing, by proposing a comparative investigation of the Velocity Profile Discrepancy Index (VPDI), a quality parameter developed for the objective assessment of both SV length and range gate registration accuracy. The index, preliminarily defined and tested in [26], is estimated from the processing of PW Doppler spectrograms by means of a customwritten image analysis method developed in MATLAB environment. Three brand-new US diagnostic systems, each of them equipped with two ultrasound probes (convex and phased array) were tested in two working conditions.
In Section 2, the estimation rationale underlying the definition of the proposed quality index will be provided. In Section 3, the experimental setup components and their main specifications will be described together with the image analysis-based method implemented for VPDI assessment. In Section 4, experimental results with their associated measurement uncertainty will be presented and discussed, focusing on the comparison among the US diagnostic systems under test. Finally, in the concluding section, the major achievements and future developments will be outlined.

VELOCITY PROFILE DISCREPANCY INDEX
By considering a cylindrical tube of infinitesimal length d exhibiting fully developed laminar flow in the reference system shown in Figure 1, the well-known parabolic velocity profile ( ) of the Hagen-Poiseuille flow [27], [28] can be written as: where is the tube radius, is the dynamic viscosity coefficient, d is the infinitesimal pressure drop and -axis is the flow direction. From (1), the maximum velocity value max can be reached for = 0, while for = | | the no-slip boundary condition applies, i.e., the relative tangential and normal velocities between the flow and the tube walls are zero: From these considerations, it is possible to derive the mathematical expression of the parabolic velocity profile ( ) as a function of max and , as follows: where is the radial coordinate. By subdividing ( ) into a fixed number of bins and assuming the limits of the -th bin ( = 1, …, ) as the SV boundaries (Figure 2a), the theoretical velocity ̅ th for each bin can be derived from (3) as follows: where SV is the SV position with respect to the tube radius and is the sample volume length ( Figure 2b). Finally, the Velocity Profile Discrepancy Index (VPDI) for the objective assessment of both sample volume length and registration accuracy is derived as [26]: where ̅ s, is the mean peak velocity estimated from the Doppler spectrogram collected at pre-set sample volume depth and length, ̅ th, is the corresponding theoretical velocity value retrieved by applying (4) for each bin, while tot, is the total standard deviation estimated for the -th bin as follows: where s is the standard deviation (SD) of the peak velocities in the Doppler spectrogram because of the intrinsic flow dispersion, r is the standard deviation of a random distribution associated with the electronic noise superimposed on the PW spectrogram, and p is the SD due to the assumption of parabolic profile: where SV is the SD associated to the sample volume position with respect to R. According to the definition proposed in (5), VPDI is expected to be 0, i.e., the acquired PW spectrograms are not affected by any variation of the sample volume length with respect to the set value and any SV registration error. On the contrary, if VPDI deviates from 0, the PW spectrograms are affected by at least one of the two error sources, i.e., unwanted SV size variation and/or a low SV registration accuracy.

Experimental setup
The experimental setup was constituted by three high technology level US diagnostic systems, equipped with a convex and a phased array probe each, and a reference test device whose main specifications are listed in Table 1. The test device used is a commercial flow phantom model (Sun Nuclear, Doppler 403 TM flow phantom [29]) consisting of a continuous flow loop vessel with a horizontal and a diagonal segment having the same inside diameter. A pump and a flow controller are designed by the manufacturer to provide user-selectable flow rates in the range (1.7-12.5) ml·s -1 . Doppler spectrograms were collected on the phantom diagonal segment set at a constant flow rate of 7.0 ml·s -1 corresponding to a nominal peak velocity of 71.3 cm·s -1 .
Tests were repeated at two different US system working conditions (Table 2): the best configuration setting provided by the product specialist (set 1), and the raw configuration setting by reducing pre-and post-processing settings (set 2). The three ultrasound diagnostic systems, manufactured by different companies, were anonymously addressed as system A, system B and system C. The main setting that distinguishes the two working conditions is the wall filter, a dedicated high-pass filter designed to reject the intense and low-frequency echoes coming from the vessel walls movement (clutter signal) [2]. In the raw configuration setting it was always set at the minimum adjustable   value. Conversely, the pulse repetition frequency (PRF), i.e., the rate at which US pulses are transmitted [2], was, in turn, determined by the US system since its value depends on the other settings. Among the latter, the insonation angle was varied according to both the probe positioning on the scanning surface and the probe model so as to keep the sample volume parallel to the flow axis throughout the acquisitions, while the spectrogram duration was set to at least 10 s (depending on the system availability) in order to allow the comparison of the results retrieved from different system-probe pairs. In addition, the sample volume length (SVL) was maintained constant throughout the measurement campaign, while the sample volume depth (SVD) was adjusted on the vessel diagonal segment to collect N = 6 Doppler spectrograms for each probesystem pair as listed in Table 2. First, the sample volume was placed in the centre of the segment diameter, C in Figure 3a, and then it was adjusted at C ± ∆ (Figure 2a), by considering ∆ the minimum sample volume depth variation that can be set in most of the ultrasound systems currently on the market. The sample volume was subsequently placed on the segment boundary, B in Figure 3b, as well as at B ± ∆ ( Figure 2b). As in [26], SVD was adjusted along the -axis by assuming that the velocity does not vary along the flow axis of the tube.

VPDI assessment
The velocity profile discrepancy index defined in [26] was assessed through an image analysis-based method that postprocesses PW Doppler images acquired at different sample volume depths. The main steps of the measurement method are described in the following and shown in Figure 4, while the specifications assumed in this study are listed in Table 3.
As a first step, each PW image is cropped to extract the box containing the spectrogram image only, i.e., excluding the patient information, the B-mode grayscale image and the ultrasound settings details. This processing step is automatically carried out based on the metadata included in the DICOM (Digital Imaging and Communications in Medicine) file.  Then, an adaptive grey level threshold ℎ g is applied to the spectrogram image to detect the pixels associated with the peak velocities. The corresponding mono-dimensional signal s through time is retrieved as in [30], [31] and the mean peak velocity ̅ s is computed together with the standard deviation s over samples (i.e., spectral lines). On the other hand, the total standard deviation tot is estimated.
In this study, the following assumptions were made for the estimation of tot as in [26]. The standard deviation r in (6) was estimated from a randomly generated gaussian distribution consisting of a number of elements equal to 15 % of and mean value half of the spectrogram full scale. In addition, SV in (7) was estimated from a uniform distribution within the range [−∆ 2 ⁄ ; ∆ 2 ⁄ ] by considering ∆ as the minimum SVL increment. Finally, both s and p in (6) were taken equal to 0 if the number of peak velocities s at 0 cm·s -1 was higher than 85 % of . Therefore, when the above condition occurred, the detected velocities were assumed as electronic noise, i.e., tot = r . Otherwise, the uncertainty propagation law was applied including s and p in the computation of tot even if outside of the vessel segment, i.e., at B − ∆ . The processing steps described above are repeated for all the acquisitions corresponding to different sample volume depths. At this point, the theoretical velocity ̅ th is determined for each SVD by applying (4), in which the maximum velocity max was chosen as the mean peak velocity determined at C , as in [26]. In the last step, the velocity profile discrepancy index is computed according to (5). VPDI uncertainty can be estimated as the standard deviation of the data distribution obtained through the Monte Carlo Simulation (MCS), a useful tool for the measurement uncertainty estimation of image analysis-based methods [32]- [36].
In this study, a MCS with 10 4 iterations was run for each probe-system pair in both working conditions. A uniform distribution with ± 2 % bounds was assigned to the adaptive threshold ℎ g and the spectral lines to be processed were randomized, at each cycle without repetition, among all the spectral lines detected in the spectrogram image. In particular, the uniform distribution was assigned to ℎ g because, in terms of standard deviation, it is a more cautious approach for the uncertainty estimation, whilst the distribution range was assumed as the maximum threshold dispersion allowing to distinguish the velocity signal from the background noise displayed in the spectrogram, as already experienced in previous studies [31], [37].

RESULTS AND DISCUSSION
VPDI experimental outcomes for systems A, B and C equipped with convex and phased array probes at different working conditions are listed in Table 4 in terms of mean ± and shown in Figure 5.
By comparing the two probe models, the convex array probes showed the best VPDI outcomes (closest to 0) in both working conditions as well as the lowest uncertainty contributions. This suggests that the PW spectrograms acquired through the phased array probes could be most affected by an error in sample volume length (with respect to the pre-set value) and in its registration, (i.e., a low SV registration accuracy). Such issue seems to be supported by flow velocity trends still visible outside of the phantom vessel segment, as shown in Figure 6.
On the other hand, by comparing the two working conditions, higher uncertainty contributions were globally found at raw configuration settings (set 2) independently of the probe model. Moreover, higher VDPI values were found in set 2 for both the probe models of system A and system C, while a specific behaviour cannot be inferred for system B. The discrepancy in results between set 1 and set 2 is mostly due to the different filtering of the clutter signal (i.e., wall filter setting in Table 2). The threshold setting at 10 % allows to properly distinguish the velocity signal from the background noise displayed in the spectrogram [30], [31].   Finally, by focusing on the comparison among the three ultrasound diagnostic systems, it should be noticed that system A equipped with the convex array probe in set 1, and system B equipped with the phased array probe in set 2 showed VPDI outcomes closer to 0.

CONCLUSIONS
In the study herein proposed, a comparative investigation on the Velocity Profile Discrepancy Index was carried out. The index, preliminarily investigated by the Authors, was developed for the objective assessment of sample volume length and range gate registration accuracy. It relies on the assumption of a fully developed laminar flow in order to quantify the discrepancy between the theoretical parabolic profile and the actual velocities. According to the definition, VDPI is expected to be 0 when the processed PW spectrograms are not affected by any variation of the SVL with respect to the pre-set nominal value and any range gate registration error. An automatic image analysis-based method post-processes PW spectrograms acquired at different sample volume depths with respect to the vessel radius of a Doppler reference device.
Three brand-new US diagnostic systems, equipped with a convex and phased array probe each, were tested in two working conditions. A commercial reference test device constituted by a continuous flow loop vessel with a horizontal and a diagonal segment was set at a constant flow rate of 7.0 ml·s -1 . Experimental data were collected on the diagonal segment of the phantom vessel. From the analysis of the results, convex array probes showed VDPI outcomes closer to 0 in both working conditions as well as the lowest uncertainty contributions. On the basis of the promising outcomes obtained and their limitations, further studies and method improvements are going to be carried out to estimate the two error sources separately. Moreover, further investigations should be performed including a higher number of diagnostic systems and probe models (e.g., linear array probes).

ACKNOWLEDGEMENT
The Authors wish to thank Jan Galo of the Clinical Engineering Service at I.R.C.C.S. Children Hospital Bambino Gesù for administrative and technical support; SAMSUNG Healthcare, PHILIPS Healthcare and MINDRAY Medical for hardware supply and technical assistance in data collection.