Applicability of multiple impulse-radar sensors for the recognition of a person’s action

The research reported in this paper is devoted to the impulse-radar technology when applied for non-intrusive monitoring of elderly persons. Specifically, this study is focused on a novel approach to the interpretation of data acquired by means of multiple impulse-radar sensors, leading to the determination of features to be used for the recognition of a monitored person’s actions. The measurement data are ﬁrst transformed into the three -dimensional coordinates of the monitored person; next, those coordinates are used as a basis for determination of features characterising the movement of that person. The results of the experimentation, based on the real-world data, show that multiple impulse-radar sensors may be successfully used for highly accurate recognition of actions such as walking, sitting


INTRODUCTION
It is expected that the share of European population aged at least 65 years will reach 25% in 2050 [1]. The problem of organised care over elderly persons is, therefore, of growing importance. This, in turn, creates the demand for various technical solutions which could be applied for non-intrusive monitoring of elderly persons in home environments and healthcare facilities. The systems for monitoring of elderly persons are expected to predict and detect dangerous events, such as falls and harmful long lies after the falls. The falls of elderly persons belong to the most frequent reasons of their admission and long-term stay in hospitals [2].
Possible solutions that could be applied for non-intrusive monitoring of elderly persons are radar-based techniques -both narrow-band [3]- [8] and broad-band [9]- [13]. The most attractive feature of these techniques is the possibility of the through-the-wall monitoring of human activity. A review of the relevant literature, including articles in scientific journals and conference papers, which appeared in the years 2019-2022, has revealed that the vast majority of researchers use radar sensors of both types for estimation of the heart rate, breathing rate and position (in two dimensions), while the attempts to detect falls are based on sensors using the Doppler principle -with two exceptions: • In [14] an attempt to detect falls on the basis of threedimensional movement trajectory obtained by means of the three impulse-radar sensors is presented. In the reported approach, the monitored person's movement is compared with two model movements: a movement with a constant speed, and a movement towards the ground with an acceleration equal to the gravitational acceleration, and the classification of the movement is based on the reliability function. Unfortunately, no systematic tests of the effectiveness of the method are presented in that paper. • In [15] the results of the simulation studies focused on the impact of the number of the impulse-radar sensors on the accuracy of the estimation of the threedimensional position is presented. Unfortunately, the simulated setup is based on an unrealistic assumption that the sensors are placed in random locations within a monitored area. In the authors' recent conference papers [16], [17], the results of the studies on the applicability of multiple impulse-radar ABSTRACT The research reported in this paper is devoted to the impulse-radar technology when applied for non-intrusive monitoring of elderly persons. Specifically, this study is focused on a novel approach to the interpretation of data acquired by means of multiple impulseradar sensors, leading to the determination of features to be used for the recognition of a monitored person's actions. The measurement data are first transformed into the three-dimensional coordinates of the monitored person; next, those coordinates are used as a basis for determination of features characterising the movement of that person. The results of the experimentation, based on the real-world data, show that multiple impulse-radar sensors may be successfully used for highly accurate recognition of actions such as walking, sitting, and lying down, although this accuracy is significantly affected by the quality of the three-dimensional movement trajectories which in turn is affected by the configuration of the impulse-radar sensors within the monitored area.
sensors for estimation of the three-dimensional movement trajectories -which could be used for detection of dangerous events such as falls -are presented. These results show that the impulse-radar sensors may be used for accurate estimation of the three-dimensional position of a monitored person if the sensors are properly located within the monitored area. In this paper, the applicability of the impulse-radar sensors for recognition of person's action, on the basis of the estimates of the threedimensional movement trajectories, is investigated.
The novelty of the research presented in this paper consists in an algorithmic basis for recognition of actions of a person, in a monitoring system based on multiple impulse-radar sensors. The processing of the raw measurement data acquired by means of the impulse-radar sensors is divided into three stages: the transformation of the measurement data into the threedimensional coordinates of the monitored person, the calculation of the features characterising the three-dimensional movement trajectories, and the classification of the movement trajectories. The usability of the proposed features is assessed in an experiment based on a set of real-world data sequences representative of three activities of daily living: walking, sitting and lying down. Moreover, the influence of the configuration of the impulse-radar sensors within the monitored area on the accuracy of the action recognition is investigated.

ESTIMATION OF MOVEMENT TRAJECTORIES
The measurement data used for the experimentation were acquired by means of six X4M02 impulse-radar sensors manufactured by Novelda [18], [19]. An exemplary data frame, acquired by means of one of these sensors, is shown in Figure 1.
To properly estimate a three-dimensional movement trajectory of a monitored person, the measurement data, acquired by means of the impulse-radar sensors, have to be subjected to processing comprising [20]: the estimation of parameters of the impulse-radar signal, the smoothing of several one-dimensional trajectories of the distance between the monitored person and the corresponding impulse-radar sensors, and the transformation of the smoothed distance trajectories into the three-dimensional movement trajectory. In the research presented here: • The parameters of the impulse-radar signal have been estimated by means of a method consisting in computing the correlation function for the received signal and a known template of the emitted pulse, and the estimation of the coordinates of the maximum of this function [20]. • The distance trajectories have been smoothed by means of a method based on weighted least-squares estimator, consisting in the approximation of a sequence of data by means of a linear combination of basis functions, with the number of these functions determined automatically [17]. • The three-dimensional movement trajectories have been obtained by means of a method consisting in solving a set of equations modelling the geometrical relationships between the three-dimensional coordinates of a person and the distances between that person and the impulse-radar sensors [17].

Acquisition of measurement data
The measurement data used for the experimentation aimed at the assessment of the accuracy of the recognition of the monitored person's actions, were acquired by means of six impulse-radar sensors located at various positions. Two configurations of the sensors have been considered (see Figure 2): • Configuration #1 according to which the impulseradar sensors (R1, …, R6) were located at positions whose  [21] for the description of the methodology for preprocessing of data from depth sensors). The radar sensors and the depth sensor were synchronised, and their data acquisition rate was set to 30 Hz. In the experimentation, three movement scenarios were considered: • According to the first scenario, two persons walked along three predefined trajectories: an oval-shaped trajectory, a straight-line trajectory and a sine-shaped trajectory; each person repeated the action 10 times for each trajectory. • According to the second scenario, two persons sat on a chair located in three different places within the monitored area; each person repeated the action 10 times for each position of the chair. • According to the third scenario, two persons lay down on a mattress, approaching it from two different sides; each person repeated the action 15 times for each side of the mattress. Thus, the whole programme of experimentation comprised the acquisition of: • 180 three-dimensional movement trajectories obtained on the basis of the data acquired by means of the impulse-radar sensors located according to Configuration #1; • 180 three-dimensional movement trajectories obtained on the basis of the data acquired by means of the impulse-radar sensors located according to Configuration #2; • 180 three-dimensional movement trajectories obtained on the basis of the data acquired by means of the depth sensor.

Generation of features
In the experimentation, the features for classification have been determined on the basis of the sequences of the estimates of the z-coordinate of the position the persons, i.e. {̂}, as well as the sequences of the estimates of the velocity and acceleration along the z-axis (i.e. the first and second derivatives of the zcoordinate, obtained by means of the forward difference method), denoted with {̂, } and {̂, }, respectively.
All the features are presented in Table 1. For the sake of simplicity, two operators have been introduced -the operator returning the empirical mean value of a data sequence { }: and the operator returning its empirical variance: Moreover, the velocity in the vertical dimension and the acceleration in that dimension are denoted with ̂, ≡ {|̂, |} and ̂, ≡ {|̂, |}, respectively.

Classification
In this study an error-correcting output codes classifier (ECOC) -suitable for multiclass classification problems -has been used [22]. The ECOC classifier has been based on multiple support vector machines (SVM) -each designed to distinguish between two selected actions. The implementation of the ECOC classifier, available in the MATLAB Statistics and Machine Learning Toolbox [23], has been used for this purpose. Before the training of the classifier, the values of the features have been standardised. The performance of the classifier has been assessed using the 10-fold cross-validation technique.
The assessment of the accuracy of the classification has been based on the inspection of: • the receiver operating characteristic (ROC) curves illustrating the relationship between the true positive rate (TPR) the false positive rate (FPR), i.e. two indicators defined as follows: where -for example, in the case of walking -TP (true positive) is the number of walks classified as walks, TN (true negative) is the number of nonwalks classified as non-walks, FP (false positive) is the number of non-walks classified as walks and FN (false negative) is the number of walks classified non-walks; the area under the ROC curve (AUC) is a single scalar value representing the performance; • the confusion matrices visualising the results of the classification: each row of such matrix represents the instances in an actual class while each column represents the instances in a predicted class. In the experiments based on the real-world data the use of the approximations of the movement trajectories is necessary since their reference shapes cannot be properly defined: a human body has a considerable volume and generates complex echoes which cannot be attributed to any of its specific points (e.g. to plexus solaris). An arbitrary choice of such a reference point would lead to an arbitrary definition of the systematic error which could be misleading. Fortunately, such a definition is not necessary for extraction of the features which are used for classification of the actions of a monitored person -the features characterising the dispersion of the values of the z-coordinate, the vertical velocity and the vertical acceleration.

RESULTS OF EXPERIMENTATION
In Figure 2, the examples of the estimates of the threedimensional movement trajectories of a monitored person, obtained by means of the procedure described in Section 2, for two configurations of the impulse-radar sensors -together with the projections on the three two-dimensional planes -are shown; in Figure 3, the dispersion of the subset of the estimates of the z-coordinate, representative of walking, sitting and lying down, is presented.
In Figure 4, the ROC curves obtained for the classification of all the three-dimensional movement trajectories, are shown; the confusion matrices are presented in Figure 5.
The analysis of the presented results is leading to the following conclusions: • Multiple impulse-radar sensors may be successfully used for estimation of the three-dimensional position of a moving person, although the configuration of those sensors has significant influence on the uncertainty of the estimation. To properly estimate the height-component of the position of a monitored person, few impulse-radar sensors should be located at a greater height than the rest of those sensors (compare Figure 2c with Figure 2d as well as Figure 2e with Figure 2f).    • The configuration of the impulse-radar sensors affects the estimates of the coordinates of a monitored person. In the case of Configuration #2 the estimates of the z-coordinate are ca. 0.5 m greater than in the case of Configuration #1; moreover, the x-y projections of the trajectories, obtained for Configuration #2, seem to be scaleddown versions of the analogous projections obtained for Configuration #1 (compare Figure 2a with Figure 2b). These discrepancies can be explained by a non-negligible volume of a human body: the impulse-radar sensors located at a greater height -and, therefore, oriented differently than the rest of those sensors -receive echoes reflected from different parts of the body of a monitored person. Moreover, in the case of Configuration #1 the changes in the z-coordinate, associated with the movement of the body towards the ground during sitting or lying down, may not be properly reflected in the estimates of the movement trajectory (see Figure 2c and Figure 2d). This phenomenon may be explained by the fact that when the person is moving towards the ground, the changes in the distance between that person and the impulse-radar sensors placed on the sides of the monitored area are not significant. In the case of the impulse-radar sensors placed on the ceiling, these changes are much greater. • The proposed features, characterising the monitored person's movement in the vertical dimension, are sufficient to recognise walking, sitting and lying down with high accuracy, although this accuracy is significantly affected by the quality of the threedimensional movement trajectories which in turn is affected by the configuration of the impulse-radar sensors (compare Figure 5a with Figure 5b). The results of the classification of the three-dimensional movement trajectories obtained on the basis of the data acquired by means of the depth sensor, are the best because the depth sensor provides the most accurate estimates of the threedimensional position of the monitored person. Nevertheless, the results of the classification of the trajectories obtained on the basis of the data acquired by means of the impulse-radar sensors located according to Configuration #2, are only slightly worse and can be likely improved by the application of more sophisticated methods for impulse-radar data processing.

CONCLUSIONS
The novelty of the research presented in this paper consists in an approach to the interpretation of measurement data acquired by means of the impulse-radar sensors, leading to the determination of features to be used for recognition of actions of three types: walking, sitting and lying down. The data are first transformed into the three-dimensional coordinates of the monitored person; next, those coordinates are used as a basis for calculation of kinematic features characterising the monitored person's movement in the vertical dimension.
The results of the experimentation based on real-world data show that multiple impulse-radar sensors may be successfully used for highly accurate recognition of walking, sitting and lying down of a monitored person. It has to be noted, however, that the accuracy of the recognition is affected by the quality of the three-dimensional movement trajectories which in turn is affected by the configuration of the impulse-radar sensors. To properly estimate the height-component of the position of the monitored person, few impulse-radar sensors should be located at a greater height than the rest of those sensors.
The results encourage the authors to focus on the development of the methods for processing of the impulse-radar data, enabling the detection of falls of the monitored person. Figure 5. The exemplary confusion matrices obtained for the classification of all the three-dimensional movement trajectories: radar-data-based trajectories obtained for Configuration #1 (a), radar-data-based trajectories obtained for Configuration #2 (b), depth-data-based trajectories (c).