Gesture recognition of sign language alphabet with a convolutional neural network using a magnetic positioning system
DOI:
https://doi.org/10.21014/acta_imeko.v10i4.1185Abstract
Gesture recognition is a fundamental step to enable efficient communication for the deaf through the automated translation of sign language. This work proposes the usage of a high-precision magnetic positioning system for 3D positioning and orientation tracking of the fingers and hands palm. The gesture is reconstructed by the MagIK (magnetic and inverse kinematics) method and then processed by a deep learning gesture classification model trained to recognize the gestures associated with the sign language alphabet. Results confirm the limits of vision-based systems and show that the proposed method based on hand skeleton reconstruction has good generalization properties. The proposed system, which combines sensor-based gesture acquisition and deep learning techniques for gesture recognition, provides a 100% classification accuracy, signer independent, after a few hours of training using transfer learning technique on well-known ResNet CNN architecture. The proposed classification model training method can be applied to other sensor-based gesture tracking systems and other applications, regardless of the specific data acquisition technology.
Downloads
Published
Issue
Section
License
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under the CC BY 4.0, Creative Commons Attribution 4.0 International License.
Users are free to
- share, i.e. copy and redistribute the material in any medium or format for any purpose, even commercially;
- adapt, i.e. remix, transform, and build upon the material for any purpose, even commercially.
At the same time, the user must give appropriate credit, provide a link to the license, and indicate if changes were made.
Additional information about the license can be found at: https://creativecommons.org/licenses/by/4.0/.
Authors are
- able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
