ORIGINAL ARTICLE
Year : 2022 | Volume
: 9 | Issue : 3 | Page : 107--113
The usage of constrained independent component analysis to reduce electrode displacement effects in real-time surface electromyography-based hand gesture classifications
Ulvi Baspinar1, Yahya Tastan2, Huseyin Selcuk Varol2,
1 Faculty of Technology, Marmara University, Istanbul, Turkey
2 Institute of Pure and Applied Sciences, Marmara University, Istanbul, Turkey
Correspondence Address:
Ulvi Baspinar
Faculty of Technology, Marmara University, Istanbul
Turkey
Abstract
Aim: In real-time control of prosthesis, orthosis, and human–computer interface applications, the displacement of surface electrodes may cause a total disruption or a decline in the classification rates. In this study, a constrained independent component analysis (cICA) was used as an alternative method for addressing the displacement problem of surface electrodes. Materials and Methods: The study was tested by classifying six-hand gestures offline and in real-time to control a robotic arm. The robotic arm has five degrees of freedom, and it was controlled using surface electromyography (sEMG) signals. The classification of sEMG signals is realized using artificial neural networks. cICA algorithm was utilized to improve the performance of classifiers due to the negative effect of electrode displacement issues. Results: In the study, the classification results of the cICA applied and unapplied sEMG signals were compared. The results showed that the proposed method has provided an increase between 4% and 13% in classifications. The average classification rates for six different hand gestures were calculated as 96.66%. Conclusions: The study showed that the cICA method enhances classification rates while minimizing the impact of electrode displacement. The other advantage of the cICA algorithm is dimension reduction, which is important in real time applications. To observe the performance of the cICA in the real-time application, a robotic arm was controlled using sEMG signals.
How to cite this article:
Baspinar U, Tastan Y, Varol HS. The usage of constrained independent component analysis to reduce electrode displacement effects in real-time surface electromyography-based hand gesture classifications.J Neurobehav Sci 2022;9:107-113
|
How to cite this URL:
Baspinar U, Tastan Y, Varol HS. The usage of constrained independent component analysis to reduce electrode displacement effects in real-time surface electromyography-based hand gesture classifications. J Neurobehav Sci [serial online] 2022 [cited 2023 Mar 24 ];9:107-113
Available from: http://www.jnbsjournal.com/text.asp?2022/9/3/107/366391 |
Full Text
Introduction
The concept of using prosthesis as a real limb for disabled persons or using muscle and limb movements to control giant robots may seem like a futuristic fantasy or science fiction novel, but the latest developments in biomedical,[1] biomechanics and sensor technology and commercially available prosthesis, such as i-Limb, Bebionics, Otto Bock's Genium Bionic prosthetic systems, and Thalmic Labs wearable gesture control systems, show that this concept is becoming a reality.
Despite these improvements, there are a number of issues that must be addressed. One of the obstacles in controlling prosthesis and in real-time human–computer interaction (HCI) applications are surface electromyography (sEMG) signals.[1] The sEMG signals are highly changeable signals. The signals can be affected by fatigue, artifacts, noise, and blood pressure. Beyond these parameters, electrode placement plays a vital role for high accuracy classifications.[2] While in motion, the prosthesis and the HCI devices' electrode places can be changed, and placing electrodes at the same point is very difficult.[3] Thus, the classification accuracy is either compromised or a retraining process is necessary. Hargrove et al. investigated how electrode displacement, size and orientation affect pattern recognition based on myoelectric control systems.[3] Hargrove et al. found that electrodes shifts, orientation, and size reduced classification accuracy by about 10%–30%.[3]
The issue with electrode displacement is that the classification algorithm is adjusted depending on a certain position for a specific gesture. When the electrodes' positions are changed, the sEMG recordings are affected from the adjacent muscle activities, so classification rates decline. This problem can be described as a source separation of sEMG signals.
There are several studies in the literature for source separation. One of the source separation algorithms is independent component analysis (ICA). The use of ICA algorithms has become popular in removing artifacts, noise, and source separation of biomedical signals, such as electrocardiography, electroencephalography, and electromyography.[4] Although ICA provides a solution for source separation, it has disadvantages, such as finding the sources in a different order each time the algorithm is run.[5]
The ICA algorithm is used in several studies on hand gesture identification. Naik et al. proposed the identification of various hand gestures for HCI using ICA[6] and a multi-run ICA-based signal processing system for recognizing hand gestures with an artificial neural network (ANN) classifier.[7] They reported that ICA alone is not suitable for sEMG due to the nature of sEMG distribution and the disorder of the estimated sources. They classified three-hand gestures for HCI and six-hand gestures with high accuracy using four electrodes in two different studies.[8] Sueaseenak et al. classified eight gestures with 16 electrodes using an ICA classifier and compared the results with an ANN classifier.[9] They found the classification performances of the ICA classifier to be at 93.3%.
The constrained ICA (cICA) algorithm, which is a type of traditional ICA, can be used for the source separation applications in which any prior information about the source of interest is known. The purpose of the cICA algorithm is to obtain an output that is statistically independent from other sources and closest to the predefined reference signal. There are also studies of the source separation of biomedical signals that use the cICA algorithm.[10]
In this study, a constrained ICA solution is proposed for the displacement problem of electrodes, and the classification of six hand gestures (hand open-close, wrist flexion-extension, radial, and ulnar movement) was conducted offline with high accuracy using the estimated sEMG signals. The classification results of the cICA applied and unapplied sEMG signals were compared. The study shows that the proposed method also increases the classification performances of the ANN classifier. As an additional application, a robotic arm was controlled in real-time using sEMG signals. A simple task, which was gripping a cube and moving it to another place, was performed using hand gestures.
Materials and Methods
There is no need for ethics committee approval.
In this section, robotic arm gestures for robotic arm control, electrode placement, data acquisition, signal processing methods, and the classification algorithms are discussed. The block diagram in [Figure 1] illustrates the main steps in the study. The study can be divided into two sections. The first section is generating a demixing matrix, which is separating mixed sEMG signals in an offline operation.[7] The second section controlling the robotic arm in real time. While controlling the robotic arm, the demixing matrix is used first, which is obtained in the first step, and then the obtained estimated sEMG signals' features are extracted and classified to control the robotic arm.{Figure 1}
Robotic arm and gestures for robotic arm control
In this study, the AX-12A Smart Robotic Arm (Crustcrawler, Inc., California) was chosen because its motion capability is similar to the commercially available prosthesis. It has five degrees of freedom. Six different hand gestures were used to control three degrees of the robotic arm.[8] These hand gestures were hand open-close, wrist flexion extension, and radial and ulnar deviations. In [Figure 2], the gestures and the controlling of the axis according to the gestures are shown.{Figure 2}
Electrode placement and data acquisition
Electrode placement plays a vital role in sEMG-based systems. In this study, we used six electrodes. Two of them were cICA reference electrodes, and the other four electrodes were test electrodes. The cICA reference electrodes were placed along the longitudinal axes of the Brachioradialis and the Flexor Carpi Radialis muscles, respectively, and the other four electrodes were placed in parallel positions (25 mm away) to the ICA reference electrodes. The placements of the electrodes are shown in [Figure 3]. The bipolar passive disposable Ag/AgCL electrodes (Myotronics Inc., WA), which were 1 mm in diameter and 22 mm in interelectrode distance, were chosen for this study.{Figure 3}
The analog sEMG signals were amplified using a homemade sEMG amplifier, and these amplified signals were converted to digital signals using the IOtech Personel Daq/3000 (IOTech, MA) series data acquisition system at 16 bits resolution with a 1 kHz sampling frequency.[11] The amplified sEMG signals were filtered using a band-pass filter, which allows for passing the signal between 20 Hz and 500 Hz.
Independent component analysis and constrained independent component analysis algorithm
ICA is a statistical method in which the goal is to decompose given multivariate data into a linear sum of statistically independent components. Recently, ICA has been used in the biomedical signals for removing artifacts, data reduction, and source separation.[12] The general ICA equation can be stated as in Eq 1 where S defines the source signal, A defines the unknown mixing matrix, X defines the mixture of source signals, W defines the unmixing matrix, and U defines the estimated source signals. The goal of ICA is to find a linear transformation W of the sensor signals X that makes the outputs U as independent as possible.[13] There are several algorithms to find the unmixing matrix W, and one algorithm is the Constrained ICA (cICA) algorithm.
[X] = [A] * [S]
[U] = [W] * [X]
Constrained ICA is an algorithm that searches statistically independent sources in mixed signals environments according to the predefined reference signal.[14] It is not necessary that this reference signal be the exact signal of interest. This knowledge is important because obtaining the perfect match for a reference signal is impossible in most cases. In studies that searched for a signal source in mixed environments, cICA is very useful because the conventional ICA finds sources in different orders, so it is difficult to determine whether or not the source of interest has been identified. So you cannot be sure it is the source that you are interested. The disadvantage of cICA is that the user must know the nature of the signal of interest to select the proper reference signal. In our study, the sEMG recordings of two electrodes were reduced to one sEMG signal according to a reference sEMG signal. The cICA model used in this study is depicted in [Figure 4].{Figure 4}
Reference surface electromyography selection
The success of the cICA algorithm completely depends on the selection of the reference signal.[15] The cICA algorithm cannot find the source signal of interest without obtaining the correct reference signal. In this study, the control of the robot arm was performed using six-hand gestures, so twelve reference signals had to be determined for the six gestures recorded for the two muscles.
In order to determine the reference signals, the signals of the gestures were recorded first, while the subject was asked to perform six gestures and repeat the same gestures ten times. Second, the recorded sEMG signals' features were extracted.[16] Third, using these features, an ANN (12 neurons at the input layer, 20 neurons at the hidden layer, and 6 neurons at the output layer) was trained. Following the training session, the gestures with the minimum classification errors were recorded. The training process was repeated ten times, and the minimum misclassified gestures' signals were selected as reference signals.
Generating a demixing matrix
The cICA algorithm generates a demixing matrix using a reference signal in order to easily generate six demixing matrices for each gesture on one muscle and to use it for classification. The problem in this study was that we had to classify six gestures, but we did not know which gesture was detected before the real-time classification, which means that we could only use one demixing matrix for each muscle.[17] Our solution was to generate six demixing matrices using preselected reference signals and to calculate the medians of these six demixing matrices (demixing matrices are the same size) to get one common demixing matrix for one muscle. We used Brachioradialis and Flexor Carpi Radialis muscles to generate one demixing matrix for each muscle.
Feature extraction
The feature-extracting process is one of the most important parameters in classifying sEMG signals because it directly affects the classification performance. There are several feature-extracting methods for sEMG signals, but they are generally categorized as time and frequency domain features. In real-time systems, the frequency domain features are not preferred because of the extra time consumed in comparison with the time domain features. In this study, six-time domain features were used. These feature vectors are variance (Var), zero crossing, wavelength, Willison amplitude (WAmp), mean absolute value, and slope sign change.[18],[19] Previous studies were considered in choosing the feature vectors.
Classification
The (ANN) is one of the most commonly used classification algorithms.[20] There are studies of the classification capability of ANN in sEMG signals.[21] The feedforward backpropagation ANN network type, which has one hidden layer with 20 neurons and one output layer with 6 neurons, is used in the study. The selected ANN has tansig and purelin activation functions at the hidden and output layers, respectively. While training the ANN, the Levenberg–Marquardt training algorithm and the mean squared error performance function were used. The classes of gestures were determined according to the majority voting result. When the gesture was detected, 1000 samples were recorded. The recorded gesture data were divided into five equal segments using a windowing technique as shown in [Figure 5]. Nonoverlapping windowing was used so that each window had 200 samples. For each segment, an ANN was trained so that means decisions were given according to the five ANNs. The results of the ANNs were evaluated using a majority voting algorithm, and the gesture with the highest vote was accepted as the winning gesture. The classifier model is depicted in [Figure 6].{Figure 5}{Figure 6}
Results
This section is divided into two subsections: offline classification results and real-time control of the robotic arm.
Offline classification results
In this section, the results were compared according to the classification performances of cICA applied and unapplied sEMG signals, which were recorded in the vicinity of the reference points and evaluated based on the aspect of the classification of six-hand gestures using one and two muscles.[22]
The ANN training process had random parameters to obtain more accurate classification results. Ten different ANNs with the same size were trained, and the classification results were recorded. In [Table 1], the maximum, the minimum, and the calculated average values of the recorded classification results are shown.{Table 1}
First, we attempted to determine what would happen if we placed electrodes (TE1, TE2, TE3, and TE4) in the vicinity of the reference points (Ref 1 and Ref 2), as shown in [Figure 3]. The results show that the classification results declined up to 8% as expected. In other words, the reference points were chosen correctly.
Second, the classification effects of the cICA algorithm were investigated on the muscles Brachioradialis and Flexor Carpi Radialis, respectively. The TE1 and TE3 electrodes were placed in the vicinity of the Ref1 point on the Brachioradialis muscle, and the TE2 and TE4 electrodes were placed in the vicinity of the Ref2 point on the flexor carpi radialis muscle. The TE1-TE3 and TE2-TE4 signals were introduced to the cICA algorithm to obtain the ESS1 and ESS2 signals, respectively. The obtained ESS1 and ESS2 signals' effects on the average classification performances were calculated as 89% and 87.66%, respectively. These rates show a minimum increase of between 3% and 6% in classification performances when compared with the other electrodes, except the Ref1 and Ref2 electrodes.
Another evaluation was conducted regarding the cICA effect on the classification of six gestures when both of the muscles were included. When we generated the binary combination of the TE1, TE2, TE3, and TE4 electrodes on the two muscles, we obtained the TE1-TE2, TE1-TE4, TE3-TE2, and TE3-TE4 combinations. When we compared the effects of these generated binary combinations and ESS1-ESS2 with the classification results, an increase between 4% and 13% was observed. In addition [Figure 7], the comparison between ESS1-ESS2, Ref1-Ref2, and all electrodes (TE1-TE2-TE3-TE4) was made. The results indicate that when features of the ESS1-ESS2 signals are applied to the ANN classifier, the average success of the classifier increased to 96.66%, while the success rates of Ref1-Ref2 and TE1-TE2-TE3-TE4 were 93% and 92.66%, respectively.{Figure 7}
Real-time control of the robotic arm
For real-time classification, four electrodes (TE1, TE2, TE3, and TE4) were used to estimate the sEMG signals (ESS1 and ESS2), as mentioned in the previous sections. The generation of reference signals, the demixing matrix calculation, and the training of the ANN were conducted offline, and the obtained demixing matrix, the network bias, and the layer weights were used in real-time classification. MATLAB software (version R2019a) was used for the calculation of the demixing matrix, the training of ANN, and the control of the robotic arm.[23] The communication and the control of the robot arm was carried out by the help of a PC, which has an AMD processor (2.80 Ghz) and 4.0 GB RAM, on a serial port. The classification duration of a gesture takes about 0.7 s after the motion is detected.
The sEMG signal was filtered with only a bandpass filter, which has the cut-off frequencies 20–500 Hz. There was no complex filtering technique applied to the raw data. The onset of sEMG signal activity was detected using the double-threshold method of et al.[24] This method operates on the raw myoelectric signal and does not require any envelope detection, so it is suitable for real time applications. A simple task was performed using an AX-12 robot arm. A small cube was placed on a desk, and the subject was asked to carry the cube to a target place using hand gestures. The subject moved the cube to the target place successfully. The photos taken during the real-time control of the robotic arm are shown in [Figure 8].{Figure 8}
Discussion
This study shows that the cICA algorithm increases the classification accuracy while reducing the dimension and offers a solution to the problem of placing electrodes at the same reference point in real-time applications.
Dimension reduction offers an extra advantage, such as decreasing the computational load of the classifier, because it deals with fewer signals instead of all records. Unlike other techniques, the dimension reduction is as simple as multiplying input signals using a demixing matrix.
In addition, the classification of six gestures was realized in this study. The number of gestures can be increased to control the unused axis of the robotic arm, but placing extra electrodes on the related muscles may be necessary.
In the real-time experiments, we did not focus on the duration of the classification. Thus, 0.7 s seems long, but the duration can be minimized by using the optimization on the algorithm and running the algorithm on a specific embedded system, such as Field Programmable Gate Arrays.
The results of this study can be applied to other complex biomedical signals, such as EEG, in order to increase its classification in HCI applications and in the diagnosis of disorders.
Conclusions
The success of the cICA algorithm was tested both in the offline classification of hand gestures and in controlling a robot arm using six different hand gestures. The offline classification results show that cICA increased the classification performances to 96.6% while providing data reduction. In the second application, the robot arm was controlled with high accuracy. This study shows that the cICA algorithm can be applied as a solution to the displacement problem of electrodes in real-time applications.
Patient informed consent
Patient informed consent was obtained.
Ethics committee approval
There is no need for ethics committee approval.
Conflicts of interest
There are no conflicts of interest to declare.
Financial support and sponsorship
No funding was received.
Author contribution subject and rate
Ulvi Baspinar (50%): Has conceived and designed the analysis, collected the data and performed the analysis. Yahya Tastan (25%): Wrote the paper, participated in its submission, and revised it.H. Selcuk Varol (25%): Has planned the study and revised the paper.
References
| 1 | Hermens H, Hermanus J, Bart F. The State of the Art on Sensors and Sensor Placement Procedures for Surface Electromyography: A Proposal for Sensor Placement Procedures. Netherlands: Roessingh Research and Development; 1997. |
| 2 | Varol HA, Sup F, Goldfarb M. Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Trans Biomed Eng 2010;57:542-51. |
| 3 | Hargrove L, Hudgins B, Englehart K, Leckey R. A comparison of surface and internally measured myoelectric signals for use in prosthetic control, engineering in medicine and biology society, 2006. EMBS'06. 28th Annual International Conference of the IEEE, IEEE, 2006. p. 2203-6. |
| 4 | Institute of Electrical and Electronics Engineers. Electrical Engineering Computing Science and Automatic Control (CCE), 2010 7th International Conference on : Date, 8-10 Sep. 2010. Institute of Electrical and Electronics Engineers; 2010. |
| 5 | Ibrahimy MI, Ahsan MR, Khalifa OO. Design and optimization of Levenberg-Marquardt based neural network classifier for EMG signals to identify hand motions. Meas Sci Rev 2013;13:142-51. |
| 6 | Naik GR, Kumar DK, Palaniswami M. Multi run ICA and Surface EMG Based Signal Processing System for Recognising Hand Gestures. Proceedings of the 8th IEEE International Conference on Computer and Information Technology (CIT 2008), 8-11 July 2008, Sydney, Australia. Published online 2008. p. 700-5. |
| 7 | Lu Z, Narayan A, Yu H. A deep learning based end-to-end locomotion mode detection method for lower limb wearable robot control. In: IEEE International Conference on Intelligent Robots and Systems. Las Vegas, NV, USA: Institute of Electrical and Electronics Engineers Inc.; 2020. p. 4091-7. |
| 8 | Stolyarov R, Carney M, Herr H. Accurate heuristic terrain prediction in powered lower-limb prostheses using onboard sensors. IEEE Trans Biomed Eng 2021;68:384-92. |
| 9 | Sueaseenak D, Wibirama S, Chanwimalueang T, Pintavirooj C, Sangworasil M. Comparison study of muscular-contraction classification between independent component analysis and artificial neural network. In: 2008 International Symposium on Communications and Information Technologies. Vientiane, Laos; ISCIT; 2008. p. 468-72. |
| 10 | Zhang ZL. Morphologically constrained ICA for extracting weak temporally correlated signals. Neurocomputing 2008;71:1669-79. |
| 11 | Park T, Lee M, Jeong T, Shin YI, Park SM. Quantitative analysis of EEG power spectrum and EMG median power frequency changes after continuous passive motion mirror therapy system. Sensors (Basel) 2020;20:2354. [Doi: 10.3390/s20082354]. |
| 12 | Hussain T, Iqbal N, Maqbool HF, Khan M, Awad MI, Dehghani-Sanij AA. Intent based recognition of walking and ramp activities for amputee using sEMG based lower limb prostheses. Biocybern Biomed Eng 2020;40:1110-23. |
| 13 | Kim Y, Stapornchaisit S, Miyakoshi M, Yoshimura N, Koike Y. The effect of ICA and non-negative matrix factorization analysis for EMG signals recorded from multi-channel EMG Sensors. Front Neurosci 2020;14:600804. |
| 14 | Dai C, Hu X. Independent component analysis based algorithms for high-density electromyogram decomposition: Systematic evaluation through simulation. Comput Biol Med 2019;109:171-81. |
| 15 | Karimi F, Kofman J, Mrachacz-Kersting N, Farina D, Jiang N. Detection of movement related cortical potentials from EEG using constrained ICA for brain-computer interface applications. Front Neurosci 2017;11:356. |
| 16 | Grushko S, Spurný T, Černý M. Control methods for transradial prostheses based on remnant muscle activity and its relationship with proprioceptive feedback. Sensors (Basel) 2020;20:4883. |
| 17 | Zhou Y, Liu J, Zeng J, Li K. Sciences HLSCT, 2019 undefined. Bio-signal based elbow angle and torque simultaneous prediction during isokinetic contraction. Springer Sci. China Technol. Sci 2019;62:21-30. https://doi.org/10.1007/s11431-018-9354-5. Available from: https://link.springer.com/article/10.1007/s11431-018-9354-5. [Last accessed on 2022 Nov 27]. |
| 18 | Abbaspour S, Lindén M, Gholamhosseini H, Naber A, Ortiz-Catalan M. Evaluation of surface EMG-based recognition algorithms for decoding hand movements. Med Biol Eng Comput 2020;58:83-100. |
| 19 | Khan AM, Sadiq A, Gul Khawaja S, Alghamdi NS, Akram MU, Saeed A. Physical action categorization using signal analysis and machine learning. arXiv preprint arXiv:2008.06971, 2020. |
| 20 | Basak H, Roy A, Lahiri JB. SVM and ANN based classification of EMG signals by using PCA and LDA Influenced by: Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals. arXiv preprint arXiv:2110.15279, 2021. |
| 21 | Batayneh W, Abdulhay E, Alothman M. Prediction of the performance of artificial neural networks in mapping sEMG to finger joint angles via signal pre-investigation techniques. Heliyon 2020;6:e03669. |
| 22 | Copaci D, Arias J, Gómez-Tomé M, Moreno L, Blanco D. sEMG-based gesture classifier for a rehabilitation glove. Front Neurorobot 2022;16:750482. |
| 23 | Ligutan DD, Abad AC, Dadios EP. Adaptive Robotic Arm Control Using Artificial Neural Network. 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management, HNICEM; 2018. Published online March 12, 2019. |
| 24 | Bonato P, D'Alessio T, Knaflitz M. A statistical method for the measurement of muscle activation intervals from surface myoelectric signal during gait. IEEE Trans Biomed Eng 1998;45:287-99. |
|
