Real-time Immersive Human-computer Interaction Based on Tracking and Recognition of Dynamic Hand Gestures

Real-time Immersive Human-computer Interaction Based on Tracking and Recognition of Dynamic Hand Gestures
Author: Gan Lu
Publisher:
Total Pages:
Release: 2011
Genre:
ISBN:
GET BOOK

Download Real-time Immersive Human-computer Interaction Based on Tracking and Recognition of Dynamic Hand Gestures Book in PDF, Epub and Kindle

With fast developing and ever growing use of computer based technologies, human-computer interaction (HCI) plays an increasingly pivotal role. In virtual reality (VR), HCI technologies provide not only a better understanding of three-dimensional shapes and spaces, but also sensory immersion and physical interaction. With the hand based HCI being a key HCI modality for object manipulation and gesture based communication, challenges are presented to provide users a natural, intuitive, effortless, precise, and real-time method for HCI based on dynamic hand gestures, due to the complexity of hand postures formed by multiple joints with high degrees-of-freedom, the speed of hand movements with highly variable trajectories and rapid direction changes, and the precision required for interaction between hands and objects in the virtual world. Presented in this thesis is the design and development of a novel real-time HCI system based on a unique combination of a pair of data gloves based on fibre-optic curvature sensors to acquire finger joint angles, a hybrid tracking system based on inertia and ultrasound to capture hand position and orientation, and a stereoscopic display system to provide an immersive visual feedback. The potential and effectiveness of the proposed system is demonstrated through a number of applications, namely, hand gesture based virtual object manipulation and visualisation, hand gesture based direct sign writing, and hand gesture based finger spelling. For virtual object manipulation and visualisation, the system is shown to allow a user to select, translate, rotate, scale, release and visualise virtual objects (presented using graphics and volume data) in three-dimensional space using natural hand gestures in real-time. For direct sign writing, the system is shown to be able to display immediately the corresponding SignWriting symbols signed by a user using three different signing sequences and a range of complex hand gestures, which consist of various combinations of hand postures (with each finger open, half-bent, closed, adduction and abduction), eight hand orientations in horizontal/vertical plans, three palm facing directions, and various hand movements (which can have eight directions in horizontal/vertical plans, and can be repetitive, straight/curve, clockwise/anti-clockwise). The development includes a special visual interface to give not only a stereoscopic view of hand gestures and movements, but also a structured visual feedback for each stage of the signing sequence. An excellent basis is therefore formed to develop a full HCI based on all human gestures by integrating the proposed system with facial expression and body posture recognition methods. Furthermore, for finger spelling, the system is shown to be able to recognise five vowels signed by two hands using the British Sign Language in real-time.

Real-Time Immersive Human-Computer Interaction

Real-Time Immersive Human-Computer Interaction
Author: Gan Lu
Publisher: LAP Lambert Academic Publishing
Total Pages: 196
Release: 2012
Genre:
ISBN: 9783847348870
GET BOOK

Download Real-Time Immersive Human-Computer Interaction Book in PDF, Epub and Kindle

Presented in this thesis is the design and development of a novel real-time HCI system based on a unique combination of a pair of data gloves based on fibre-optic curvature sensors to acquire finger joint angles, a hybrid tracking system based on inertia and ultrasound sensors to capture hand position and orientation, and a stereoscopic display to provide an immersive visual feedback. The potential and effectiveness of the proposed system is demonstrated through a number of hand gesture based applications, namely, virtual object manipulation and visualisation, direct sign writing, and finger spelling. For virtual object manipulation and visualisation, the system is shown to allow a user to manipulate and visualise virtual objects in 3D space using natural hand gestures in real-time. For direct sign writing, the system is shown to be able to display immediately the corresponding SignWriting symbols signed by a user using a range of complex hand gestures. Furthermore, for finger spelling, the system is shown to be able to recognise five vowels signed by two hands using the British Sign Language in real-time.

Human Computer Interaction Using Hand Gestures

Human Computer Interaction Using Hand Gestures
Author: Prashan Premaratne
Publisher: Springer Science & Business Media
Total Pages: 182
Release: 2014-03-20
Genre: Technology & Engineering
ISBN: 9814585696
GET BOOK

Download Human Computer Interaction Using Hand Gestures Book in PDF, Epub and Kindle

Human computer interaction (HCI) plays a vital role in bridging the 'Digital Divide', bringing people closer to consumer electronics control in the 'lounge'. Keyboards and mouse or remotes do alienate old and new generations alike from control interfaces. Hand Gesture Recognition systems bring hope of connecting people with machines in a natural way. This will lead to consumers being able to use their hands naturally to communicate with any electronic equipment in their 'lounge.' This monograph will include the state of the art hand gesture recognition approaches and how they evolved from their inception. The author would also detail his research in this area for the past 8 years and how the future might turn out to be using HCI. This monograph will serve as a valuable guide for researchers (who would endeavour into) in the world of HCI.

Real-time Hand Gesture Detection and Recognition for Human Computer Interaction

Real-time Hand Gesture Detection and Recognition for Human Computer Interaction
Author: Nasser Hasan Abdel-Qader Dardas
Publisher:
Total Pages:
Release: 2012
Genre: Gesture
ISBN:
GET BOOK

Download Real-time Hand Gesture Detection and Recognition for Human Computer Interaction Book in PDF, Epub and Kindle

This thesis focuses on bare hand gesture recognition by proposing a new architecture to solve the problem of real-time vision-based hand detection, tracking, and gesture recognition for interaction with an application via hand gestures. The first stage of our system allows detecting and tracking a bare hand in a cluttered background using face subtraction, skin detection and contour comparison. The second stage allows recognizing hand gestures using bag-of-features and multi-class Support Vector Machine (SVM) algorithms. Finally, a grammar has been developed to generate gesture commands for application control. Our hand gesture recognition system consists of two steps: offline training and online testing. In the training stage, after extracting the keypoints for every training image using the Scale Invariance Feature Transform (SIFT), a vector quantization technique will map keypoints from every training image into a unified dimensional histogram vector (bag-of-words) after K-means clustering. This histogram is treated as an input vector for a multi-class SVM to build the classifier. In the testing stage, for every frame captured from a webcam, the hand is detected using my algorithm. Then, the keypoints are extracted for every small image that contains the detected hand posture and fed into the cluster model to map them into a bag-of-words vector, which is fed into the multi-class SVM classifier to recognize the hand gesture. Another hand gesture recognition system was proposed using Principle Components Analysis (PCA). The most eigenvectors and weights of training images are determined. In the testing stage, the hand posture is detected for every frame using my algorithm. Then, the small image that contains the detected hand is projected onto the most eigenvectors of training images to form its test weights. Finally, the minimum Euclidean distance is determined among the test weights and the training weights of each training image to recognize the hand gesture. Two application of gesture-based interaction with a 3D gaming virtual environment were implemented. The exertion videogame makes use of a stationary bicycle as one of the main inputs for game playing. The user can control and direct left-right movement and shooting actions in the game by a set of hand gesture commands, while in the second game, the user can control and direct a helicopter over the city by a set of hand gesture commands.

Real-Time Vision for Human-Computer Interaction

Real-Time Vision for Human-Computer Interaction
Author: Branislav Kisacanin
Publisher: Springer Science & Business Media
Total Pages: 324
Release: 2005-08-23
Genre: Computers
ISBN: 9780387276977
GET BOOK

Download Real-Time Vision for Human-Computer Interaction Book in PDF, Epub and Kindle

The need for natural and effective Human-Computer Interaction (HCI) is increasingly important due to the prevalence of computers in human activities. Computer vision and pattern recognition continue to play a dominant role in the HCI realm. However, computer vision methods often fail to become pervasive in the field due to the lack of real-time, robust algorithms, and novel and convincing applications. This state-of-the-art contributed volume is comprised of articles by prominent experts in computer vision, pattern recognition and HCI. It is the first published text to capture the latest research in this rapidly advancing field with exclusive focus on real-time algorithms and practical applications in diverse and numerous industries, and it outlines further challenges in these areas. Real-Time Vision for Human-Computer Interaction is an invaluable reference for HCI researchers in both academia and industry, and a useful supplement for advanced-level courses in HCI and Computer Vision.

Real-time 2D Static Hand Gesture Recognition and 2D Hand Tracking for Human-Computer Interaction

Real-time 2D Static Hand Gesture Recognition and 2D Hand Tracking for Human-Computer Interaction
Author: Pavel Alexandrovich Popov
Publisher:
Total Pages:
Release: 2020
Genre:
ISBN:
GET BOOK

Download Real-time 2D Static Hand Gesture Recognition and 2D Hand Tracking for Human-Computer Interaction Book in PDF, Epub and Kindle

The topic of this thesis is Hand Gesture Recognition and Hand Tracking for user interface applications. 3 systems were produced, as well as datasets for recognition and tracking, along with UI applications to prove the concept of the technology. These represent significant contributions to resolving the hand recognition and tracking problems for 2d systems. The systems were designed to work in video only contexts, be computationally light, provide recognition and tracking of the user's hand, and operate without user driven fine tuning and calibration. Existing systems require user calibration, use depth sensors and do not work in video only contexts, or are computationally heavy requiring GPU to run in live situations. A 2-step static hand gesture recognition system was created which can recognize 3 different gestures in real-time. A detection step detects hand gestures using machine learning models. A validation step rejects false positives. The gesture recognition system was combined with hand tracking. It recognizes and then tracks a user's hand in video in an unconstrained setting. The tracking uses 2 collaborative strategies. A contour tracking strategy guides a minimization based template tracking strategy and makes it real-time, robust, and recoverable, while the template tracking provides stable input for UI applications. Lastly, an improved static gesture recognition system addresses the drawbacks due to stratified colour sampling of the detection boxes in the detection step. It uses the entire presented colour range and clusters it into constituent colour modes which are then used for segmentation, which improves the overall gesture recognition rates. One dataset was produced for static hand gesture recognition which allowed for the comparison of multiple different machine learning strategies, including deep learning. Another dataset was produced for hand tracking which provides a challenging series of user scenarios to test the gesture recognition and hand tracking system. Both datasets are significantly larger than other available datasets. The hand tracking algorithm was used to create a mouse cursor control application, a paint application for Android mobile devices, and a FPS video game controller. The latter in particular demonstrates how the collaborating hand tracking can fulfill the demanding nature of responsive aiming and movement controls.

Novel Cost Measures for Robust Recognition of Dynamic Hand Gestures

Novel Cost Measures for Robust Recognition of Dynamic Hand Gestures
Author: Ameya Kulkarni
Publisher:
Total Pages:
Release: 2011
Genre:
ISBN:
GET BOOK

Download Novel Cost Measures for Robust Recognition of Dynamic Hand Gestures Book in PDF, Epub and Kindle

Computer vision aided automatic hand gesture recognition system plays a vital role in real world human computer interaction applications such as sign language recognition, game controls, virtual reality, intelligent home appliances and assistive robotics. In such systems, when input with a video sequence, the challenging task is to locate the gesturing hand (spatial segmentation) and determine when the gesture starts and ends (temporal segmentation). In this thesis, we use a framework which at its principal has a dynamic space time warping (DSTW) algorithm to simultaneously localize gesturing hand, to find an optimal alignment in time domain between query-model sequences and to compute a matching cost (a measure of how well the query sequence matches with the model sequence) for the query-model pair. Within the context of DSTW, the thesis proposes few novel cost measures to improve the performance of the framework for robust recognition of hand gesture with the help of translation and scale invariant feature vectors extracted at each frame of the input video. The performance of the system is evaluated in a real world scene with cluttered background and in presence of multiple moving skin colored distractors in the background.

Robust Hand Gesture Recognition for Robotic Hand Control

Robust Hand Gesture Recognition for Robotic Hand Control
Author: Ankit Chaudhary
Publisher: Springer
Total Pages: 108
Release: 2017-06-05
Genre: Technology & Engineering
ISBN: 9811047987
GET BOOK

Download Robust Hand Gesture Recognition for Robotic Hand Control Book in PDF, Epub and Kindle

This book focuses on light invariant bare hand gesture recognition while there is no restriction on the types of gestures. Observations and results have confirmed that this research work can be used to remotely control a robotic hand using hand gestures. The system developed here is also able to recognize hand gestures in different lighting conditions. The pre-processing is performed by developing an image-cropping algorithm that ensures only the area of interest is included in the segmented image. The segmented image is compared with a predefined gesture set which must be installed in the recognition system. These images are stored and feature vectors are extracted from them. These feature vectors are subsequently presented using an orientation histogram, which provides a view of the edges in the form of frequency. Thereby, if the same gesture is shown twice in different lighting intensities, both repetitions will map to the same gesture in the stored data. The mapping of the segmented image's orientation histogram is firstly done using the Euclidian distance method. Secondly, the supervised neural network is trained for the same, producing better recognition results. An approach to controlling electro-mechanical robotic hands using dynamic hand gestures is also presented using a robot simulator. Such robotic hands have applications in commercial, military or emergency operations where human life cannot be risked. For such applications, an artificial robotic hand is required to perform real-time operations. This robotic hand should be able to move its fingers in the same manner as a human hand. For this purpose, hand geometry parameters are obtained using a webcam and also using KINECT. The parameter detection is direction invariant in both methods. Once the hand parameters are obtained, the fingers’ angle information is obtained by performing a geometrical analysis. An artificial neural network is also implemented to calculate the angles. These two methods can be used with only one hand, either right or left. A separate method that is applicable to both hands simultaneously is also developed and fingers angles are calculated. The contents of this book will be useful for researchers and professional engineers working on robotic arm/hand systems.

Gesture Recognition

Gesture Recognition
Author: Qiguang Miao
Publisher: Elsevier
Total Pages: 225
Release: 2024-07-26
Genre: Computers
ISBN: 0443289603
GET BOOK

Download Gesture Recognition Book in PDF, Epub and Kindle

Gesture Recognition: Theory and Applications covers this important topic in computer science and language technology that has a goal of interpreting human gestures via mathematical algorithms. The book begins by examining the computer vision-based gesture recognition method, focusing on the theory and related research results of various recent gesture recognition technologies. The book takes the evolutions of gesture recognition technology as a clue, systematically introducing gesture recognition methods based on handcrafted features, convolutional neural networks, recurrent neural networks, multimodal data fusion, and visual attention mechanisms. Three gesture recognition-based HCI (Human Computer Interaction) practical cases are introduced. Finally, the book looks at emerging research trends and application. Focuses on the theory and application of gesture recognition, providing a systematic introduction to commonly used datasets in the field as well as algorithms based on handcrafted features, convolutional neural networks, multimodal fusion, and attention mechanisms Introduces the practical applications of gesture recognition in real-world scenarios, enabling readers to enhance their practical application skills while learning about relevant technologies Demonstrates four main categories of gesture recognition methods and analyzes their associated challenges

Entertainment Computing

Entertainment Computing
Author: Ryohei Nakatsu
Publisher: Springer
Total Pages: 538
Release: 2013-04-17
Genre: Computers
ISBN: 0387356606
GET BOOK

Download Entertainment Computing Book in PDF, Epub and Kindle

This volume is the Proceedings of the First International Workshop on Entertainment Computing (IWEC 2002). Entertainment has been taking very important parts in our life by refreshing us and activating our creativity. Recently by the advancement of computers and networks new types of entertainment have been emerging such as video games, entertainment robots, and network games. As these new games have a strong power to change our lives, it is good time for people who work in this area to discuss various aspects of entertainment and to promote entertainment related researches. Based on these considerations, we have organized a first workshop on entertainment computing. This workshop brings together researchers, developers, and practitioners working in the area of entertainment computing. It covers wide range of entertainment computing such as theoretical issues, hardware/software issues, systems, human interfaces, and applications. The particular areas covered by the workshop are: 1. Computers & Games Computer game algorithms, modeling of players, web technologies for networked games, human interface technologies for game applications. 2. Home/Arcade Games and Interactive Movies Video game computer technologies, motion capture technologies, real-time computer graphics technologies, interactive movie systems, story generation for games/movies, human factors of video games.