Kaiserslautern - Fachbereich Informatik
Refine
Document Type
- Doctoral Thesis (2)
Language
- English (2)
Has Fulltext
- yes (2)
Keywords
- Pattern Recognition (2) (remove)
Faculty / Organisational entity
This thesis investigates how smart sensors can quantify the process of learning. Traditionally, human beings have obtained various skills by inventing technologies. Those who integrate technologies into daily life and enhance their capabilities are called augmented humans. While most existing augmenting human technologies focus on directly assisting specific skills, the objective of this thesis is to assist learning -- the meta-skill to master new skills -- with the aim of long-term augmentations.
Learning consists of cognitive activities such as reading, writing, and watching. It has been considered that tracking them by motion sensors (in the same way as the recognition of physical activities) is a challenging task because dynamic body movements could not be observed during cognitive activities. I have solved this problem with smart sensors monitoring eye movements and physiological signals.
I propose activity recognition methods using sensors built into eyewear computers. Head movements and eye blinks measured by an infrared proximity sensor on Google Glass could classify five activities including reading with 82% accuracy. Head and eye movements measured by electrooculography on JINS MEME could classify four activities with 70% accuracy. In a wild experiment involving seven participants who wore JINS MEME more than two weeks, deep neural networks could detect natural reading activities with 74% accuracy. I demonstrate Wordometer 2.0, an application to estimate the number of rear words on JINS MEME, which was evaluated in a dataset involving five readers with 11% error rate.
Smart sensors can recognize not only activities but also internal states during the activities. I present an expertise recognition method using an eye tracker which performs 70% classification accuracy into three classes using one minute data of reading a textbook, a positive correlation between interest and pupil diameter (p < 0.01), a negative correlation between mental workload and nose temperature measured by an infrared thermal camera (p < 0.05), an interest detection on newspaper articles, and effective gaze and physiological features to estimate self-confidence while solving multiple choice questions and spelling tests of English vocabulary.
The quantified learning process can be utilized for feedback to each learner on the basis of the context. I present HyperMind, an interactive intelligent digital textbook. It can be developed on HyperMind Builder which may be employed to augment any electronic text by multimedia aspects activated via gaze.
Applications mentioned above have already been deployed at several laboratories including Immersive Quantified Learning Lab (iQL-Lab) at the German Research Center for Artificial Intelligence (DFKI).
The recognition of patterns and structures has gained importance for dealing with the growing amount of data being generated by sensors and simulations. Most existing methods for pattern recognition are tailored for scalar data and non-correlated data of higher dimensions. The recognition of general patterns in flow structures is possible, but not yet practically usable, due to the high computation effort. The main goal of this work is to present methods for comparative visualization of flow data, amongst others, based on a new method for efficient pattern recognition on flow data. This work is structured in three parts: At first, a known feature-based approach for pattern recognition on flow data, the Clifford convolution, has been applied to color edge detection, and been extended to non-uniform grids. However, this method is still computationally expensive for a general pattern recognition, since the recognition algorithm has to be applied for numerous different scales and orientations of the query pattern. A more efficient and accurate method for pattern recognition on flow data is presented in the second part. It is based upon a novel mathematical formulation of moment invariants for flow data. The common moment invariants for pattern recognition are not applicable on flow data, since they are only invariant on non-correlated data. Because of the spatial correlation of flow data, the moment invariants had to be redefined with different basis functions to satisfy the demands for an invariant mapping of flow data. The computation of the moment invariants is done by a multi-scale convolution of the complete flow field with the basis functions. This pre-processing computation time almost equals the time for the pattern recognition of one single general pattern with the former algorithms. However, after having computed the moments once, they can be indexed and used as a look-up-table to recognize any desired pattern quickly and interactively. This results in a flexible and easy-to-use tool for the analysis of patterns in 2d flow data. For an improved rendering of the recognized features, an importance driven streamline algorithm has been developed. The density of the streamlines can be adjusted by using importance maps. The result of a pattern recognition can be used as such a map, for example. Finally, new comparative flow visualization approaches utilizing the streamline approach, the flow pattern matching, and the moment invariants are presented.