Kaiserslautern - Fachbereich Informatik
Refine
Year of publication
Document Type
- Doctoral Thesis (235) (remove)
Has Fulltext
- yes (235)
Keywords
- Visualisierung (20)
- Visualization (8)
- Deep Learning (7)
- Computergraphik (5)
- Evaluation (4)
- Robotik (4)
- Artificial Intelligence (3)
- Bildverarbeitung (3)
- Geoinformationssystem (3)
- Machine Learning (3)
- Mustererkennung (3)
- Optische Zeichenerkennung (3)
- Ray casting (3)
- Recommender Systems (3)
- Scientific Visualization (3)
- Semantic Web (3)
- Unsicherheit (3)
- computer graphics (3)
- document analysis (3)
- machine learning (3)
- optical character recognition (3)
- verification (3)
- visualization (3)
- AG-RESY (2)
- Activity recognition (2)
- Algorithmus (2)
- Automatische Differentiation (2)
- Code Generation (2)
- Computational Fluid Dynamics (2)
- Datenanalyse (2)
- Datenbank (2)
- Dreidimensionale Bildverarbeitung (2)
- Effizienter Algorithmus (2)
- Endlicher Automat (2)
- Implementierung (2)
- Interaction (2)
- Interaktion (2)
- Layout (2)
- Machine learning (2)
- Mensch-Maschine-Kommunikation (2)
- Merkmalsextraktion (2)
- Model Checking (2)
- Modellierung (2)
- Natural Language Processing (2)
- Network Calculus (2)
- Optimierung (2)
- Pattern Recognition (2)
- Raumakustik (2)
- Room acoustics (2)
- SDL (2)
- Scattered-Data-Interpolation (2)
- Simulation (2)
- Softwareentwicklung (2)
- Topology (2)
- UML (2)
- Uncertain Data (2)
- Uncertainty Visualization (2)
- Visual Analytics (2)
- Voronoi-Diagramm (2)
- Wearable computing (2)
- WiFi (2)
- artificial intelligence (2)
- computer vision (2)
- deep learning (2)
- image processing (2)
- layout analysis (2)
- semantic web (2)
- virtual acoustics (2)
- 3D Gene Expression (1)
- 3D Point Data (1)
- ASM (1)
- AUTOSAR (1)
- Ableitungsschätzung (1)
- Abrechnungsmanagement (1)
- Abstraction (1)
- Abstraction-Based Controller Design (1)
- Abstraktion (1)
- Accelerometer (1)
- Accounting Agent (1)
- Ad-hoc networks (1)
- Ad-hoc-Netz (1)
- Adaptive Data Structure (1)
- Affine Arithmetic (1)
- Akquisition (1)
- Algorithm (1)
- Algorithmic Differentiation (1)
- Amharic, Attention, Factored Convolutional Neural Network, OCR (1)
- Anomaly Detection (1)
- Application Framework (1)
- Association (1)
- Ausdrucksfähig (1)
- Ausdrucksfähigkeit (1)
- Automat <Automatentheorie> (1)
- Automatic Differentiation (1)
- Automatic Image Captioning (1)
- Automation (1)
- Automatisches Beweisverfahren (1)
- Autonomer Agent (1)
- Autonomer Roboter (1)
- Bahnplanung (1)
- Befahrbarkeitsanalyse (1)
- Benutzer (1)
- Beschränkte Arithmetik (1)
- Bibligraphic References (1)
- Bio-inspired (1)
- Bioinformatik (1)
- Bipedal Locomotion (1)
- Bluetooth (1)
- Boosting (1)
- CBR (1)
- Caching (1)
- Carbon footprint (1)
- Channel Hopping (1)
- Channel Scheduling (1)
- Channel hopping (1)
- Channel sensing (1)
- Classification (1)
- Classification of biomedical signals (1)
- Cluster-Analyse (1)
- Clustering (1)
- Cognitive Amplification (1)
- Collaboration (1)
- Combinatorial Testing (1)
- Computer Graphic (1)
- Computer Supported Cooperative Work (1)
- Computer graphics (1)
- Computer-Aided Diagnosis (1)
- Computerphysik (1)
- Computersimulation (1)
- Computertomographie (1)
- Computervision (1)
- Concurrent data structures (1)
- Congitive Radio Networks (1)
- Containertypen (1)
- Containertypes (1)
- Context-sensitive Assistance (1)
- Controller Synthesis (1)
- Correct-by-Design Controller Synthesis (1)
- Cyber-Physical Systems (1)
- Cycle Accuracy (1)
- DCE <Programm> (1)
- DFG (1)
- DPN (1)
- Data Modeling (1)
- Dataset (1)
- Datenbanken (1)
- Datenfusion (1)
- Decision Support Systems (1)
- Delaunay-Triangulierung (1)
- Derivative Estimation (1)
- Dienstgüte (1)
- Dienstschnittstellen (1)
- Direct Numerical Simulation (1)
- Discrete Event Simulation (DES) (1)
- Distributed Rendering (1)
- Distributed system (1)
- Dreidimensionale Rekonstruktion (1)
- Dreidimensionale Strömung (1)
- Duplicate Identification (1)
- Duplikaterkennung (1)
- Dynamischer Test (1)
- ESTELLE (1)
- Effizienz (1)
- Effizienzsteigerung (1)
- Empfehlungssysteme (1)
- Energie (1)
- Engineering 4.0 (1)
- Ensemble Visualization (1)
- Entwurf (1)
- Eventual consistency (1)
- Evolutionary Algorithm (1)
- Experiment (1)
- Experimentation (1)
- Explainability (1)
- Explainable Artificial Intelligence (1)
- Exposed Datapath Architectures (1)
- Expressiveness (1)
- Extrapolation (1)
- Eyewear Computing (1)
- FERAL (1)
- Fahrtkostenmodelle (1)
- Fast Mode-Signaling (1)
- Fault Tree Analysis (1)
- Feasibility study (1)
- Feature (1)
- Feature Detection (1)
- Feature Extraction (1)
- Fehlerbaumanalyse (1)
- Fließanalyse (1)
- Flow Visualization (1)
- Forgetting-enabled Information Systems (1)
- Formale Beschreibungstechnik (1)
- Formale Grammatik (1)
- Formale Sprache (1)
- Formaler Beweis (1)
- Framework (1)
- Frequenzsprungverfahren (1)
- Functional Safety (1)
- Funktionale Sicherheit (1)
- Fusion (1)
- Future Internet (1)
- GPU (1)
- Gateway (1)
- Gauß-Filter (1)
- Gefahren- und Risikoanalyse (1)
- Gene expression programming (1)
- Generierung (1)
- Geovisualization (1)
- Google Earth (1)
- HPC (1)
- Hazard Analysis (1)
- Heterogeneous (1)
- Hitting families (1)
- Human Pose (1)
- Human-Computer Interaction (1)
- Human-Robot-Coexistence (1)
- Human-Robot-Cooperation (1)
- Hybrid CBR (1)
- Hypergraph (1)
- IEEE 802.15.4 (1)
- IMU (1)
- IP Address (1)
- IP Traffic Accounting (1)
- IP-XACT (1)
- ISO 26262 (1)
- ISO26262 (1)
- Image Processing (1)
- Imote2 (1)
- Implementation (1)
- Incremental recomputation (1)
- Induktive logische Programmierung (1)
- Information Extraction (1)
- Information Management (1)
- Information Visualization (1)
- Intensity estimation (1)
- Intentional Forgetting (1)
- Intermediate Composition (1)
- Interpolation (1)
- Invariante (1)
- Invariante Momente (1)
- Kellerautomat (1)
- Klassifikation (1)
- Klima (1)
- Knowledge Management (1)
- Knowledge Work (1)
- Knuth-Bendix completion (1)
- Knuth-Bendix-Vervollständigung (1)
- Kommunikation (1)
- Komponentenmodell (1)
- Kompression (1)
- Kontextbezogenes System (1)
- Korrelationsanalyse (1)
- Kurve (1)
- Künstliche Intelligenz (1)
- LIDAR (1)
- LIR-Tree (1)
- LOADBAL (1)
- Large Data (1)
- Large High-Resolution Displays (1)
- Laser Wakefield Particle Accelerator (1)
- Learning Analytics (1)
- Leistungsmessung (1)
- Link Metric (1)
- Linked Data (1)
- Linking Data Analysis and Visualization (1)
- Lokalisierung (1)
- Lärmbelastung (1)
- Lärmimmission (1)
- MAC protocols (1)
- MIP-Emissionsspektroskopie (1)
- MIP-Massenspektrometrie (1)
- Magnetfeldbasierter Lokalisierung (1)
- Magnetfelder (1)
- Magnetometer (1)
- Manufacturing (1)
- MapReduce (1)
- Maschinelle Übersetzung (1)
- Matrix Completion (1)
- Maximum Intensity Projection (1)
- Medical Image Analysis (1)
- Mehragentensystem (1)
- Memory Architecture (1)
- Memory Consistency (1)
- Mensch-Roboter-Koexistenz (1)
- Mensch-Roboter-Kooperation (1)
- Menschenmenge (1)
- Merkmalsraum (1)
- Mesh-Free (1)
- Meter (1)
- Microelectromechanical Systems (1)
- Mikroskopie (1)
- Minimal Cut Set Visualization (1)
- Minimal training (1)
- Mobile Computing (1)
- Mobile system (1)
- Mobiler Roboter (1)
- Mode-Based Scheduling with Fast Mode-Signaling (1)
- Modellgetriebene Entwicklung (1)
- Modelling (1)
- Modularisierung (1)
- Modusbasierte Signalisierung (1)
- Molekulare Bioinformatik (1)
- Moment Invariants (1)
- Moment-Generating Functions (1)
- Mood-based Music Recommendations (1)
- Multi-Edge Graph (1)
- Multi-Field (1)
- Multi-Variate Data (1)
- Multidisciplinary Optimization (1)
- Multifield Data (1)
- Nachhaltigkeit (1)
- Natural Neighbor (1)
- Natural Neighbor Interpolation (1)
- Natürliche Nachbarn (1)
- Navigation (1)
- Network Architecture (1)
- Netzwerk (1)
- Neural Architecture Search (1)
- Neural Networks (1)
- NoSQL (1)
- Node-Link Diagram (1)
- Numerische Strömungssimulation (1)
- OCR (1)
- OWL (1)
- Object-orientation (1)
- Objektorientierung (1)
- Off-road Robotics (1)
- Off-road Robotik (1)
- Online chain partitioning (1)
- Open Estelle (1)
- Optimierender Compiler (1)
- Optimization (1)
- Paralleler Hybrid (1)
- Pareto Optimality (1)
- Partial Differential Equations (1)
- Partially ordered sets (1)
- Participatory Sensing (1)
- Performance (1)
- Personalisation (1)
- Pervasive health (1)
- Physical activity monitoring (1)
- Planar Pressure (1)
- Privacy (1)
- Process Data (1)
- Processor Architecture (1)
- Processor Architectures (1)
- Protocol Composition (1)
- Prototyp (1)
- Prototype (1)
- Prozessvisualisierung (1)
- Quality (1)
- Qualität (1)
- Quantitative Bildanalyse (1)
- Quartz (1)
- Quicksort (1)
- Random testing (1)
- Rate Gyro (1)
- Raumordnung (1)
- Ray tracing (1)
- Rechtecksgitter (1)
- Rectilinear Grid (1)
- Redundanzvermeidung (1)
- Regelung (1)
- Reinforcement Learning (1)
- Rekonstruktion (1)
- Representation (1)
- Risikobewertung (1)
- Risk Assessment (1)
- Roboter (1)
- Robotics (1)
- Routing (1)
- Räumliche Statistik (1)
- SAHARA (1)
- SCAD (1)
- SDL extensions (1)
- SIMERO (1)
- SPARQL (1)
- SPARQL query learning (1)
- Safety Analysis (1)
- Scalar (1)
- Scheduling (1)
- Schema <Informatik> (1)
- Schematisation (1)
- Schematisierung (1)
- Scientific Community Analysis (1)
- Scientific Computing (1)
- Self-organization (1)
- Self-supervised Learning (1)
- Semantic Desktop (1)
- Semantic Index (1)
- Semantic Wikis (1)
- Semantische Modellierung (1)
- Semantisches Datenmodell (1)
- Sensing (1)
- Sensor (1)
- Sensor Fusion (1)
- Sensors (1)
- Service Access Points (1)
- Shared Resource Modeling (1)
- Sicherheitsanalyse (1)
- Similarity Join (1)
- Similarity Joins (1)
- Simulationen (1)
- Skalar (1)
- Smart City (1)
- Smart Device (1)
- Smart Mobile Device (1)
- Smart Textile (1)
- Smartphone (1)
- Smartwatch (1)
- Socio-Semantic Web (1)
- Software (1)
- Software Comprehension (1)
- Software Dependencies (1)
- Software Engineering (1)
- Software Evolution (1)
- Software Maintenance (1)
- Software Measurement (1)
- Software Testing (1)
- Software Visualization (1)
- Software engineering (1)
- Software transactional memory (1)
- Software-Architektur (1)
- Softwaremetrie (1)
- Softwarespezifikation (1)
- Softwarewartung (1)
- Sprachdefinition (1)
- Sprache (1)
- Sprachprofile (1)
- Statistical Independence (1)
- Stimmungsbasierte Musikempfehlungen (1)
- Stochastic Dependence (1)
- Stochastic Network Calculus (1)
- Stokes Equations (1)
- Streaming (1)
- Strukturiertes Gitter (1)
- Strömung (1)
- Surface Reconstruction (1)
- Symbolic Methods (1)
- Synchronous Control Asynchronous Dataflow (1)
- SystemC (1)
- Systemarchitektur (1)
- Systemdesign (1)
- Task-based (1)
- Temporal Decoupling (1)
- Temporal Logic (1)
- Temporal data processing (1)
- Tensor (1)
- Tensorfeld (1)
- Tesselation (1)
- Tetraeder (1)
- Tetraedergi (1)
- Tetrahedral Grid (1)
- Tetrahedral Mesh (1)
- Textual CBR (1)
- Themenbasierte Empfehlungen von Ressourcen (1)
- Time Series (1)
- Time series classification (1)
- Time-motion-Ultraschallkardiographie (1)
- Time-slotted (1)
- Topic-based Resource Recommendations (1)
- Topologie (1)
- Topology visualization (1)
- Transaction Level Modeling (TLM) (1)
- Transport Protocol (1)
- Traversability Analysis (1)
- UML Activity (1)
- Ubiquitous system (1)
- Ultraschallkardiographie (1)
- Umweltinformatik (1)
- Uncertainty Estimation (1)
- Unobtrusive instrumentations (1)
- Unorganized Data (1)
- Unstrukturiertes Gitter (1)
- Urban sprawl (1)
- UrbanSim (1)
- User Model (1)
- VIACOBI (1)
- Validierung (1)
- Vector (1)
- Vector Field (1)
- Vektor (1)
- Vektorfelder (1)
- Verifikation (1)
- Verzerrungstensor (1)
- Virtual Prototyping (1)
- Visual Queries (1)
- Visualization Theory (1)
- Volume rendering (1)
- Volumen-Rendering (1)
- Voronoi diagram (1)
- Weak Memory Model (1)
- Wearable Computing (1)
- Wi-Fi (1)
- Wide-column stores (1)
- Wissenschaftliches Rechnen (1)
- XDBMS (1)
- XEC (1)
- XML (1)
- XML query estimation (1)
- XML summary (1)
- Yaroslavskiy-Bentley-Bloch Quicksort (1)
- Zielverfolgung (1)
- acoustic modeling (1)
- affective user interface (1)
- affine arithmetic (1)
- analysis of algorithms (1)
- artificial neural network (1)
- associations (1)
- autoencoder (1)
- automated theorem proving (1)
- automotive (1)
- average-case analysis (1)
- behaviour-based system (1)
- binary countdown protocol (1)
- black bursts (1)
- body-IMU calibration (1)
- classification (1)
- climate (1)
- clustering (1)
- collaborative information visualization (1)
- collaborative mobile sensing (1)
- collective intelligence (1)
- computational biology (1)
- computational fluid dynamics (1)
- computer-supported cooperative work (1)
- concurrent (1)
- constraint exploration (1)
- content-and-structure summary (1)
- continuous master theorem (1)
- crowd condition estimation (1)
- crowd density estimation (1)
- crowd scanning (1)
- crowd sensing (1)
- crowdsourcing (1)
- curves and surfaces (1)
- data race (1)
- data sets (1)
- data-flow (1)
- dataset (1)
- decidability (1)
- dependable systems (1)
- design (1)
- deterministic arbitration (1)
- directed graphs (1)
- distributed (1)
- distributed real-time systems (1)
- distributed tasks (1)
- embedded (1)
- embedding (1)
- emotion visualization (1)
- end-to-end learning (1)
- ensemble (1)
- environmental noise (1)
- evolutionary algorithm (1)
- explainability (1)
- firewall (1)
- flow visualization (1)
- foundational translation validation (1)
- gaussian filter (1)
- geographic information systems (1)
- geology (1)
- graph drawing algorithm (1)
- graph embedding (1)
- graph layout (1)
- hand pose, hand shape, depth image, convolutional neural networks (1)
- historical documents (1)
- human body motion tracking (1)
- hypergraph (1)
- iB2C (1)
- image analysis (1)
- implementation (1)
- inertial sensors (1)
- information systems (1)
- interference (1)
- interpolation (1)
- interval arithmetic (1)
- invariant (1)
- inverses Pendel (1)
- language definition (1)
- language modeling (1)
- language profiles (1)
- linked data (1)
- long short-term memory (1)
- long tail (1)
- machine-checkable proof (1)
- magnetic field based localization (1)
- magnetometer calibration (1)
- matrix visualization (1)
- message-passing (1)
- metadata (1)
- microelectronics ontology (1)
- mixed-signal (1)
- modularisation (1)
- moment (1)
- multicore (1)
- multidimensional datasets (1)
- multinomial regression (1)
- multithreading (1)
- multitype code coupling (1)
- multiway partitioning (1)
- nestable tangibles (1)
- optimization correctness (1)
- oscillating magnetic fields (1)
- out-of-order (1)
- parallel (1)
- participatory sensing (1)
- path cost models (1)
- pattern recognition (1)
- pivot sampling (1)
- point cloud (1)
- proof generating optimizer (1)
- quality assurance (1)
- ray casting (1)
- ray tracing (1)
- real-time tasks (1)
- reasoning (1)
- relaxed memory models (1)
- risk management (1)
- robustness (1)
- scalar field (1)
- scene flow (1)
- secondary structure prediction (1)
- sensor fusion (1)
- small-multiples node-link visualization (1)
- social media (1)
- software comprehension (1)
- software engineering (1)
- software engineering task (1)
- sparse-to-dense (1)
- spatial statistics (1)
- static software structure (1)
- stationary sensing (1)
- statistics (1)
- structural summary (1)
- subjective evaluation (1)
- symbolic simulation (1)
- synchronous (1)
- system architecture (1)
- tabletop (1)
- task sequence (1)
- tensor (1)
- tensorfield (1)
- terrain rendering (1)
- time-dependent (1)
- time-varying flow fields (1)
- touch surfaces (1)
- translation contract (1)
- ultrasound signals (1)
- urban planning (1)
- user-centered design (1)
- vector field visualization (1)
- vectorfield (1)
- virtual reality (1)
- visual analytics (1)
- visual structure (1)
- wearable systems (1)
- weighted finite-state transducers (1)
- wireless signal (1)
- xai (1)
- zeitabhängige Strömungen (1)
- Ähnlichkeit (1)
- Übersetzung (1)
Faculty / Organisational entity
Distributed message-passing systems have become ubiquitous and essential for our daily lives. Hence, designing and implementing them correctly is of utmost importance. This is, however, very challenging at the same time. In fact, it is well-known that verifying such systems is algorithmically undecidable in general due to the interplay of asynchronous communication (messages are buffered) and concurrency. When designing communication in a system, it is natural to start with a global protocol specification of the desired communication behaviour. In such a top-down approach, the implementability problem asks, given such a global protocol, if the specified behaviour can be implemented in a distributed setting without additional synchronisation. This problem has been studied from two perspectives in the literature. On the one hand, there are Multiparty Session Types (MSTs) from process algebra, with global types to specify protocols. Key to the MST approach is a so-called projection operator, which takes a global type and tries to project it onto every participant: if successful, the local specifications are safe to use. This approach is efficient but brittle. On the other hand, High-level Message Sequence Charts (HMSCs) study the implementability problem from an automata-theoretic perspective. They employ very few restrictions on protocol specifications, making the implementability problem for HMSCs undecidable in general. The work in this thesis is the first to formally build a bridge between the world of MSTs and HMSCs. To start, we present a generalised projection operator for sender-driven choice. This allows a sender to send to different receivers when branching, which is crucial to handle common communication patterns from distributed computing. Despite this first step, we also show that the classical MST projection approach is inherently incomplete. We present the first formal encoding from global types to HMSCs. With this, we prove decidability of the implementability problem for global types with sender-driven choice. Furthermore, we develop the first direct and complete projection operator for global types with sender-driven choice, using automata-theoretic techniques, and show its effectiveness with a prototype implementation. We are the first to provide an upper bound for the implementability problem for global types with sender-driven (or directed) choice and show it to be in PSPACE. We also provide a session type system that uses the results from our projection operator. Last, we introduce protocol state machines (PSMs) – an automata-based protocol specification formalism – that subsume both global types from MSTs and HMSCs with regard to expressivity. We use transformations on PSMs to show that many of the syntactic restrictions of global types are not restrictive in terms of protocol expressivity. We prove that the implementability problem for PSMs with mixed choice, which requires no dedicated sender for a branch but solely all labels to be distinct, is undecidable in general. With our results on expressivity, this answers an open question: the implementability problem for mixed-choice global types is undecidable in general.
To increase situational awareness of the crane operator, the aim of this thesis is to develop a vision-based deep learning object detection from crane load-view using an adaptive perception in the construction area. Conventional worker detection methods are based on simple shape or color features from the worker's appearances. Nonetheless, these methods can fail to recognize the workers who do not wear the protective gears. To find out an image representation of the object from the top view manually or handcrafted feature is crucial. We, therefore, employed deep learning methods to automatically learn those features.
To yield optimal results, deep learning methods require mass amount of data.
Due to the data deficit especially in the construction domain, we developed the photorealistic world to create the data in addition to our samples collected from the real construction area. The simulated platform does not benefit only from diverse data types, but also concurrent research development which speeds up the pipeline at a low cost.
Our research findings indicate that the combination of synthetic and real training samples improved the state-of-the-art detector. In line with previous studies to bridge the gap between synthetic and real data, the results of preprocessed synthetic images are substantially better than using the raw data by approximately 10%.
Finding the right deep learning model for load-view detection is challenging.
By investigating our training data, it becomes evident that the majority of bounding box sizes are very small with a complex background.
In addition, we gave the priority to speed over accuracy based on the construction safety criteria. Finally, RetinaNet is chosen out of the three primary object detection models.
Nevertheless, the data-driven detection algorithm can fail to handle scale invariance, especially for detectors whose input size is changed in an extremely wide range.
The adaptive zoom feature can enhance the quality of the worker detection.
To avoid further data gathering and extensive retraining, the proposed automatic zoom method of the load-view crane camera supports the deep learning algorithm, specifically in the high scale variant problem. The finite state machine is employed for control strategies to adapt the zoom level to cope not only with inconsistent detection but also abrupt camera movement during lifting operation. Consequently, the detector is able to detect a small size object by smooth continuous zoom control without additional training.
The adaptive zoom control not only enhances the performance of the top-view object detection but also reduces the interaction of the crane operator with camera system, reducing the risk of fatality during load lifting operation.
In recent years, there has been a growing need for accurate 3D scene reconstruction. Recent developments in the automotive industry have led to the increased use of ADAS where 3D reconstruction techniques are used, for example, as part of a collision detection system. For such applications, scene geometry reconstruction is usually performed in the form of depth estimation, where distances to scene objects are obtained.
In general, depth estimation systems can be divided into active and passive. Both systems have their advantages and disadvantages, but passive systems are usually cheaper to produce and easier to assemble and integrate than active systems. Passive systems can be stereo- or multiple-view based. Up to a certain limit, increasing the number of views in multi-view systems usually results in improved depth estimation accuracy.
One potential problem for ensuring the reliability of multi-view systems is the need to accurately estimate the orientation of their optical sensors. One way to ensure sensor placement for multi-view systems is to rigidly fix the sensors at the manufacturing stage. Unlike arbitrary sensor placement, using of a simplified and known sensor placement geometry further simplifies the depth estimation.
We meet with the concept of light field, which parameterizes all visible light passing through all viewpoints by their intersection with angular and spatial planes. When applied to computer vision, this gives us a 2D set of 2D images, where the physical distances between each image are fixed and proportional to each other.
Existing light field depth estimation methods provide good accuracy, which is suitable for industrial applications. However, the main problems of these methods are related to their running time and resource requirements. Most of the algorithms presented in the literature are typically sharpened for accuracy, can only be run on high-performance machines and often require a significant amount of time to process and obtain results.
Real-world applications often have running time requirements. Also, often there is a power-consumption limitation. In this dissertation, we investigate the problem of building a depth estimation system with an light field camera that satisfies the operating time and power consumption constraints without significant loss of estimation accuracy.
First, an algorithm for calibrating light field cameras is proposed, together with an algorithm for automatic calibration refinement, that works on arbitrary captured scenes. An algorithm for classical geometric depth estimation using light field cameras is proposed. Ways to optimize the algorithm for real-time use without significant loss of accuracy are presented. Finally, the ways how the presented depth estimation methods can be extended using modern deep learning paradigms under the two previously mentioned constraints are shown.
Knowledge workers face an ever increasing flood of information in their daily work. They live in a “multi-tasking craziness”, involving activities like creating, finding, processing, assessing or organizing information while constantly switching from one context to another, each being associated with different tasks, documents, mails, etc. Hence, their personal information sphere consisting of file, mail and bookmark folders as well as their content, calendar entries, etc. is cluttered with information that has become irrelevant. Finding important information thus gets harder and much of previously gained knowledge is practically lost.
This thesis explores new ways of solving this problem by investigating the potential of self-(re)organizing and especially forgetting-enabled personal knowledge assistants in the given scenario. It utilizes so-called Managed Forgetting, which is an escalating set of measures to overcome the binary keep-or-delete paradigm, ranging from temporal hiding, to condensation, to adaptive reorganization, synchronization, archiving and deletion. Managed Forgetting is combined with two other major ideas: First, it uses the Semantic Desktop as an ecosystem, which brings Semantic Web and thus knowledge graph technologies to a user’s desktop, making it possible to capture and represent major parts of a user’s personal mental model in a machine-understandable way and exploit it in many different applications. Second, the system uses explicated context information – so-called Context Spaces: context is seen as an explicit interaction element users can work with (i.e. a “tangible” object similar to a folder) and in (immersion). The thesis is structured according to the basic interaction cycle with such a system, ranging from evidence collection to information extraction and context elicitation, followed by information value assessment and the actual support measures consisting of self-(re)organization decisions (back-end) and user interface updates (front-end). The system’s data foundation are personal or group knowledge graphs as well as native data. This work makes contributions to all of these aspects, whereas several of them have been investigated and developed in interdisciplinary research with cognitive scientists. On a more general level, searching and trust in such highly autonomous assistants have also been investigated.
In summary, a self-(re)organizing and especially forgetting-enabled support system for information management and knowledge work has been realized. Its different features vary in maturity: the most mature ones are already in practical use (also in industry), while the latest are just well elaborated (position papers) or rough ideas. Different evaluation strategies have been applied ranging from mere data-driven experiments to various user studies. Some of them were rather short-term with controlled laboratory conditions, others less controlled but spanning several months. Different benefits of working with such a system could be quantified, e.g. cognitive offloading effects and reduced task switching/resumption time. Other benefits were gathered qualitatively, e.g. tidiness of the information sphere and its better alignment with the user’s mental model. The presented approach has been shown to hold a lot of potential. In some aspects, however, only first steps have been taken towards tapping it, e.g. several support measures can be further refined and automation further increased.
This thesis focuses on the operation of reliability-constrained routes in wireless ad-hoc networks. A complete communication protocol that is capable of guaranteeing a statistical minimum reliability level would have to support several functionalities: first, routes that are capable of supporting the specified Quality of Service requirement have to be discovered. During operation of discovered routes, the current Quality of Service level has to be monitored continuously. Whenever significant deviations are detected and the required level of Quality of Service is endangered, route maintenance has to ensure continuous operation. All four functionalities, route discovery, route operation, route maintenance and collection and distribution of network status information, will be addressed in this thesis.
In the first part of the thesis, we propose a new approach for Quality-of- Service routing in wireless ad-hoc networks called rmin-routing, with the provision of statistical minimum route reliability as main route selection criterion. To achieve specified minimum route reliabilities, we improve the reliability of individual links by well-directed retransmissions, to be applied during the operation of routes. To select among a set of candidate routes, we define and apply route quality criteria concerning network load.
High-quality information about the network status is essential for the discovery and operation of routes and clusters in wireless ad-hoc networks. This requires permanent observation and assessment of nodes, links, and link metrics, and the exchange of gathered status data. In the second part of the thesis, we present cTEx, a configurable topology explorer for wireless ad-hoc networks that efficiently detects and exchanges high-quality network status information during operation.
In the third part, we propose a decentralized algorithm for the discovery and operation of reliability-constrained routes in wireless ad-hoc networks called dRmin-routing. The algorithm uses locally available network status information about network topology and link properties that is collected proactively in order to discover a preliminary route candidate. This is followed by a distributed, reactive search along this preselected route to remove imprecisions of the locally recorded network status before making a final route selection. During route operation, dRmin-routing monitors routes and performs different kinds of route repair actions to maintain route reliability in order to overcome varying link reliabilities.
Modeling and Simulation of Internet of Things Infrastructures for Cyber-Physical Energy Systems
(2024)
This dissertation presents a novel approach to the model-based development and simulation-based validation of Internet of Things (IoT) infrastructures within the context of Cyber-Physical Energy Systems (CPES). CPES represents an evolution in energy management, seamlessly blending physical and cyber components for efficient, secure, and dependable energy distribution. However, the intricate interplay of these components demands innovative modeling and simulation strategies.
The work begins by establishing a robust foundation, exploring essential background elements such as requirements engineering, model-based systems engineering, digitalization approaches, and the intricacies of IoT platforms. It introduces the novel concept of homomorphic encryption, a critical enabler for securing IoT data within CPES.
In the exploration of the state of the art, the dissertation delves into the multifaceted landscape of IoT simulation, emphasizing the significance of versatility, community support, scalability, and synchronization.
The core contribution emerges in the chapter on simulating IoT networks. It introduces a sophisticated framework that encompasses hardware-in-the-loop, software-in-the-loop, and human-in-the-loop simulation. This innovative framework extends the boundaries of conventional simulation, enabling holistic evaluations of IoT systems.
A practical case study on smart energy usage showcases the application of the framework. Detailed SysML models, including requirements, package diagrams, block definition diagrams, internal block diagrams, state machine diagrams, and activity diagrams, are meticulously examined. The performance evaluation encompasses diverse aspects, from hardware and software validation to human interaction.
In conclusion, this dissertation represents a significant leap forward in the integration of IoT infrastructures within CPES. Its contributions extend from a comprehensive understanding of foundational elements to the practical implementation of a holistic simulation framework. This work not only addresses the current challenges but also outlines a path for future research, shaping the landscape of IoT integration within the dynamic realm of CPES. It offers invaluable insights for researchers, engineers, and stakeholders working towards resilient, secure, and energy-efficient infrastructures.
Weak memory consistency models capture the outcomes of concurrent
programs that appear in practice and yet cannot be explained by thread
interleavings. Such outcomes pose two major challenges to formal
methods. First, establishing that a memory model satisfies its
intended properties (e.g., supports a certain compilation scheme) is
extremely error-prone: most proposed language models were initially
broken and required multiple iterations to achieve soundness. Second,
weak memory models make verification of concurrent programs much
harder, as a result of which there are no scalable verification
techniques beyond a few that target very simple models.
This thesis presents solutions to both of these problems.
First, it shows that the relevant metatheory of weak memory
models can be effectively decided (sparing years of manual proof
efforts), and presents Kater, a tool that can answer metatheoretic
queries in a matter of seconds. Second, it presents GenMC, the first
(and only) scalable stateless model checker that is parametric in the
choice of the memory model, often improving the prior state of the art
by orders of magnitude.
In one-dimensional (1-D) Ultrasound (US) measurements, signals are
acquired that form the basis of more sophisticated two-dimensional (2-D) or
three-dimensional (3-D) US imaging. These 1-D signals contain a lot of raw
information about the US wave propagation and interaction with the
medium that is only processed in parts during image generation. While
image representations are easy to interpret for humans, the analysis of US
wave signals is hard to perform without applying algorithms to extract
desired features.
This work investigates reliable and fast 1-D US signal classifications to
distinguish between different stages or states in biomedical US scenarios and
shows how the new field of Machine Learning (ML) on raw US wave data
provides advantages and different applications. To achieve good results, the
input signals are treated as time series, which requires the deployment of
comparatively complex Time Series Classification (TSC) algorithms.
The literature shows that a lot of research efforts have previously only
tackled the classification and segmentation of US Brightness mode (B-Mode)
images, while neglecting approaches to classify 1-D signals to a large extent.
This research contributes by developing, deploying and evaluating
classification approaches for three distinct biomedical US classification tasks
and finds that respective signal classifications for different scenarios are
possible with varying degrees of accuracies. It entails the comparison of
several combinations of data types (e.g. temporal, spectral and statistical
features or raw signals), ML models and pre-processing steps to provide a
strong foundation for robust, binary classifications of 1-D US signals for
scenarios based on low-cost wearable, mobile and stationary devices. This
research addresses scientific questions not answered before by informing on
detailed descriptions of beneficial domain specific knowledge (domain specific
knowledge (DSK)), achieved accuracies and times needed for training and
evaluation of the examined ML models.
The resulting ML pipelines includes solutions based on data acquired from
custom experimental setups or clinical trials. Possible real-world applications
might include muscle contraction trackers, muscle fatigue detectors,
epiphyseal radius bone closure detectors or devices providing information
about advanced liver disease stages.
Automated machine-assisted
classifications requiring as little DSK as possible from the end user enable
application scenarios ranging from fitness or rehabilitation trackers as
consumer devices to solutions providing diagnostic support without requiring
extensive knowledge from professional medical practitioners. For example,
decision support systems for bone age assessments in clinical use or liver
health assessment systems for gastroenterologists.
This work shows that reliable, robust and fast classifications based on 1-D
US signals are possible with high degrees of accuracies depending on the
examined scenario with achieved F 1 -scores ranging from ≈ 70% to ≈ 87%.
These results prove that real-life applications for recreational purposes are
already possible and that critical applications for clinical use are highly likely
to be achieved once the presented approaches are further optimized in the future.
The field of 3D reconstruction is one of the most important areas in computer
vision. It is not only of theoretical importance, but it is also increasingly
used in practical applications, be it in reverse engineering, quality control or
robotics. In practical applications, where high precision reconstructions are
required for a large variety of different objects, structured light reconstruction
is often the method of choice. It allows to achieve accurate and dense
point correspondences over the entire scene, regardless of object texture or
features. Techniques that project phase-shifted sinusoidals are widely used
because, based on the harmonic addition theorem, they theoretically allow
surface encoding in full camera resolution invariant to the object’s coloring.
In this thesis, a fully-automatic reconstruction pipeline based on the sinusoidal
structured light technique is presented. From the projection of the
fringe patterns for encoding the object’s surface, the robust matching of the
point correspondences in sub-pixel accuracy, the auto-calibration of the setup
including the active device, up to the fully-automatic alignment of the partial
reconstructions, all steps will be described and examined in detail. During
that, improvements will be achieved in the area of matching, obtaining highly
accurate and topologically consistent correspondences in sub-pixel precision
between all the devices used. Furthermore, the auto-calibration from point
correspondences, based on the epipolar geometry of the structured light system
is improved. Weaknesses of previous methods in the extraction of focal
lengths from the fundamental matrices are discovered and addressed. The partial
point clouds, reconstructed from the auto-calibrated devices, are finally
pre-aligned using a neural network approach, based on light-resistant optical
flow estimation and subsequently refined using a global approach.
The weaknesses of the structured light method itself will also be addressed
and partially fixed during the course of this work. Since it is an active reconstruction method, certain surface properties can affect the quality of the
reconstruction. It will be shown how these problems can be eliminated or at
least be reduced using an iterative approach that combines fringe patterns with
an inverse texture. Another weakness of the method is its time-consuming acquisition procedure. Typically, a large number of horizontal and vertical fringe
patterns are projected onto the scene to achieve high-precision encoding despite
the limited dynamic range and resolution of the projector. Therefore, a
method will be presented which allows to combine the horizontal and vertical
patterns for a simultaneous two dimensional surface encoding.
During our daily lives, we are confronted with vast amounts of data, the processing of which can dramatically influence our lives, both positively and negatively. The enormous amount of data (images, texts, tables, and time series), its variety and possible applications are not always obvious. Due to advancements in the internet of things (IoT), there exist billions of sensors that produce time series which can be found everywhere, whether in medicine, the financial sector or the agricultural economy. This incredible amount of time series data has many hidden features which are useful for industry as well as for daily use, e.g. improving the cancer prediction can save real human lives. Recently, several deep learning methods have been proposed for analyzing this time series data. However, due to their black box nature, their applicability is limited in critical sectors like medicine, finance, and communication. In addition, it is now a compulsion as per artificial intelligence (AI) Act and per General Data Protection Regulation (GDPR) to protect any sensitive data and provide explanations in safety-critical domains. To enable use of DNNs in a broader domain scope, this thesis presents a framework for privacy-preserved and interpretable time series analysis. TimeFrame consists of four main components, namely, post-hoc interpretability, intrinsic interpretability, direct privacy, and indirect privacy. Interpretability is indispensable to avoid damaging people or the infrastructure. In the past years, the development mostly focused on image data, which prevented the full potential of DNNs in time series processing from being exploited. To overcome this limitation, TimeFrame introduces five (Time to Focus, TSViz, TimeREISE, TSInsight, Data Lens) novel post-hoc and two (PatchX, P2ExNet) novel intrinsic interpretability components. TimeFrame addresses multiple perspectives such as attribution, compression, visualization, influence, prototyping, and hierarchical splitting. Compared to existing methods, the components show better explanations, robustness, and scalability. Another crucial factor is the privacy when dealing with sensitive data and deep learning. In this context, TimeFrame introduces two (PPML, PPML x XAI) components for direct and one (From Private to Public) component for indirect privacy. These components benchmark privacy approaches, their effect on interpretability, and the synthetic generation of data to overcome privacy concerns. TimeFrame offers a large set of interpretability and privacy components that can be combined and consider numerous different aspects. Furthermore, the novel approaches have shown to consistently outperform twenty existing state-of-the-art methods across up to 20 different datasets. To guarantee the fairness, various metrics were used including performance change, Sensitivity, Infidelity, Continuity, runtime, model dependency, compression rate, and others. This broad set of metrics makes it possible to provide guidelines for a more appropriate use of existing state-of-the-art approaches as well as the novel components included in TimeFrame.
Highly Automated Driving (HAD) vehicles represent complex and safety critical systems. They are deployed in an open context i.e., an intricate environment which undergoes continual changes. The complexity of these systems and insufficiencies in sensing and understanding the open context may result in unsafe and uncertain behaviour. The safety critical nature of the HAD vehicles requires modelling of root causes for unsafe behaviour and their mitigation to argue sufficient reduction of residual risk.
Standardization activities such as ISO 21448 provide guidelines on the Safety Of The Intended Functionality (SOTIF) and focus on the analysis of performance limitations under the influence of triggering conditions that can lead to hazardous behaviour. SOTIF references traditional safety analyses methods e.g., Failure Mode and Effect Analysis (FMEA) and Fault Tree Analysis (FTA) to perform safety analysis. These analyses methods are based on certain assumptions e.g., single point failure in FMEA and independence of basic events in FTA. Moreover, these analyses are generally based on expert knowledge i.e., data-based models or hybrid approaches (expert and data) are seldom practised. The resulting safety model is fixed i.e., it is generally seen as a one-time artefact. Open context environment may contain triggering conditions which may not be evident to the expert. Open context also evolves over time and new phenomena may emerge.
This thesis explores the applicability of the traditional safety analyses techniques to provide safety models for HAD vehicles operating in the open context, under the light of modelling assumptions taken by traditional safety analyses techniques. Moreover, incorporating uncertainties into safety analyses models is also explored. An explicit distinction between the inherent uncertainty of a probabilistic event (aleatory) and uncertainty due to lack of knowledge (epistemic) is made to formalize models to perform SOTIF analysis. A further distinction is made for conditions of complete ignorance and termed as ontological uncertainty. The distinction is important as for HAD vehicles operating in open context the ontological uncertainty can never be completely disregarded.
This thesis proposes a novel framework of SOTIF to model, estimate and dis cover triggering conditions relevant to performance limitations. The framework provides the ability to model uncertainties while also providing a hybrid approach i.e., supporting inclusion of expert knowledge as well as data driven engineering processes. Two representative algorithms are provided to support the framework. Bayesian Network (BN) and p-value hypothesis testing are utilised in this regard. The framework is implemented on a real-world case study in which LIDARs based perception systems are used as vehicle detection system.
This doctoral dissertation is comprised of nine published articles covering different
methods for ‘Fast, Robust Rigid and Non-Rigid Registration for Globally Consistent
3D Scene and Shape Reconstruction’. Overall the contributing articles are separated
and discussed in three stages – The first part of the thesis i.e., chapter 2 explains
three novel method classes of rigid point set registration namely Gravitational Approach (GA), Fast Gravitational Approach (FGA), and RPSRNet. GA was introduced as the first physics-based rigid point set registration. It includes elegant modeling of rigid by dynamics using Newtonian mechanics. The method proposed many new avenues for other types of pattern matching tasks thank point set registration. Next, FGA method, published 4 years after GA presented as an extension that breaks the algorithmic complexity of GA from O(M N ) to O(M log N ) using Barnes-Hut tree representation of point cloud. It also eliminates the requirement of heuristic optimization parameter settings by GA, and achieve state-of-the-art alignment accuracy on LiDAR odometry. Finally, RPSRNet presents deep learning version of FGA, with custom convolution layers for hierarchical point feature embedding. RPSRNet is robust and the fastest among SoA methods for LiDAR data registration. The second part, i.e., chapter 3, of the thesis introduces NRGA as the fist physics-based non-rigid point set
registration method which is computationally slow but robust against noisy and partial inputs. NRGA preserves structural consistency as it coherently regularize motion of deformable vertices. For articulated hand shape reconstruction, a tailored version of NRGA -- Articulated-NRGA -- is effective to refine final hand shape. Collision and penetration avoidance between source and target surfaces are tackled by constrained optimization in NRGA. This setting has improved hand and object interaction reconstruction. Next contribution FoldMatch method remodels the shape deformation by introducing wrinkle vector field (WVF) for capturing complex clothing and garment details while fitting body models onto 3D Scans. Quantitative evaluation of FoldMatch and NRGA shows their effectiveness in geometrically consistent surface modeling and reconstruction tasks. Finally, the third part of the thesis explains globally consistent outdoor scene reconstruciton, odometry estimation, and uncertainty guided pose-graph optimization in a novel LiDAR-based localization and map building method, called Deep Evidential LiDAR Odometry (DELO). This is the first Odometry method to use predictive uncertainty modeling for sensor pose prediction network.
From industrial fault detection to medical image analysis or financial fraud prevention: Anomaly detection—the task of identifying data points that show significant deviations from the majority of data—is critical in industrial and technological applications. For efficient and effective anomaly detection, a rich set of semantic features are required to be automatically extracted from the complex data. For example, many recent advances in image anomaly detection are based on self-supervised learning, which learns rich features from a large amount of unlabeled complex image data by exploiting data augmentations. For image data, predefined transformations such as rotations are used to generate varying views of the data. Unfortunately, for data other than images, such as time series, tabular data, graphs, or text, it is unclear what are suitable transformations. This becomes an obstacle to successful self-supervised anomaly detection on other data types.
This thesis proposes Neural Transformation Learning, a self-supervised anomaly detection method that is applicable to general data types. In contrast to previous methods relying on hand-crafted transformations, neural transformation learning learns the transformations from data and uses them for detection. The key ingredient is a novel objective that encourages learning diverse transformations while preserving the relevant semantic content of the data. We prove theoretically and empirically that it is more suited than existing objectives for transformation learning.
We also introduce the extensions of neural transformation learning for anomaly detection within time series and graph-level anomaly detection. The extensions combine transformation learning and other learning paradigms to incorporate vital prior knowledge about time series and graph data. Moreover, we propose a general training strategy for deep anomaly detection with contaminated data. The idea is to infer the unlabeled anomalies and utilize them for updating parameters alternatively. In setups where expert feedback is available, we present a diverse querying strategy based on the seeding algorithm of K-means++ for active anomaly detection.
Our extensive experiments and analysis demonstrate that neural transformation learning achieves remarkable and robust anomaly detection performance on various data types. Finally, we outline specific paths for future research.
Semi-structured data is a common data format in many domains.
It is characterized by a hierarchical structure and a schema that is not fixed.
Efficient and scalable processing of this data is therefore challenging, as many existing indexing and processing techniques are not well-suited for this data format.
This dissertation presents a novel approach to processing large JSON datasets.
We describe a new data processor, JODA, that is designed to process semi-structured data by using all available computing resources and state-of-the-art techniques.
Using a custom query language and a vertically-scaling pipeline query execution engine, JODA can process large datasets with high throughput.
We optimize JODA by using a novel optimization for iterative query workloads called delta trees, which succinctly represent the changes between two documents.
This allows us to process iterative and exploratory queries efficiently.
We improve the filtering performance of JODA by implementing a holistic adaptive indexing approach that creates and improves structural and content indices on the fly, depending on the query load.
No prior knowledge about the data is required, and the indices are automatically improved over time.
JODA is also modularized and can be extended with new user-defined predicates, functions, indices, import, and export functionalities.
These modules can be written in an external programming language and integrated into the query execution pipeline at runtime.
To evaluate this system against competitors, we introduce a benchmark generator, coined BETZE, which aims to simulate data scientists exploring unknown JSON datasets.
The generator can be tweaked to generate query workload with different characteristics, or predefined presets can be used to quickly generate a benchmark.
We see that JODA outperforms competitors in most tasks over a wide range of datasets and use-cases.
3D joint angles based human pose is needed for applications like activity recognition, musculoskeletal health, sports biomechanics and ergonomics. The microelectromechanical systems (MEMS) based magnetic-inertial measurement units (MIMUs) can estimate 3D orientation. Due to small size, MIMUs can be attached to the body as wearable sensors for obtaining full 3D human pose and this system is termed as inertial motion capture (i-Mocap). But the MIMUs suffer from sensor errors and disturbances, due to which orientation estimated from individual MIMUs can be erroneous. Accurate sensor calibration is essential and subsequently alignment of these sensors to body segments must also be precisely known, which is called sensor-to-segment calibration. Sensor fusion is employed to address the disturbances and noise in MIMUs. Many state-of-art inertial motion capture approaches ignore the magnetometer and only use IMUs to reduce the error arising from inhomogeneous magnetic field. These algorithms rely on kinematic constraints and assumptions regarding joints and are based on IMUs located on the adjacent body segments. The full body coverage requires 13-17 such units and can be quite obtrusive. The setting up and calibration of so many wearable sensors also take time.
This thesis focuses on 3D human pose estimation from a reduced number of MIMUs and deals with this problem systematically. First we propose an accurate simultaneous calibration of multiple MIMUs, which also learns the uncertainty of individual sensors. We then describe a novel sensor fusion algorithm for robust orientation estimation from an MIMU and for updating sensors calibration online. The residual errors in both sensor calibration and fusion can result in drift error in the joint angles. Therefore, we present anatomical (sensor-to-segment) calibration in which an orientation offset correction term is updated and used for online correction of residual drift in individual joint angles. Subsequently we demonstrate that 3D human joint angle constraints can be learned using a data-driven approach in a high dimensional latent space. Owing to temporal and joint angle constraints, it is possible to use only a reduced set of sensors (as opposed to one sensor per segment) and still obtain 3D human pose. But the spatial and temporal prior learning from data is often limited due to finite set of movement patterns in most datasets. This introduces uncertainty while estimating 3D human pose from sparse MIMU sensors. We propose a magnetometer robust orientation parameterization and a data-driven deep learning framework to predict 3D human pose with associated uncertainty from sparse MIMUs. The model is evaluated on real MIMU data and we show that the uncertainty predicted by the trained model is well-correlated with actual error and ambiguity.
Though Computer Aided Design (CAD) and Simulation software are mature, well established, and in wide professional use, modern design and prototyping pipelines are challenging the limits of these tools. Advances in 3D printing have brought manufacturing capability to the general public. Moreover, advancements in Machine Learning and sensor technology are enabling enthusiasts and small companies to develop their own autonomous vehicles and machines. This means that many more users are designing (or customizing) 3D objects in CAD, and many are testing machine autonomy in Simulation. Though Graphical User Interfaces (GUIs) are the de-facto standard for these tools, we find that these interfaces are not robust and flexible. For example, designs made using GUI often break when customized, and setting up large simulations can be quite tedious in GUI. Though programmatic interfaces do not suffer from these limitations, they are generally quite difficult to use, and often do not provide appropriate abstractions and language constructs.
In this Thesis, we present our work on bridging the ease of use of GUI with the robustness and flexibility of programming. For CAD, we propose an interactive framework that automatically synthesizes robust programs from GUI-based design operations. Additionally, we apply program analysis to ensure customizations do not lead to invalid objects. Finally, for simulation, we propose a novel programmatic framework that simplifies building of complex test environments, and a test generation mechanism that guarantees good coverage over test parameters. Our contributions help bring some of the advantages of programming to traditionally GUI-dominant workflows. Through novel programmatic interfaces, and without sacrificing ease of use, we show that the design and customization of 3D objects can be made more robust, and that the creation of parameterized simulations can be simplified.
Faces deliver invaluable information about people. Machine-based perception can be of a great benefit in extracting that underlying information in face images if the problem is properly modeled. Classical image processing algorithms may fail to handle the diverse data available today due to several challenges related to varying capturing locations, and conditions. Advanced machine learning methods and algorithms are now highly beneficial due to the rapid development of powerful hardware, enabling feasible advanced solutions based on data learning and summarization into powerful models. In this thesis, novel solutions are provided to the problems of head orientation estimation and gender prediction. Initially, classical machine learning algorithms were used to address head orientation estimation but were limited by their inability to handle large datasets and poor generalization. To overcome these challenges, a new highly accurate head pose dataset was acquired to tackle the identified problems. Novel trained deep neural networks have been exploited, that use the acquired data and provide novel architectures. The information about head pose is then represented in the network weights, thus, allowing predicting the head orientation angles given a new unseen face. The acquired dataset, named AutoPOSE opens the door for further studies in the field of computer vision and especially, face analysis. The problem of gender prediction has also been explored, but unlike humans who can easily identify gender from a face, computers face difficulties due to facial similarities. Therefore, hand-crafted features are not effective for generalization. To address this, a new deep learning method was developed and evaluated on multiple public datasets, with identified challenges in both still images and videos addressed. Finally, the effect of facial appearance changes due to head orientation variation has been investigated on gender prediction accuracy. A novel orientation-guided feature maps recalibration method is presented, that significantly increased the accuracy of gender prediction.
In conclusion, two problems have been addressed in this thesis, independently and joined together. Existing methods have been enhanced with intelligent pre-processing methods and new approaches have been introduced to tackle existing challenges, that arise from pose, illumination, and occlusion variations. The proposed methods have been extensively evaluated, showing that head orientation and gender prediction can be estimated with high accuracy using machine learning-based methods. Also, the evaluations showed that the use of head orientation information consistently improved the gender prediction accuracy. Scientific contributions have been presented, and the new acquired highly accurate dataset motivates the research community to push the state-of-the-art forward.
Undocumented enterprise data can easily pile up in companies in form of datasets and personal information. In absence of a data management strategy, such data becomes rather messy and may not fit for its intended use. Since there is often no documentation available, only a limited number of domain experts are aware of its contents. Therefore, for companies it becomes increasingly difficult to use such data to its full potential. To provide a solution, this PhD thesis investigates the construction of enterprise and personal knowledge graphs by semantically enriching messy data with meaning using semantic technologies. Since real world entities and their interrelations are organized in a graph, knowledge graphs serve as a semantic bridge between domain conceptualization and raw data. Spreadsheets are a prominent example of such enterprise data, since they are widely used by knowledge workers in the industrial sector. Two distinct approaches are investigated to construct knowledge graphs from them: a global extraction & annotation method and a local mapping technique. The latter is further complemented with a predictor of mapping rules on messy data. Different human-in-the-loop strategies are considered to include experts depending on their user group. Since non-technical users usually lack understanding of semantic technologies, they need appropriate tools to be able to give feedback. In case of developers, approaches are proposed to close the technology gap between industry and Semantic Web related concepts. Semantic Web practitioners participate with ontology modeling and linked data applications. Enterprise and personal data is typically confidential which is why it cannot be shared with a research community to discuss its challenges. However, for evaluation and reproducibility reasons publicly available datasets are mandatory. The thesis proposes ways to generate synthetic datasets with the goal to be as authentic as possible. Besides that, for internal evaluations a crawler of personal data on desktops is implemented. There are further contributions related to this thesis in diverse domains. One is about the motivation to support users in their daily work using personal knowledge assistants. Others are the agricultural field and the data science domain which also benefit from knowledge graph approaches. In conclusion, this PhD thesis contributes to the construction of knowledge graphs from especially messy enterprise data, while users from different groups take part in this process in various ways.
This thesis focuses on novel methods to establish the utility of wearable devices along with machine learning and pattern recognition methods for formal education and address the open research questions posed by existing methods. Firstly, state-of-the-art methods are proposed to analyse the cognitive activities in the learning process, i.e., reading, writing, and their correlation. Furthermore, this thesis presents real-time applications in wearable space as an experimental tool in Physics education, and an air-writing system.
There are two critical components in analysing the reading behaviour, i.e., WHERE a person looks at (gaze analysis) and WHAT a person looks at (content analysis). This thesis proposes novel methods to classify the reading content to address the WHAT AT component. The proposed methods are based on a hybrid approach, which fuses the traditional computer vision methods with deep neural networks. These methods, when evaluated on publicly available datasets, yield state-of-the-art results to define the structure of the document images. Moreover, extensive efforts were made to refine and correct ICDAR2017-POD dataset along with a completely new FFD dataset.
Traditionally, handwriting research focuses on character and number recognition without looking into the type of writing, i.e. text, math, and drawing. This thesis reports multiple contributions for on-line handwriting classification. First, it presents a public dataset for on-line handwriting classification OnTabWriter, collected using iPen and an iPad. In addition, a new feature set is introduced for on-line handwriting classification to establish the benchmark on the proposed dataset to classify handwriting as plain text, mathematical expression, and plot/graph. An ablation study is made to evaluate the performance of the proposed feature set in comparison to existing feature sets. Lastly, this thesis evaluates the importance of context for on-line handwriting classification.
Analysing reading and writing activities individually is not enough to provide insights to identify the student's expertise unless their correlations are analysed. This thesis presents a study where reading data from wearable eye-trackers and writing data from sensor pen are analysed together in correlation to correlate the expertise of the users in Physics education with their actual knowledge. Initial results show a strong correlation between individual's expertise and understanding of the subject.
Augmented reality & virtual applications can play a vital role in making classroom environments more interactive and engaging both for teachers and learners. To validate the hypothesis, different applications are developed and evaluated. First, smart glasses are used as an experimental tool in Physics education to help the learners perform experiments by providing assistance and feedback on head mounted display in understanding acoustics concepts. Second, a real-time application of air-writing with the finger on an imaginary canvas using a single IMU as the FAirWrite system is also presented. FAirWrite system is further equipped with DL methods to classify the air-written characters.
Due to its performance, the field of deep learning has gained a lot of attention, with neural networks succeeding in areas like \( \textit{Computer Vision} \) (CV), \( \textit{Neural Language Processing} \) (NLP), and \( \textit{Reinforcement Learning} \) (RL). However, high accuracy comes at a computational cost as larger networks require longer training time and no longer fit onto a single GPU. To reduce training costs, researchers are looking into the dynamics of different optimizers, in order to find ways to make training more efficient. Resource requirements can be limited by reducing model size during training or designing more efficient models that improve accuracy without increasing network size.
This thesis combines eigenvalue computation and high-dimensional loss surface visualization to study different optimizers and deep neural network models. Eigenvectors of different eigenvalues are computed, and the loss landscape and optimizer trajectory are projected onto the plane spanned by those eigenvectors. A new parallelization method for the stochastic Lanczos method is introduced, resulting in faster computation and thus enabling high-resolution videos of the trajectory and second-order information during neural network training. Additionally, the thesis presents the loss landscape between two minima along with the eigenvalue density spectrum at intermediate points for the first time.
Secondly, this thesis presents a regularization method for \( \textit{Generative Adversarial Networks} \) (GANs) that uses second-order information. The gradient during training is modified by subtracting the eigenvector direction of the biggest eigenvalue, preventing the network from falling into the steepest minima and avoiding mode collapse. The thesis also shows the full eigenvalue density spectra of GANs during training.
Thirdly, this thesis introduces ProxSGD, a proximal algorithm for neural network training that guarantees convergence to a stationary point and unifies multiple popular optimizers. Proximal gradients are used to find a closed-form solution to the problem of training neural networks with smooth and non-smooth regularizations, resulting in better sparsity and more efficient optimization. Experiments show that ProxSGD can find sparser networks while reaching the same accuracy as popular optimizers.
Lastly, this thesis unifies sparsity and \( \textit{neural architecture search} \) (NAS) through the framework of group sparsity. Group sparsity is achieved through \( \ell_{2,1} \)-regularization during training, allowing for filter and operation pruning to reduce model size with minimal sacrifice in accuracy. By grouping multiple operations together, group sparsity can be used for NAS as well. This approach is shown to be more robust while still achieving competitive accuracies compared to state-of-the-art methods.