Advanced sensing systems, sophisticated algorithms, and increasing computational resources continuously enhance the advanced driver assistance systems (ADAS). To date, despite that some vehicle based approaches to driver fatigue/drowsiness detection have been realized and deployed, objectively and reliably detecting the fatigue/drowsiness state of driver without compromising driving experience still remains challenging. In general, the choice of input sensorial information is limited in the state-of-the-art work. On the other hand, smart and safe driving, as representative future trends in the automotive industry worldwide, increasingly demands the new dimensional human-vehicle interactions, as well as the associated behavioral and bioinformatical data perception of driver. Thus, the goal of this research work is to investigate the employment of general and custom 3D-CMOS sensing concepts for the driver status monitoring, and to explore the improvement by merging/fusing this information with other salient customized information sources for gaining robustness/reliability. This thesis presents an effective multi-sensor approach with novel features to driver status monitoring and intention prediction aimed at drowsiness detection based on a multi-sensor intelligent assistance system -- DeCaDrive, which is implemented on an integrated soft-computing system with multi-sensing interfaces in a simulated driving environment. Utilizing active illumination, the IR depth camera of the realized system can provide rich facial and body features in 3D in a non-intrusive manner. In addition, steering angle sensor, pulse rate sensor, and embedded impedance spectroscopy sensor are incorporated to aid in the detection/prediction of driver's state and intention. A holistic design methodology for ADAS encompassing both driver- and vehicle-based approaches to driver assistance is discussed in the thesis as well. Multi-sensor data fusion and hierarchical SVM techniques are used in DeCaDrive to facilitate the classification of driver drowsiness levels based on which a warning can be issued in order to prevent possible traffic accidents. The realized DeCaDrive system achieves up to 99.66% classification accuracy on the defined drowsiness levels, and exhibits promising features such as head/eye tracking, blink detection, gaze estimation that can be utilized in human-vehicle interactions. However, the driver's state of "microsleep" can hardly be reflected in the sensor features of the implemented system. General improvements on the sensitivity of sensory components and on the system computation power are required to address this issue. Possible new features and development considerations for DeCaDrive are discussed as well in the thesis aiming to gain market acceptance in the future.

”In contemporary electronics 80% of a chip may perform digital functions but the 20% of analog functions may take 80% of the development time.” [1]. Aggravating this, the demands on analog design is increasing with rapid technology scaling. Most designs have moved away from analog to digital domains, where possible, however, interacting with the environment will always require analog to digital data conversion. Adding to this problem, the number of sensors used in consumer and industry related products are rapidly increasing. Designers of ADCs are dealing with this problem in several ways, the most important is the migration towards digital designs and time domain techniques. Time to Digital Converters (TDC) are becoming increasingly popular for robust signal processing. Biological neurons make use of spikes, which carry spike timing information and will not be affected by the problems related to technology scaling. Neuromorphic ADCs still remain exotic with few implementations in sub-micron technologies Table 2.7. Even among these few designs, the strengths of biological neurons are rarely exploited. From a previous work [2], LUCOS, a high dynamic range image sensor, the efficiency of spike processing has been validated. The ideas from this work can be generalized to make a highly effective sensor signal conditioning system, which carries the promise to be robust to technology scaling. The goal of this work is to create a novel spiking neural ADC as a novel form of a Multi-Sensor Signal Conditioning and Conversion system, which • Will be able to interface with or be a part of a System on Chip with traditional analog or advanced digital components. • Will have a graceful degradation. • Will be robust to noise and jitter related problems. • Will be able to learn and adapt to static errors and dynamic errors. • Will be capable of self-repair, self-monitoring and self-calibration Sensory systems in humans and other animals analyze the environment using several techniques. These techniques have been evolved and perfected to help the animal sur- vive. Different animals specialize in different sense organs, however, the peripheral neural network architectures remain similar among various animal species with few ex- ceptions. While there are many biological sensing techniques present, most popularly used engineering techniques are based on intensity detection, frequency detection, and edge detection. These techniques are used with traditional analog processing (e.g., colorvi sensors using filters), and with biological techniques (e.g. LUCOS chip [2]). The local- ization capability of animals has never been fully utilized. One of the most important capabilities for animals, vertebrates or invertebrates, is the capability for localization. The object of localization can be predator, prey, sources of water, or food. Since these are basic necessities for survival, they evolve much faster due to the survival of the fittest. In fact, localization capabilities, even if the sensors are different, have convergently evolved to have same processing methods (coincidence detection) in their peripheral neurons (for e.g., forked tongue of a snake, antennae of a cockroach, acoustic localization in fishes and mammals). This convergent evolution increases the validity of the technique. In this work, localization concepts based on acoustic localization and tropotaxis are investigated and employed for creation of novel ADCs. Unlike intensity and frequency detection, which are not linear (for e.g. eyes saturate in bright light, loose color perception in low light), localization is inherently linear. This is mainly because the accurate localization of predator or prey can be the difference between life and death for an animal. Figure 1 visually explains the ADC concept proposed in this work. This has two parts. (1) Sensor to Spike(time) Conversion (SSC), (2) Spike(time) to Digital Conversion(SDC). Both of the structures have been designed with models of biological neurons. The combination of these two structures is called SSDC. To efficiently implement the proposed concept, a comparison of several biological neural models is made and two models are shortlisted. Various synapse structures are also studied. From this study, Leaky Integrate and Fire neuron (LIF) is chosen since it fulfills all the requirements of the proposed structure. The analog neuron and synapse designs from Indiveri et. al. [3], [4] were taken, and simulations were conducted using cadence and the behavioral equivalence with biological counterpart was checked. The LIF neuron had features, that were not required for the proposed approach. A simple LIF neuron stripped of these features and was designed to be as fast as allowed by the technology. The SDC was designed with the neural building blocks and the delays were designed using buffer chains. This SDC converts incoming Time Interval Code (TIC) to sparse place coding using coincidence detection. Coincidence detection is a property of spiking neurons, which is a time domain equivalent of a Gaussian Kernel. The SDC is designed to have an online reconfigurable Gaussian kernel width, weight, threshold, and refractory period. The advantage of sparse place codes, which contain rank order coding wasvii Figure 1: ADC as a localization problem (right), Jeffress model of sound localization visualized (left). The values t 1 and t 2 indicate the time taken from the source to s1 and s2 respectively. described in our work [5]. A time based winner take all circuit with memory was created based on a previous work [6] for reading out of sparse place codes asynchronously. The SSC was also initially designed with the same building blocks. Additionally, a differential synapse was designed for better SSC. The sensor element considered wasviii a Wheatstone full bridge AMR sensor AFF755 from Sensitec GmbH. A reconfigurable version of the synapse was also designed for a more generic sensor interface. The first prototype chip SSDCα was designed with 257 modules of coincidence detectors realizing the SDC and the SSC. Since the spike times are the most important information, the spikes can be treated as digital pulses. This provides the capability for digital communication between analog modules. This creates a lot of freedom for use of digital processing between the discussed analog modules. This advantage is fully exploited in the design of SSDCα. Three SSC modules are multiplexed to the SDC. These SSC modules also provide outputs from the chip simultaneously. A rising edge detecting fixed pulse width generation circuit is used to create pulses that are best suited for efficient performance of the SDC. The delay lines are made reconfigurable to increase robustness and modify the span of the SDC. The readout technique used in the first prototype is a relatively slow but safe shift register. It is used to analyze the characteristics of the core work. This will be replaced by faster alternatives discussed in the work. The area of the chip is 8.5 mm 2 . It has a sampling rate from DC to 150 kHz. It has a resolution from 8-bit to 13-bit. It has 28,200 transistors on the chip. It has been designed in 350 nm CMOS technology from ams. The chip has been manufactured and tested with a sampling rate of 10 kHz and a theoretical resolution of 8 bits. However, due to the limitations of our Time-Interval-Generator, we are able to confirm for only 4 bits of resolution. The key novel contributions of this work are • Neuromorphic implementation of AD conversion as a localization problem based on sound localization and tropotaxis concepts found in nature. • Coincidence detection with sparse place coding to enhance resolution. • Graceful degradation without redundant elements, inherent robustness to noise, which helps in scaling of technologies • Amenable to local adaptation and self-x features. Conceptual goals have all been fulfilled, with the exception of adaptation. The feasibility for local adaptation has been shown with promising results and further investigation is required for future work. This thesis work acts as a baseline, paving the way for R&D in a new direction. The chip design has used 350 nm ams hitkit as a vehicle to prove the functionality of the core concept. The concept can be easily ported to present aggressively-scaled-technologies and future technologies.

In current practices of system-on-chip (SoC) design a trend can be observed to integrate more and more low-level software components into the system hardware at different levels of granularity. The implementation of important control functions and communication structures is frequently shifted from the SoC’s hardware into its firmware. As a result, the tight coupling of hardware and software at a low level of granularity raises substantial verification challenges since the conventional practice of verifying hardware and software independently is no longer sufficient. This calls for new methods for verification based on a joint analysis of hardware and software. This thesis proposes hardware-dependent models of low-level software for performing formal verification. The proposed models are conceived to represent the software integrated with its hardware environment according to the current SoC design practices. Two hardware/software integration scenarios are addressed in this thesis, namely, speed-independent communication of the processor with its hardware periphery and cycle-accurate integration of firmware into an SoC module. For speed-independent hardware/software integration an approach for equivalence checking of hardware-dependent software is proposed and an evaluated. For the case of cycle-accurate hardware/software integration, a model for hardware/software co-verification has been developed and experimentally evaluated by applying it to property checking.