Next-Generation Sensing and Perception Enabled by Large-Area Electronics

Next-generation sensing and perception applications, such as smart agriculture, industrial automation, and surgical rooms of the future, are characterized by complex environments, generating a vast, distributed, and interrelated array of embedded signals across modalities. Such smart environments motivate automated systems that generate robust and sophisticated responses to the complex and dynamic environmental state. This requires a generalized framework for perception, based on the co-design of algorithms and technology platforms for sensor fusion.<br/><br/>This talk starts by discussing such a framework, based on combining remote and embedded sensing. Remote sensing, such as vision, enables scalable data acquisition, across wide and dynamic fields of view, while embedded sensing, such as object-attached tags, enables multi-modal data acquisition with invariant association to embedded processes/objects. We present a class of deep-learning models for sensor fusion, that leverages the scale of remote sensing and the structure introduced by embedded sensing to enhance the robustness and efficiency of perception. The key principle for fusion is aligning spatially-dense remote-sensing feature maps with spatially-sparse embedded-sensing feature maps. This requires indexing a large number of distributed embedded point sensors by spatial location, to create feature maps.<br/><br/>We next discuss how Large-Area Electronics (LAE) provides a platform for spatially-distributed, physically-integrated embedded sensing arrays. In the past, this has been restricted to low-frequency form-fitting multi-modal tactile sensing sheets. We review hybrid systems, combining CMOS and LAE, that leverage in-sensor computing for scalable and efficient sparse-signal sensing from large-scale conformal sensor arrays. The architectures employ simple operations implemented by integrated thin-film-transistor (TFT) circuits to perform sensor data compression for scalable readout from the arrays.<br/><br/>More recently, we show how innovations in TFT technology, devices, and circuits, enable Giga-Hertz LAE systems, opening up wireless functionality. Such systems can achieve large radiative apertures, providing a high-degree of control over radiated signals, in the typical frequency range for low-power embedded radios (i.e., hundreds of MHz to few GHz). At the technology level, we leverage high-mobility zinc-oxide semiconductor for improved TFT transconductance. At the device level, we leverage source-drain self alignment, via backside exposure for photo-lithographic patterning through transparent zinc oxide, and conformal zinc-oxide deposition via atomic layer deposition for thick (low-resistance) bottom-gate electrode, to maximize TFT power-gain frequency (f_MAX). At the circuit level, we leverage high-quality factor LAE inductors for resonant cancelation of TFT parasitic capacitances in the multi-GHz regime, to create radio-frequency switches and controllable oscillators.<br/><br/>Based on these advances, we demonstrate LAE wireless systems having large radiative apertures for radiation-pattern control in wireless sensing. First, we demonstrate a GHz 3-element phased-array transmitter, with directional control realized by injection-locked LAE TFT oscillators. Second, we demonstrate a 2.4 GHz reconfigurable antenna, based on an 11x11 array of patch antennas, interconnected by TFT RF switches to control the surface-current distribution for frequency, polarization, and radiation pattern configuration. Third, we demonstrate a 2.4 GHz passive backscattering beamforming transmitter, leveraging the geometry of a van Atta array for directional frequency-shift-keying signal reflection to an active receiver. Together, these provide a picture for the required elements and technology pathways for realizing large-scale distributed multi-modal sensing and perception systems.