Machine Vision with Programmable Floating-Gate Phototransistor for Color-Mixed Image Recognition

The success of artificial neural networks (ANN) in machine vision techniques drives hardware researchers to explore more efficient computing elements for energy-expensive operations like vector-matrix multiplication (VMM). But the energy consumption and preprocessing time required for capturing the digitalized image are seldom considered. In this work, the InP-based floating-gate photo-field-effective transistors (FG-PFETs) are demonstrated as the promising computing element at the sensor level and enable optical signal sensing and processing simultaneously. Simulated optical neural network (ONN) constructed from the measured performance of FG-PFETs indicates the high image recognition accuracy (>94%) for color-mixed MNIST handwritten digits. And the heterogeneous gate dielectric structure allows the devices to work offline after training. Notably, the back-end CMOS compatible processes were implemented for the device integration, which paved the way for FG-PFETs as competitive candidates for energy-efficient machine vision.