Optical Sensing in High Dimensions

Optical sensors measure the physical properties of incident light, such as the power, polarization state, or spectrum. Recently, the development of integrated sensing solutions using miniaturized devices together with dedicated machine learning algorithms has accelerated rapidly, and optical sensor research has become a highly interdisciplinary field encompassing device and materials engineering, condensed matter physics, and machine learning. Physical properties of light and the corresponding sensor output signals can be viewed as points in two high-dimensional vector spaces, and the sensing process can be understood as a mapping of one vector space to the other. This mapping can be linear or nonlinear, with an artificial neural network being the method of choice in the latter case. Based on this framework, we present examples in which a reconfigurable optical sensor can directly detect spectral and spatial features of incident light, enabled by the reconfigurability of the device and the implementation of machine learning algorithms for information encoding and decoding. Photosensors for image classification, spectral mixture analysis, autoencoding, and compressed sensing will be discussed. In all these devices, the computation is performed at the lowest possible level of the sensor system hierarchy – the physical level of photon detection – and does not require any external processing of the measurement data.