Artificial Intelligence-Driven Autonomous Agent for Smart Hybrid Energy Storage Systems with Integrated Renewable Sources

Hybrid energy storage systems (ESS) utilizing both green hydrogen and lithium batteries provide sustainable energy storage for microgrid applications. Rapid-deployment microgrids are crucial for energy infrastructure in disaster response, military applications, and off-grid communities. Working with the Shinnecock Indian Nation as a case study, we worked to design an reinforcement learning (RL) based artificial intelligence agent which can optimally allocate generated energy to either hydrogen storage or lithium batteries for the community’s rapidly-built emergency housing units.<br/>The Shinnecock Nation generates power through a 96-panel solar array, with peak wattage providing around 22 kW. Over a three week period in July, we found an average daily generation of 136 kWh per day. During the same time period, we identified that the average consumption was 41 kWh per day. We assumed a battery with nominal energy of 15.5 kWh, nominal power of 4.5 kW, and a hydrogen fuel cell with nominal energy of 223 kWh, nominal power 20 kW. Formulating the microgrid’s objectives and constraints mathematically, our model aimed to minimize load and energy curtail. Before implementation of an energy storage system (ESS), our computational model provides crucial data on whether solar energy generation is sufficient for off-grid capabilities — the assessment showed that an appropriate ESS can entirely meet the energy demands of the homes.<br/>We then implemented a Q-learning RL algorithm to teach an AI agent optimal energy management. The agent’s action space includes storing, discharging, or selling energy to the grid, modeled as a Markov Decision Process. After each action, the solar generation and energy demand were randomly updated, simulating non-stationary energy dynamics. Using an epsilon-greedy strategy, the agent initially explores random actions, then exploits the best actions for long term reward based on a learned Q-table, updated via the Bellman equation. This off-policy approach emphasizes robust policy learning with a goal of better reflecting real-world energy uncertainties.<br/>While the more efficient batteries are better for short-duration energy storage, we found that higher energy density hydrogen fuel cells are more effective in the long-term. A machine-learning trained agent can optimize the power flow between an energy-generation solar cell, lithium-ion phosphate battery, and hydrogen fuel cell, solving a crucial challenge of hybrid energy storage systems. Predictive ability enables an agent to dynamically make intelligent storage decisions — using lithium battery storage to offset the short-term loss in generation on a cloudy day, for example. Therefore, an autonomous system provides the ability for improved efficiency in the combined use of a hydrogen fuel-cell and lithium-ion battery.<br/>Our research supports the goal of continuous operation for the Shinnecock Nation with zero load curtailment and grid import. An AI agent can identify optimal solutions to hybrid energy storage automation decisions, enabling efficient microgrid management through the use of hybrid lithium-ion and hydrogen fuel cell storage with a predictive system for optimal energy balancing. Such an agent makes hybrid energy storage more viable for widespread usage, with the ability to dynamically adjust to complex trends, aligning with global efforts for clean energy storage.<br/><br/>This work acknowledges the United States Department of Naval Research award N00014-23-1-2124 issued by the Office of Naval Research.<br/><br/>References<br/>[1] Arnoldi, D., Matsil, E., & Harris, M. (2024). Solar-to-Hydrogen Microgrid Evaluation for Hunter Homes of Shinnecock Nation. Journal of Undergraduate Chemical Engineering Research, 13(9). https://drive.google.com/file/d/1NNqSUM5MvlixmIl91v7mOS509gyu8TvB/view<br/>[2] Lin, C., Zhang, P., Shamash, Y. A., Lin, Z., & Lu, X. (2024). Resilience-Assuring Hydrogen-Powered Microgrids. IEnergy, 3(2), 77–81. https://doi.org/10.23919/ien.2024.0015