SCALEX: SCALability EXploration of multi-agent reinforcement learning agents in grid-interactive efficient buildings



Journal Title

Journal ISSN

Volume Title



The transition to renewable energy sources and the decarbonization of buildings bring new challenges to grid-interactive efficient building (GEB) communities such as grid stability issues and energy system integration. The dynamic and stochastic nature of intermittent renewable energy production makes it challenging for conventional building control systems to maximize them. To overcome this, advanced control architecture and energy storage are utilized to achieve energy flexibility. Reinforcement learning (RL) offers potential solutions, but its scalability and computational demands in large-scale settings remain unclear. This thesis examines the scalability of Soft-Actor Critic (SAC) in multi-agent systems, comparing decentralized-independent SACs and centralized SACs using CityLearn, an OpenAI Gym environment. We consider neighborhoods consisting of 2 to 64 single-family residential buildings, each equipped with cooling and heating storage devices, domestic hot water storage devices, electrical storage devices, and solar PV systems. In this work, we delve into the challenges faced by these controllers when scaled in a multi-agent system. Our findings suggest that decentralized-independent controllers outperform the centralized controller with an increasing number of buildings. We also show that the performance on the building level can differ from the aggregated performance.


LCSH Subject Headings