DEEP Q-NETWORK-BASED ADAPTIVE EDGE INTELLIGENCE FOR ENERGY-EFFICIENT AIR QUALITY MONITORING IN IOT NETWORKS

Donatien KADIMA MUAMBA

doi:10.56327/ijiscs.v10i2.1905

DEEP Q-NETWORK-BASED ADAPTIVE EDGE INTELLIGENCE FOR ENERGY-EFFICIENT AIR QUALITY MONITORING IN IOT NETWORKS

Donatien KADIMA MUAMBA⁽¹⁾,

(1) Faculté des Sciences Informatiques/ Université Révérend KIM Ecole de Télécommunication et Télédétection Spatiale/ Université Pédagogique Nationale

Corresponding Author

Abstract

The rapid deployment of Internet of Things (IoT) networks enables environmental monitoring, but conventional fixed-interval sensing causes excessive energy consumption, communication overhead, and limited adaptability. This study proposes an adaptive edge intelligence framework using Deep Q-Network (DQN) for energy-efficient air quality monitoring. The system integrates environmental sensors with an ESP32-based edge node to measure PM2.5, PM10, CO₂, NOx, SOx, NHx, temperature, humidity, and traffic-related information. A DQN agent dynamically selects sensing and communication actions using nine environmental and contextual state variables. A customized reward function balances air quality conditions, energy consumption, communication costs, and monitoring requirements. Experimental data were collected through an IoT platform and stored in Supabase. The agent was trained using 103 valid observations over 200 episodes and independently evaluated on 114 observations. Training rewards increased from −43.62 to approximately 270 after convergence. Evaluation achieved a total reward of 316.092, an average reward of 2.773 per decision step, and an average energy cost of 0.20 per step. Under stable environmental conditions, the agent consistently selected SLOW_SAMPLE, reducing unnecessary sensing operations. These findings demonstrate the feasibility of DQN-based adaptive monitoring and provide a scalable foundation for energy-efficient, self-optimizing IoT environmental sensing systems under varying environmental conditions.

Keywords

Internet of Things (IoT); Edge Intelligence; Deep Q-Network (DQN); Reinforcement Learning; Air Quality Monitoring; Energy Efficiency; Adaptive Sampling; Smart Environment.

References

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015, doi: 10.1038/nature14236.

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.

H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with Double Q-learning,” in Proc. AAAI Conf. Artificial Intelligence, vol. 30, no. 1, pp. 2094–2100, 2016, doi: 10.1609/aaai.v30i1.10295.

T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” in Proc. Int. Conf. Learning Representations (ICLR), 2016.

Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling network architectures for deep reinforcement learning,” in Proc. Int. Conf. Machine Learning (ICML), 2016.

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.

J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of Things (IoT): A vision, architectural elements, and future directions,” Future Generation Computer Systems, vol. 29, no. 7, pp. 1645–1660, 2013.

W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016, doi: 10.1109/JIOT.2016.2579198.

Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” Proceedings of the IEEE, vol. 107, pp. 1738–1762, 2019, doi: 10.1109/JPROC.2019.2918951.

J. Lin, W. Yu, N. Zhang, X. Yang, H. Zhang, and W. Zhao, “A survey on Internet of Things: Architecture, enabling technologies, security and privacy, and applications,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1125–1142, 2017.

C. Perera, C. H. Liu, and S. Jayawardena, “Context-aware computing in the Internet of Things: A survey on Internet of Things from industrial market perspective,” IEEE Communications Surveys & Tutorials, vol. 16, no. 1, pp. 414–454, 2014.

R. Li, Z. Zhao, Q. Sun, C.-L. I, C. Yang, X. Chen, M. Zhao, and H. Zhang, “Deep reinforcement learning for resource management in network slicing,” IEEE Access, vol. 6, pp. 74429–74441, 2018, doi: 10.1109/ACCESS.2018.2881964.

S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,” IEEE Access, vol. 5, pp. 6757–6779, 2017.

Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.

World Health Organization, WHO Global Air Quality Guidelines: Particulate Matter (PM2.5 and PM10), Ozone, Nitrogen Dioxide, Sulfur Dioxide and Carbon Monoxide. Geneva, Switzerland: World Health Organization, 2021.

A. Kumar, I. P. Singh, and S. P. Singh, “IoT-based air quality monitoring and forecasting systems: A review,” IEEE Sensors Journal, [publication details to be verified according to the final selected source].

A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of Things: A survey on enabling technologies, protocols, and applications,” IEEE Communications Surveys & Tutorials, vol. 17, no. 4, pp. 2347–2376, 2015.

F. Samie, L. Bauer, and J. Henkel, “From cloud down to things: An overview of machine learning in Internet of Things,” IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4921–4934, 2019.

M. Chen, Y. Miao, X. Jia, and K. Hwang, “Intelligent resource management for Internet of Things based on deep reinforcement learning,” IEEE Internet of Things Journal, [publication details to be verified according to the final selected source].

W. Zhang, Y. Wen, K. Guan, D. Kilper, and H. Luo, “Energy-optimal mobile cloud computing under stochastic wireless channel,” IEEE Transactions on Wireless Communications, vol. 12, no. 9, pp. 4569–4581, 2013.

Full Text: PDF