Security Enhancement Through Compiler-Assisted Software Diversity With Deep Reinforcement Learning

Security Enhancement Through Compiler-Assisted Software Diversity With Deep Reinforcement Learning

Junchao Wang, Jin Wei, Jianmin Pang, Fan Zhang, Shunbin Li
Copyright: © 2022 |Volume: 14 |Issue: 2 |Pages: 18
ISSN: 1941-6210|EISSN: 1941-6229|EISBN13: 9781668466308|DOI: 10.4018/IJDCF.302878
Cite Article Cite Article

MLA

Wang, Junchao, et al. "Security Enhancement Through Compiler-Assisted Software Diversity With Deep Reinforcement Learning." IJDCF vol.14, no.2 2022: pp.1-18. http://doi.org/10.4018/IJDCF.302878

APA

Wang, J., Wei, J., Pang, J., Zhang, F., & Li, S. (2022). Security Enhancement Through Compiler-Assisted Software Diversity With Deep Reinforcement Learning. International Journal of Digital Crime and Forensics (IJDCF), 14(2), 1-18. http://doi.org/10.4018/IJDCF.302878

Chicago

Wang, Junchao, et al. "Security Enhancement Through Compiler-Assisted Software Diversity With Deep Reinforcement Learning," International Journal of Digital Crime and Forensics (IJDCF) 14, no.2: 1-18. http://doi.org/10.4018/IJDCF.302878

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Traditional software defenses take corresponding actions after the attacks are discovered. The defenders in this situation are comparatively passive because the attackers may try many different ways to find vulnerability and bugs but the software remains static. This leads to the imbalance between offense and defense. Software diversity alleviates the current threats by implementing a heterogeneous software system. The N-Variant eXecution (NVX) systems, effective and applicable runtime diversifying methods, apply multiple variants to imporove software security. Higher diversity can lead to less vulnerabilities that attacks can exploit. However, runtime diversifying methods such as address randomization and reverse stack can only provide limited diversity to the system. Thus, we enhance the diversity of variants with a compiler-assisted approach. We use a Deep Reinforcement Learning-based algorithm to generate variants, ensuring the high diversity of the system. For different numbers of variants, we show the results of the Deep Q Network algorithm under different parameter settings.