Can We Trust Embodied Agents?
Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems

Ruochen Jiao*¹     Shaoyuan Xie*²     Justin Yue²     Takami Sato²    
Lixu Wang¹     Yixuan Wang¹     Qi Alfred Chen²     Qi Zhu<sup>1    

1</sup>Northwestern University     ²University of California, Irvine    
*Equal contribution

   

Overview

Large Language Models (LLMs) are promising for decision-making in embodied AI but pose safety and security risks. We introduce BALD, a framework for Backdoor Attacks on LLM-based systems, exploring attack surfaces and triggers. We propose three attack mechanisms: word injection, scenario manipulation, and knowledge injection. Our experiments on GPT-3.5, LLaMA2, and PaLM2 in autonomous driving and home robot tasks show high success rates and stealthiness. Our findings highlight critical vulnerabilities and the need for robust defenses in embodied LLM systems.

Experiments

Comming soon.