Recent reports highlight a security vulnerability in Google’s Gemini tool for Workspace that could allow malicious actors to generate email summaries containing deceptive instructions. The exploit involves crafting emails with hidden directives that prompt Gemini to generate misleading content, potentially leading users to phishing sites without any overt links or attachments.
This method of invasion, characterized as a prompt-injection attack, was uncovered by Marco Figueroa, the GenAI Bug Bounty Programs Manager at Mozilla, and detailed in a blog post on 0din. The attacker can conceal harmful commands using invisible text, which remains undetectable in Gmail’s interface, making it easier for the malicious email to bypass filtration systems and reach users’ inboxes.
Once an unsuspecting recipient requests a summary from Gemini, the AI tool can execute the hidden instructions, resulting in misleading security alerts. For instance, it could warn users about supposedly compromised Gmail passwords and present a fraudulent support number, which could easily be mistaken as a legitimate warning given the trust placed in Google’s technology.
Figueroa suggests various strategies for organizations and security teams to combat such threats, including stripping hidden styles from email body text and implementing filters to identify urgent messages with potentially harmful content. In communication with BleepingComputer, a Google spokesperson emphasized the company’s ongoing efforts to enhance its defenses against adversarial exploits. They noted that while no incidents have been confirmed relating to this specific exploit, safeguards are being improved as part of their security measures detailed in a Google blog post.