AI infiltrating computers through online images? PCs acting without clicks: The true nature of "malicious pixels" deceiving AI agents

A study by Oxford University has demonstrated that "invisible commands" embedded in desktop wallpapers, social media images, advertisements, PDFs, and more can hijack AI OS agents that operate by taking screenshots of screens, potentially enabling them to perform actions such as web navigation, downloads, and information transmission. These malicious image patches (MIPs) are effective across layouts and models and can withstand resizing and compression. Although no actual damage has been reported yet, Trail of Bits has shown the potential for data extraction in real services using an alternative method where commands are exposed through downscaling processes, making the threat more tangible. Countermeasures include rejecting all on-screen derived instructions, implementing two-factor verification for dangerous APIs, normalizing and fixing dimensions before input, sandboxing, and strengthening through adversarial learning. As the era of agent proliferation advances, it is urgent to update the design principles on the implementation side.