Security researchers have identified a critical security flaw within the artificial intelligence as a service provider Replicate, potentially allowing threat actors access to proprietary AI models and sensitive information.
According to a report released this week by cloud security company Wiz, exploiting this vulnerability could grant unauthorized users access to AI prompts and results across all clients on the Replicate platform.
The issue stems from AI models being typically packaged in formats that allow for arbitrary code execution. Attackers could weaponize malicious models to execute cross-tenant attacks.
Replicate utilizes an open-source tool called Cog to containerize and package machine learning models for deployment in self-hosted environments.
Wiz demonstrated creating a malicious Cog container and uploading it to Replicate, successfully achieving remote code execution on the service infrastructure with elevated privileges.
“We suspect this pattern of code execution is a vulnerability that companies and organizations face when running AI models from untrusted sources, even those potentially containing malicious code,” said security researchers Shir Tamari and Sagi Tzadik.
The attack technique devised by the researchers leveraged established TCP connections associated with Redis server instances hosted on Google Cloud Platform’s Kubernetes clusters, injecting arbitrary commands.
Moreover, centralized Redis servers used to manage queues for multiple client requests and responses could be exploited to facilitate cross-tenant attacks by tampering with this process to insert malicious tasks that could potentially impact other clients’ model outcomes.
Such malicious manipulations not only threaten the integrity of AI models but also pose significant risks to the accuracy and reliability of AI-driven outputs.
“Attackers could potentially query private AI models of clients, exposing proprietary knowledge or sensitive data involved in model training,” the researchers noted. Additionally, intercepted prompts could expose sensitive data, including personally identifiable information (PII).
The vulnerability was responsibly disclosed in January 2024 and has since been addressed by Replicate. There is currently no evidence indicating exploitation of the vulnerability in the wild to compromise client data.
Wiz also detailed risks, now patched, in platforms like Hugging Face, where vulnerabilities could allow threat actors to escalate privileges and gain cross-tenant access to other clients’ models or even take over continuous integration and deployment (CI/CD) pipelines.
“Malicious models pose significant risks to AI systems, particularly for AI as a service providers, where attackers could exploit these models for cross-tenant attacks,” the researchers concluded.
“The potential impact could be devastating, as attackers might gain access to millions of private AI models and applications stored within AI as a service providers.”
