Choosing a provider
ModeLeak: flaws in Google's Vertex AI cloud platform
In November 2024 researchers at Palo Alto Networks showed two flaws in Google Vertex AI: a malicious job or model could reach a project's storage, data tables and other fine-tuned models.
Vertex AI is Google's cloud platform on which companies build, train, fine-tune and run machine learning models. On 12 November 2024 Unit 42, the research team of Palo Alto Networks, published two related flaws in it under the name ModeLeak.
The first concerned so-called custom jobs. These run in the background with the rights of a service account managed by Google, and by default that account had far broader permissions than needed. With a single permission, the right to create a custom job, the researchers ran their own code and could list, read and export the project's storage buckets and BigQuery data tables.
The second needed a “poisoned” model hiding malicious code, which a data scientist might, for example, deploy unsuspectingly from a public repository. By deploying such a model, the researchers moved further into the internal systems that run the models, and could copy the other deployed models in the same project, including fine-tuning adapters for large language models, which can carry a company's proprietary information.
Both attacks had to start from inside the project: they needed a permission there, or someone deploying a malicious model. Unit 42 reported the flaws to Google, which has since fixed them. The researchers recommend strict control over model deployment and keeping development environments separate from production.
What can we learn from it?
Large cloud platforms can have flaws too, such as overly broad default permissions that only researchers notice. Researchers report them and providers fix them, as happened here. Confiella also uses Vertex AI, but it only calls generally available models on an EU regional endpoint; it does not train or deploy its own models there, so it does not use the features these flaws affected. The best protection against a provider's mistakes is to send to the cloud only what the task needs, pseudonymised.
Sources
- Unit 42: ModeLeak: Privilege Escalation to LLM Model Exfiltration in Vertex AI
- eWeek: Google Vertex AI Vulnerability Exposes Customer Data — What You Need to Know
This article is based on the public sources listed. If you find a mistake, write to us: [email protected]