Google's Open-Source Compiler Lets AI Run on Data It Can Never Read

Google has unveiled HEIR, an open-source compiler that automatically converts pre-trained AI models so they can perform inference directly on encrypted data using homomorphic encryption. The company demonstrated the tool on four real applications — private recommendations, credit-card fraud detection, intrusion detection on encrypted traffic, and hotword detection — all running on ordinary CPUs.

Key Takeaways

Google has unveiled HEIR, an open-source compiler that automatically converts pre-trained AI models so they can perform inference directly on encrypted data using homomorphic encryption.
The company demonstrated the tool on four real applications — private recommendations, credit-card fraud detection, intrusion detection on encrypted traffic, and hotword detection — all running on ordinary CPUs.

AI synthesis3 sources7

Google has unveiled HEIR, an open-source compiler that lets pre-trained AI models run directly on encrypted data — a step toward making "private AI inference" practical for real products.

Homomorphic encryption, the technology at the heart of the project, allows computations to be performed on ciphertexts without ever decrypting them. A cloud service could serve recommendations without seeing the user's features, or scan network traffic for anomalies without reading the packets. The trade-off has always been cost: encrypted computation is far slower than plaintext computation. But Google argues that overhead is falling fast, turning the capability-versus-privacy trade-off into a question of economics rather than possibility.

HEIR — the Homomorphic Encryption Intermediate Representation — is an MLIR-based compiler toolchain that converts pre-trained models operating on unencrypted data into versions that operate on encrypted inputs. The project began inside Google in 2023, and the company's vision is to make it "a one-click solution" that lets non-experts ship encrypted inference without hiring a team of cryptographers.

"Manually converting an existing program to use homomorphic encryption efficiently requires a team of cryptographers," Google researchers wrote. HEIR aims to remove that bottleneck.

To show the technology has left the lab, Google published four applications compiled with HEIR and benchmarked on single-threaded CPUs: a deep-learning recommendation model (joint work with Belfort Labs, LG and NYU) that serves private content recommendations; a credit-card fraud detector built with Niobium and hardshell.ai; the Kitsune network-anomaly detector, compiled with Niobium so providers can flag intrusions without seeing packet contents; and a hotword detector with Belfort Labs that lets an audio AI agent listen for trigger words without recording everything around it.

Google has also partnered with homomorphic-encryption hardware accelerator firms — Belfort, Niobium, Cornami and Optalysys — and says the compiler has become a research platform used by universities including Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, the University of Edinburgh and Tsinghua. The full source code is available on GitHub.