Researchers at MIT and IBM have unveiled GIFT (Geometric Inference Feedback Tuning), a new framework that dramatically improves how AI models generate computer-aided design (CAD) programs from 2D images — a capability that could accelerate rapid prototyping across aerospace, automotive, and consumer product industries.
Presented at the International Conference on Machine Learning (ICML), GIFT teaches vision-language models (VLMs) to convert a simple 2D image and text description into executable Python code that generates functional 3D CAD models. The system outperformed competing techniques while using only about 20 percent as much computation.
Learning from Mistakes
The core innovation behind GIFT is its ability to generate "model-aware" training data. Rather than augmenting datasets with random variations — the conventional approach — GIFT first probes the model to understand its specific weaknesses. It asks the model to solve a CAD generation problem multiple times in parallel, checks which attempts are nearly correct, and transforms those "near-misses" into teaching examples.
"If we sample the model 10 times and it generates 10 correct answers, there is not much for it to learn. We care about the in-between cases, where the model might only solve the problem 50 percent of the time," says lead author Giorgio Giannone, a research affiliate in MIT's DeCoDE Lab and a principal scientist at Red Hat.
From Airplanes to Appliances
Nearly every physical product — from airplane components to kitchen appliances — begins as a CAD model. Engineers use these models to simulate crash tests, aerodynamic performance, and durability before building physical prototypes. Automating the conversion from 2D concept to 3D model could slash development timelines and costs.
The GIFT system uses a technique called inference-time scaling, which allows a static, already-trained model to generate better outputs without the high cost of full retraining. This means users can tailor the system to their time and budget constraints.
"Industry teams are eager for AI that can help speed up the creation of designs, but today's models often produce simple shapes inadequate for practice," says co-senior author Faez Ahmed, associate professor of mechanical engineering at MIT. "What excites me is that GIFT gives models a way to improve themselves, learning from their own errors rather than waiting for more human-made data."
Next Steps
The MIT team plans to expand GIFT so it can teach models to optimize 3D models for performance and manufacturability. They also aim to apply the system to larger models and more diverse CAD generation tasks.
The research was conducted in collaboration with the MIT-IBM Computing Research Lab, with additional support from Red Hat.
