MIT, Google, Northeastern Unveil InstructMesh, Letting Novices Fix 90% of Flawed 3D Models
Updated
Updated · MIT News · Oct 1
MIT, Google, Northeastern Unveil InstructMesh, Letting Novices Fix 90% of Flawed 3D Models
1 articles · Updated · MIT News · Oct 1
Summary
Nearly 80% of AI-generated Thingiverse-style models were structurally flawed, but novices using InstructMesh identified and fixed those problems about 90% of the time, according to expert review.
InstructMesh combines Microsoft's TRELLIS 3D generator with GPT-4 so users can highlight parts of a model and request edits in natural language before printing.
Researchers used the tool to produce functional customized objects including a dragon-handled mug, butterfly-wing glasses, a denim-look knee brace and a shrimp-like bristle bot enclosure.
The project targets a core weakness in generative 3D AI—models know how objects should look, not how they should work—and aims to make functional fabrication accessible to non-experts.
The team will present the work at ACM UIST in November, while future versions may add physics simulation, TRELLIS.2 integration and possible use in Google's AR platform.