"Next-Generation Creation" by 3D Generative AI ── SIGGRAPH 2025 Frontline Report

1. Introduction: Surpassing the "Image→3D" Barrier in 2025

Generative AI (GenAI), which gained attention for text and image generation, began to significantly penetrate the 3D domain from 2024. With the advent of OpenAI's Shap-E and NVIDIA's Instant N-eRF, "prompt input→3D mesh in seconds" became the norm, drastically transforming the initial stages of production pipelines. As of 2025, the evolution is accelerating from "single object generation" to "scene reconstruction," "animation generation," and "clothing generation with physical properties."github.comgaragefarm.net

2. Three Breakthroughs Presented at SIGGRAPH 2025

2-1 CAST: Transforming a Single Photo into a "Touchable 3D Space"

CAST (Component-Aligned Scene Reconstruction) generates a 3D scene from a single RGB image, estimating hidden surfaces and depths while maintaining contact and gravity relationships between objects. The research team,

• Object Recognition: Simultaneous detection of hundreds of classes using a large-scale vision Transformer

• Camera Pose Estimation: Correcting perspective distortion through scene optimization

• Geometric Consistency Loss: Completing "invisible back surfaces" with a NeRF-derived model
  
  Applications are anticipated in furniture VR, robot simulation, and movie previsualization.digitalmedianet.com

2Anim: Storyboard→3D Motion

Storyboard creation, widely used in the film and animation industries, still has strong analog elements. Sketch2Anim analyzes pose sequences and motion lines using a diffusion model to automatically generate actions like walking and jumping. It allows for immediate import in FBX format to Unity/Unreal, enabling pre-shoot previsualization and instant previews for avatar presentations.digitalmedianet.com

2-3 Dress-1-to-3: "Sewing Pattern Included" Clothing from a Single Photo

Current virtual fitting solutions face challenges with unnatural fabric movement due to static meshes. Dress-1-to-3 uses a diffusion model and physical optimization (C-IPC) to estimate **fabric parameters (Young's modulus, bending stiffness)**, achieving behavior consistent with real-time game simulations. This is expected to reduce return rates in e-commerce.digitalmedianet.com

3. Technical Background: Fusion of Diffusion Models and NeRF

Diffusion models achieve high-quality generation by reversing the noise reduction process, but in 3D, multi-view consistency becomes a challenge. CAST introduced view consistency loss, Sketch2Anim introduced temporal consistency loss, and Dress-1-to-3 introduced physical loss to overcome consistency issues. NeRF-based methods consistently learn image→3D→rendering through a differentiable renderer, reducing computational costs by several tens of times.garagefarm.net

4. Expansion of Industrial Applications

5. Impact on Japanese Companies

• CG/Animation: Fusion with Cel Animation Culture, Semi-Automation of Layout Process
  
  

• Apparel: ZOZO and Uniqlo Enhance Virtual Fitting to Strengthen Domestic and International Sales
  
  

• Manufacturing: Toyota and Mitsubishi Heavy Industries Rapidly Construct Digital Twins, Streamlining Line Changes
  In Japan, labor shortages are a bottleneck, but generative AI can shift work styles from "worker→supervisor".

6. Issues - Copyright, Ethics, Environmental Impact

1. Copyright: Risk of infringement due to mixing existing 3D assets
   
   

2. Ethics: Misuse of fake products and disguised fittings
   
   

3. Environmental Impact: CO₂ emissions from massive GPU training
   Both legal development and corporate governance are required.

7. Implementation Steps and Recommended Tools

1. PoC: Small-scale verification with Shap-E and LumaAI
   
   

2. Workflow Integration: Add diffusion plugins to Blender / Houdini
   
   

3. CI/CD Implementation: Automate generation scripts with Git + cloud GPU
   
   

4. Guideline Formulation: Establish AI usage terms and quality check flow
   Low-code tools with Japanese UI are also increasing, making it easier for non-engineers to try.

8. Future Outlook - Development into a "World Model"

In the future, 3D generative AI will evolve into a World ModelWorld Modelthat encompasses physics, common sense, and task planning, becoming central to robot control and XR learning environments. There is also a prediction that a "multimodal AGI" combining language LLM and world model will be practical around 2027.businessinsider.com

9. Conclusion

3D generative AI transforms the concept of traditional 3DCG production from "creating from scratch" to "arising by conveying intent." The emergence of CAST, Sketch2Anim, and Dress-1-to-3 pushes "3D for everyone" to a practical level, directly enhancing the competitiveness of Japanese companies. Now is the perfect opportunity to experiment small and leverage big.

Reference Article List

• Digital Media Net "3D Generative AI Transforms How We Create, Design, Interact With Digital Content"digitalmedianet.com

• GarageFarm "Generative AI in 3D Modeling: Transforming Creativity and Workflows"garagefarm.net

• OpenAI GitHub "Shap-E: Generating Conditional 3D Implicit Functions"github.com

• Business Insider "Top AI Researchers Say Language Is Limiting… World Models"businessinsider.com