This talk will explore current generative AI models and new developments for creating complex 3D content efficiently with less input.
Generative AI (GenAI) tools like ChatGPT and GPT-4o have revolutionized how we generate text and images — but creating fast, controllable, and high-quality 3D content remains a major challenge. Unlike text or 2D images, 3D scenes are inherently more complex and data-hungry, and suitable training data is often scarce. Recent breakthroughs like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made it possible to turn ordinary 2D photos into richly detailed 3D scenes. These methods work well when many input images are available, aiming to reconstruct scenes with high fidelity. However, when only a few images are provided, the AI must not only reconstruct what's visible, but also generate unseen content — drawing from prior knowledge, all while remaining efficient. In this talk, Professor Jianfei Cia will start with describing the current trends of GenAI and briefly introducing some frontline 2D GenAI research done by my group at Monash University. Then, he will focus on sharing how his team is advancing the 3D GenAI area through models like MVSplat, MVSplat360, and PanSplat, which aim to generate realistic 3D scenes more efficiently and with less input — bringing us closer to fast, flexible, and accessible 3D creation.
Professor Jianfei Cai
Monash University
Jianfei Cai is a Professor at Faculty of IT, Monash University, where he had served as the inaugural Head for the Data Science & AI Department. Before that, he was Head of Visual and Interactive Computing Division and Head of Computer Communications Division in Nanyang Technological University (NTU). His major research interests include computer vision, deep learning and multimedia. He has successfully trained 40+ PhD students with three getting NTU SCSE Outstanding PhD thesis award and one getting Monash FIT Graduate Research Student Excellence Award. Many of his PhD students joined leading IT companies or become faculty members in reputable universities. He is a co-recipient of paper awards in ACCV, ICCM, IEEE ICIP and MMSP, and a winner of Monash FIT’s Dean's Researcher of the Year Award. He serves or has served as an Associate Editor for TPAMI, IJCV, IEEE T-IP, T-MM, and T-CSVT as well as serving as Senior/Area Chair for CVPR, ICCV, ECCV, IJCAI, ACM Multimedia, ICME, ICIP and ISCAS. He was the Chair of IEEE CAS VSPC-TC during 2016-2018. He had served as the leading TPC Chair for IEEE ICME 2012, the best paper award committee chair & co-chair for IEEE T-MM 2020 & 2019, and the leading General Chair for ACM Multimedia 2024. He is a Fellow of IEEE.
