严天宇Ph.D. student · DUT
I build AI companions that remember
Advised by Prof. Huchuan Lu. I work on multimodal LLMs, LLM agents and role-playing, interactive narrative and AI societies, and visual segmentation.
6 papers
as first, student-first or co-first author, at CVPR, ECCV, IJCAI, IEEE TIP, IEEE TGRS and ACCV.
Four threads run through my work, from Segment Anything models to AI companions and the societies they live in.
Proactive assistance in continuous egocentric video.
02Digital humans with long-term memory and a consistent persona, such as MIO.
03Persistent worlds people can follow and steer, such as Synthetic World.
04Segment Anything for marine animals and remote sensing: Dual-SAM, HFP-SAM, MAS-SAM.
Two systems I built at Alaya Lab.
A conversational digital human served through an OpenAI-compatible API, so it plugs into custom front ends, companion devices, and live-stream interaction. Its memory keeps growing with every conversation.
Users define a world and its characters; AI residents post, reply, and react as events unfold. Visitors can watch, or steer the story with "Next Act". Past events and character states shape what happens next.
Segment Anything models for marine animals and remote sensing, and proactive assistance in egocentric video.
* equal contribution. Student-first: first student author after the advisor. Animations are schematic sketches drawn in code, not model outputs.
A wavelet frequency prior steers a frozen SAM: FGA injects frequency masks into lightweight adapters, FPS turns high-frequency windows into point prompts, and Full-View Mamba refines the mask at linear cost.
Two SAM branches see the original and a gamma-corrected image, linked by multi-level coupled prompts. DFAM fuses their features, the two decoders supervise each other with pseudo-labels, and C3P predicts each pixel's criss-cross connectivity instead of a plain mask.
Built the algorithms and systems behind the digital human MIO and the AI society platform Synthetic World, from the models to the live product.
Researched proactive assistance in continuous egocentric video; the resulting paper, Vinci2, is accepted to ECCV 2026.
Applied work in medical imaging and media.
A 3D Segment Anything Model with visual Mamba that segments the lesion in MRI, then diagnoses placenta accreta spectrum from the segmented region.
Learns a vector for each original work in the library and compares new submissions against it to check originality, with 3D reconstruction of original content.