Virtual Pets: Animatable Animal Generation in 3D Scenes

[cpvr]

I recently came across a report by Cornell University, where they discuss 3d Scenes in Animated Animal Generations

Virtual Pets: Animatable Animal Generation in 3D Scenes

Toward unlocking the potential of generative models in immersive 4D experiences, we introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal species within a 3D environment. To circumvent the limited availability of 3D motion data aligned with...

arxiv.org

Toward unlocking the potential of generative models in immersive 4D experiences, we introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal species within a 3D environment. To circumvent the limited availability of 3D motion data aligned with environmental geometry, we leverage monocular internet videos and extract deformable NeRF representations for the foreground and static NeRF representations for the background. For this, we develop a reconstruction strategy, encompassing species-level shared template learning and per-video fine-tuning. Utilizing the reconstructed data, we then train a conditional 3D motion model to learn the trajectory and articulation of foreground animals in the context of 3D backgrounds. We showcase the efficacy of our pipeline with comprehensive qualitative and quantitative evaluations using cat videos. We also demonstrate versatility across unseen cats and indoor environments, producing temporally coherent 4D outputs for enriched virtual experiences.


There's also some replica that they've generated with their research methods as well:

arXiv + Replicate | Replicate

replicate.com

To my knowledge, this seems tied into OpenAI and other Ai software as well. This research paper was published and documented back in 2023.
The following researchers were involved in the project:
Yen-Chi Cheng, Chieh Hubert Lin, Chaoyang Wang, Yash Kant, Sergey Tulyakov, Alexander Schwing, Liangyan Gui, Hsin-Ying Lee


The PDF file is also available to read and access:

https://arxiv.org/pdf/2312.14154

 

[Kross]

This would bring more realism to virtual pet gaming. With it, Virtual pets won't be making repetitive moves. Just imagine virtual pets being able to adapt to our environments. That would be quite exciting. Seems like what would bring deeper levels of immersion in virtual pet games.

 

[uptrendfinancialsignal]

The 3D stuff is getting crazy these days. Some of these pets look more alive than actual cartoons. 😂 I like seeing how animations add personality, small tail movements, blinking, jumping around. It makes the whole thing feel less static. But at the same time, I hope pet sites don’t abandon the simple 2D charm. Not everybody wants high-end graphics. Still, seeing pets in full 3D scenes is cool, especially when they animate smoothly instead of moving like robots.

 

[cpvr]

The 3D stuff is getting crazy these days. Some of these pets look more alive than actual cartoons. 😂 I like seeing how animations add personality, small tail movements, blinking, jumping around. It makes the whole thing feel less static. But at the same time, I hope pet sites don’t abandon the simple 2D charm. Not everybody wants high-end graphics. Still, seeing pets in full 3D scenes is cool, especially when they animate smoothly instead of moving like robots.

The cartoon version of pet sites would be quite interesting to see and I do hope that one day we'll see that occur, but currently, there are some pet sites that are in development with 3'd scene & animations.

We are also entering an era where robototic-pets are coming out too, which is something that is really cool to see, as it's extension of the Tamagotchi and Furby era.

I also believe that this will be in effect a great new twist to the virtual pet site scene in the future too.

But to be honest, Pokemon and Digimon are generally in the realm of "Cartoon" versions of virtual worlds/pet sites, but they're not exactly in the same realm of virtual pet sites like Neopets.