Conversational AI for Hubs NPCs

[heaversm]

What is the possibility of incorporating chatGPT (or other conversational) AI into Mozilla Hubs? Would it be possible / require managing your own Hubs Cloud instance?

It would be cool to be able to have non-scripted interactions with NPCs in a hubs world - potentially these could even be trained on data related to your hubs world, to keep conversation contextual to the experience vs. open-ended.

Thought of it when I saw this NVIDIA demo

https://www.youtube.com/watch?v=nAEQdF3JAJo

[Jon]

Thanks for submitting an idea to the Mozilla Connect community! Your idea is now open to votes (aka kudos) and comments.

[michaelhmorran]

Hey @heaversm! Thanks for the idea. I believe one of our users in Japan is currently working on this using their hubs cloud instance: https://twitter.com/kiradragonp/status/1644542146891091968

I myself have connected a personal hubs cloud instance to the OpenAI API for Dalle2 image generation as part of my work with the OnBoardXR theatre festival.

As far as I know, the team does not have any immediate plans to make any generative AI offerings part of the core hubs product due to the privacy concerns, however we love to see the awesome experiments that people are able to accomplish just through client-side development and deployment. 

I and other members of the Hubs team would love to chat more with you about how you see generative AI's role in the virtual world applications!

[heaversm]

Thanks for the response and the link! Looks like a cool implementation. Maybe I'll play around with this and try to get something working on my own instance.