Speech based AI childrens game

Overview

I wanted to create an AI prototype that would demonstrate the potential of real-time AI voice interaction in a way that was genuinely fun to use. I chose to build an animal guessing game for children – a playful and brutaly honest user group – that could guide them through the complete game, teaching the rules, rewarding them with the correct answer, and helping them if they get stuck… autonomously.

I built the project using Next.js and TypeScript for rapid development, easy deployment, and testing with children. I integrated WebRTC with OpenAI’s Realtime API for voice conversation, DALL-E (and later GPT-IMAGE-1) for dynamic image generation, custom FFT audio analysis for visual feedback, a blur-to-reveal game mechanic, AI function calling for game state management, netlify blob image caching to reduce costs, and comprehensive error handling for robust testing with children.

The final application successfully demonstrates seamless integration of four different AI technologies (GPT-4, GPT-IMAGE-1, Realtime API, function calling), real-time voice interaction with sub-100ms latency, dynamic image generation prompting, and a clean, accessible interface suitable for young users. Children can naturally converse with an AI to play the game, with visual feedback making the experience engaging and intuitive. The progressive reveal system creates anticipation and excitement and helps if they start to struggle.

This project is hidden online to save bandwidth and API costs. Please email me for access.