Skip to content

ai16z Infinite Show

Date: 11/23/24

M3TV Setup with Eliza Agents

Overview

Plan to build a dynamic AI-powered streaming service similar to popular AI-generated channels (e.g., Seinfeld AI). This includes designing characters, sets, and tech infrastructure while incorporating Eliza agents for conversational interactions.

Backrooms

  • Currently agents are interacting with each other + humans mostly in backrooms chats + X or telegram
  • We want to surface interactions into a livestream show format

To-Do List

1. Technical Infrastructure Setup

  • Environment
  • Use local + VPS for stability.
  • Consider a dedicated AI machine for running Eliza agents

  • Keep critical apps like Solana wallet separate for security

  • Hardware

  • Leverage high-performance setups like Mac Minis (M series) for rendering or running AI models (64GB unified memory recommended).

  • Web Solution

  • Consider web approach with fewer dependencies.
  • Use streaming platforms (e.g., Twitch) for scalability instead of individual visitor API calls.

2. AI Characters & Scripts

  • Character Design
  • Utilize characters people already created

  • Use JSON to organize scripts, scene setups, and character interactions.

  • Script Generation

  • Automate writing with AI

  • Topics: Decide on themes or key areas of interaction for characters.

  • Live Interactions

  • Enable human interjections during live streams to enhance humor and prevent infinite AI loops.
  • Develop a "Director Mode" where humans can influence live scenes.

3. Technical Details

  • Tools & Frameworks
  • Investigate Unity / threejs / PlayCanvas for scene control.
    • Perhaps collaborate with ashxn + Waybound?

  • Utilize OBS for live streaming and scene switching.

  • Lipsync

  • Research server-side solutions for TTS with phoneme synchronization (e.g., node.js libraries).

4. Entertainment Features

  • Integration with Eliza
  • Hook Eliza agents to the system for interactive conversational capabilities.

  • Configure Director Mode to control:

    • Camera movements.
    • AI interactions.
    • Scene transitions.

  • AI Infinite Loop

  • Let AI agents interact autonomously to "learn" and generate content.
  • Simulate long-term AI "thought" processes for innovation.

5. Development Workflow

  • Collaborators

  • Engage developers for:

    • Frontend design (R3F/Three.js experience preferred).
    • Backend systems and OBS configurations.

  • Iteration

  • Test with small sets of agents and environments.
  • Optimize response times and avoid echoing/double responses in scripts.

6. Long-Term Vision

  • Incorporate:
  • Calandy automation for scheduling AI interactions.
  • Scalable AI agent management for evolving content.
  • Experiment with autonomous AI R&D loops for innovation and story development.

References

  • Video Inspiration: https://www.youtube.com/watch?v=zD9wofGof80
  • Tools Mentioned:
  • Node.js
  • PlayCanvas
  • OBS
  • Eliza agents
  • Collaborators:
  • SM Sith Lord for backend.
  • Ashxn for "Director Mode."
  • Saori for R3F/Three.js design.

Next Steps

  • Assign roles and finalize collaborators.
  • Begin prototyping with basic character designs and AI scripts.
  • Test a minimal viable product (MVP) for streaming functionality.