Pokémon Playroom: exploring React as a game engine

I explored both PlayroomKit and Convex for multiplayer. PlayroomKit made rooms quick to set up; I preferred Convex's reactive queries and WebSocket connection, with more control over the backend than PlayroomKit exposed.

Composing the player

React Three Fiber brings React's declarative component patterns to Three.js. I built on that to compose players and expose their actions through refs. This simplified example sits inside EngineCanvas:

function Player({ health }: { health: number }) {
  const playerRef = useRef<PokemonModelHandle>(null)

  useEffect(() => {
    async function enter() {
      const player = playerRef.current
      if (!player) return

      // Wait for the entrance before starting the attack.
      await player.play('spawn')
      await player.play('attack')
    }

    void enter()
  }, [])

  return (
    <PokemonModel ref={playerRef} species='charmander'>
      <WorldHealthBar health={health} species='charmander' />
    </PokemonModel>
  )
}

Inside PokemonModel, useImperativeHandle defines the actions available to that ref:

useImperativeHandle(forwardedRef, () => ({ play, reset }), [play, reset])

A component ref gives the parent access to a player's actions. Camera and effect systems elsewhere in the scene also need a way to find that player. Alongside the battle, I explored a Zustand registry. Each entry held an ID, an object ref, and its commands.

This simplified excerpt from EngineGroup shows the registration pattern for a Three.js group. It adds the group on mount and removes it on unmount:

useImperativeHandle(ref, () => internalRef.current)

useEffect(() => {
  addGroup(id, internalRef.current, { setSpawn: setIsSpawned })
  return () => removeGroup(id)
}, [id, addGroup, removeGroup])

Other systems could call spawnGroup(id, true) without passing refs through the component tree. The helper reads the Zustand store directly, looks up the registered group, and calls its setSpawn command.

I also tried an entity registry backed by a Map, with queries for entities containing particular components. React subscribed to store changes; imperative systems could read the current store directly.

These patterns let JSX describe what exists while commands control what happens. Animation and physics keep their own update loops, so a declarative scene doesn't require React to render every animation frame.

Three.js's TSL shader language takes a similar approach to shader logic traditionally written in GLSL. I used its composable JavaScript API for flame, grass, and grid experiments. TSL generates shaders for WebGPU or WebGL, so those effects can share the web's module system while still running on the GPU.

I used Blender to rig and animate the supplied 3D Pokémon models and assemble the scenes.

The interface stays in HTML. Pokémon cards are labelled buttons with selection states, battle messages use aria-live, and transitions respect reduced-motion preferences. Keeping that UI in the DOM preserves standard keyboard and focus behaviour and access to components from libraries such as Radix. A canvas shouldn't mean giving up those foundations.

Camera and controls

Experiences like Coastal World made me interested in moving through a 3D world with one thumb or a pointer. The camera needs to follow and turn with the player, so movement doesn't require a second hand just to manage the view.

Use the Camera button to switch to Automatic follow, then try the controller on a phone. One thumb moves Ash while the camera follows his heading. Keyboard, pointer controls, and touch share movement logic built around Rapier physics.

The controller reads Drei's Zustand-backed keyboard store inside useFrame and updates the physics body through rigidBodyRef. Input stays reactive without sending every movement through React state, following React Three Fiber's guidance for frequent updates.

Effects and sequencing

Three.js's chainable animation actions inspired my effects API. I made the chains awaitable so a sequence could wait for an effect to finish.

This sandbox led to reusable SpritePlane and SpriteStream components for the battle. A sprite's ref exposes a chainable API; Charmander's Ember uses this to grow its charging flare:

await fireFlareRef.current
  ?.reset()
  .moveTo(0, 0.4, -1.5)
  .scaleTo(0.2, 0.2, 0.2)
  .to({ position: [0, 0.6, -1.6], scale: [2, 2, 2], duration: 0.7 })

Each method appends work to a Promise queue and returns the same handle. Its then() method delegates to that queue, which makes the whole chain awaitable. useImperativeHandle exposes the handle through the sprite's ref, so callers don't need to know how its tweens or sprite frames run.

Scene coordination

The scene composer brings Pokémon, camera paths, and effects into a sequence of beats. Choose Director, then Play to watch the Pokémon actions and camera movements run together.

The composer uses Legend State to track completion across actors, camera, and VFX. Actions within a beat run together; the next beat waits for them.

Back in the battle, the same idea connects animation to game state. This excerpt moves the camera, overlaps the attack and effect, then applies damage and plays the opponent's reaction:

await commands.camera('player-attack')
await Promise.all([
  commands.animate('player', 'attack'),
  commands.effect(player.element, battlePositions.player, battlePositions.opponent),
])
const nextOpponentHealth = await commands.damage('opponent', PLAYER_DAMAGE)
await commands.animate('opponent', 'hit')

Damage updates React state, which flows back into the health prop on WorldHealthBar. Character clips run through Three.js's animation mixer; effects keep their own animation routines. The common command API coordinates both.

React for games

Unity's browser builds connect their runtime to the surrounding page through JavaScript integration. In this multiplayer proof of concept, the scene shares React components and state with the page from the beginning. I want game state and persistence to use familiar web APIs too, through tools such as Convex or Drizzle.

The data layer is the next piece to improve. My registry made it possible to find an object and call its methods. I'm exploring entities with traits such as position, health, and current action, and systems that query those traits directly. I want components to work with that data without custom wiring for every query.

I'd like to build a shared game framework around these patterns. An encounter could load its own Pokémon, effects, and rules as a module, reusing the accessible interface and controls. Developers could extend those pieces through familiar React components while keeping direct control over graphics and timing.

Keep exploring

WWJD: 2.65x Recurring Revenue per VisitorFull-film views per visitor rose 91% after an end-to-end rebuild of the viewing and donation journey./blog/wwjdfilmRead case study

Gamified Social Impact Survey for UNDPA multilingual UNDP research survey redesigned as an accessible, gamified web experience with familiar controls, Three.js feedback, and personal results./blog/undp-surveyRead case study