Product · Game Design · Research Tooling

We Built the Game.
Then I Automated the Study.

No off-the-shelf game gave us the control we needed. The system that runs it went from 60 manual steps to one click.

Role

Product Owner · Game Designer · Research Engineer

Tools

Unity, Node.js, PsychoPy, OBS Studio, iMotions

Approach

AI-Assisted Development

Timeline

2025–2026

Wildlight gameplay: the title screen introduces Enoki, a forest explorer foraging for gold, then play begins.

The Game

The control layer running: a terminal executing node server.js, logging each system as it connects and each recording as it starts.

The System

The Research Question

“How do players build, lose, and rebuild understanding while navigating a complex game environment?”

Nothing on the market let us control agent behavior at that level. So we built one.

01 — THE GAME

The Game

Project timeline: seven months of game design across three playtest rounds, followed by six weeks of system build, with two failure points marked in red
Seven months of design. Six weeks of build. Two failures, both fixed the same day.

I owned the design. An SDE and I built the first playable in weeks, not months. Then we tested it. 36 participants played it across 3 rounds. Each round told us what was too hard, too easy, or too confusing. We changed the level curve, the agent behavior, and the death-retry loop based on what we saw.

The final version does one thing well. It makes players build a mental model, break it, and rebuild it — on purpose.

The same level three playtest rounds apart. Early prototype, January 2025: generic stimuli borrowed from stock images, rules explained in text beside the screen, no feedback and no scoring surface. Final build, July 2025: four custom agents with fixed HP and point values, a health bar, gold counter and visible action radius, with every gain and loss shown on screen.

36

Playtesters · 3 rounds

6

Levels, difficulty-calibrated

4

Game agents

0→1

Prototype to research build

Group 1 — Unambiguous

These two teach the rules. Good is good. Bad is bad.

Lucky Leaf game sprite

Lucky Leaf

HP
+20
Points
+20
Toxic Vine game sprite

Toxic Vine

HP
-20
Points
-20

Group 2 — Coactivation

These two break the rules. Health and score move in opposite directions. Every encounter forces a choice.

Fat Mushroom game sprite

Fat Mushroom

HP
+20
Points
-20
Tall Mushroom game sprite

Tall Mushroom

HP
-20
Points
+20

The first two agents let players build a model. The last two make that model useless. What players do next is the data.

How It Was Designed

Three rounds, 36 playtesters. Every round produced a change, and every change went back through the same loop.

Iteration loop diagram: Spec (me — game mechanics, agent rules, art direction) leads to Build (SDE — Unity implementation), then Playtest (real participants, N=5 to N=10 to N=21), then Change (rebalance, then re-spec), returning to Spec. Center label reads 3 rounds, 36 playtesters. Below: round 1 N=5 March 2025 rebalanced agent damage and reward values; round 2 N=10 May 2025 adjusted agent density, combinations, and difficulty; round 3 N=21 June 2025 rebuilt the feedback system with clearer mouse instructions and visible HP and point loss.

Tracked in Notion — 5 sprints, 40+ tickets.

The game was done. The harder problem was running it 50 times without a mistake.

02 — THE SYSTEM

The Problem

The game worked. Running the study didn't.

Every session needed five programs to stay in sync. One mistimed click and the data was unusable. 60+ minutes of manual work per participant.

Five programs open at once during a session: iMotions showing physiological signals, OBS screen recording, the Wildlight game, the PsychoPy experiment script with a rating slider, and the web practice game. Headline reads: one mistake ruins the session, 60+ minutes per participant.
Every session. Five programs, kept in sync by hand.
🎮

Unity game

6 levels, each needs its own recording and event marker

🌐

Web practice game

No API. No way to know when it starts or ends

🎥

OBS

Start, stop, and rename 7 files without missing a frame

🧠

iMotions

Mark 6 level boundaries by hand, on a second computer

🖥️

PsychoPy

Run the script, replay videos, collect CRM ratings and 2 questionnaires

60+ minutes. Zero margin for error.

02.5 — THE HARD PART

No API. So it reads the screen.

Participants train on the CRM rating scale using a browser game. We don't own that game. We can't modify it. The system still needs to know the exact moment it starts and the exact moment it ends.

  • No APInothing to call
  • No logsnothing to read
  • No eventsnothing to listen for

CRM — participants rate their experience continuously while re-watching their own gameplay.

The browser training game as participants see it: gold monkey figures and a red target on a black field, with a score counter in the corner. The gold-on-black palette is the color signature the detector counts.
The training game. Off-the-shelf, and closed.

Read the pixels instead.

The detector samples the screen in real time and counts pixels matching the game's color signature.

That alone wasn't enough. Before the study begins, PsychoPy plays a tutorial that explains the CRM scale. It uses Pac-Man as the example, because everyone knows Pac-Man. Pac-Man is gold. The detector fired on the tutorial and started recording the wrong screen.

So it checks two things instead of one. Gold pixels above threshold, and no PsychoPy gray frame. Both have to be true. Edge regions stay excluded so the browser interface never triggers it.

Tutorial video

Gold pixel ratio ≥ 3%
yes
PsychoPy gray frame
present
Recording
does not start

Practice game

Gold pixel ratio ≥ 3%
yes
PsychoPy gray frame
absent
Recording
starts
The pixel detector running: a live analysis panel tracks the gold pixel ratio against a 3 percent threshold and the sidebar gray ratio against a 5 percent ceiling, while the captured display shows the detect region and the excluded edge bands. When both conditions are met, OBS starts recording.
The detector running live. Gold pixel ratio crosses 3% and recording starts.
Sample rate
continuous, per frame
Signal 1
gold pixel ratio ≥ 3%
Signal 2
no PsychoPy gray frame
Exclusion
browser chrome and screen edges

One signal was easy. Knowing it wasn't enough took a false positive to find.

The Build

Enter a participant ID.
Everything else is automatic.

System architecture: the researcher enters a Participant ID and makes one display switch. That feeds server.js, the control layer, which orchestrates five systems — on Computer A the Unity game (6 levels, TCP event per level), the web practice game (no API, screen pixel detection), OBS (7 recordings, WebSocket and auto-rename), and PsychoPy (replay and rating, retry read with a 10-second window); on Computer B, iMotions (event markers, direct TCP, under 1 millisecond). The researcher touches one layer. The system touches five.

01

The Controller

Game TCP events auto-trigger OBS recording per level.

server.js → OBS WebSocket

02

Practice Detection

Screen pixel analysis detects a web game with no API.

Gold pixel ratio ≥ 3% + edge exclusion

03

Experiment Script

Auto-replay + real-time rating after each level.

PsychoPy rewrite: 2500 → 920 lines

04

Physio Integration

Sub-millisecond event markers to Computer B.

Direct TCP → iMotions (no middleware)

05

Edge Hardening

Every live test exposed new failures. Every failure became a fix.

10× retry, 30s wait, display config, cursor fix

What Broke

Two failures in 50 sessions. Both fixed before the next participant.

Participant 10: the screens

Two monitors. The connection dropped. Screen detection failed, so recording never started, so PsychoPy had no video to load. The session crashed.

Fixed the same day. The system now checks the display configuration before every session and confirms the recording started.

Participant 25: the timing

OBS doesn't release a finished file at a fixed time. It saves, renames, then frees it. PsychoPy tried to read the file too early and crashed.

Fixed the same day. PsychoPy now retries for 10 seconds. The researcher can also go back one page and try again. No session is lost.

The Result

One session, start to finish: the researcher types a participant ID, then all five systems connect and verify, a welcome screen gives instructions, and the practice game trains the CRM scale. Six levels then repeat the same loop automatically — play the level, OBS auto-records, iMotions takes an event marker under one millisecond, and PsychoPy replays it for rating. The researcher does nothing during this. Afterwards PsychoPy administers the SAM questionnaires, seven videos are verified and renamed, and the session ends with no manual cleanup. One input at the top. Seven video files and six markers at the bottom.

60+→ 1

Manual steps per session

50

Sessions run

0

Data-loss incidents

5

Systems orchestrated

<1ms

Marker latency

03 — THE METHOD

How It Was Built

I wrote the requirements. The AI wrote the code. Real hardware decided what shipped.

This only works if the requirements are testable. Knowing the experiment protocol was the hard part. Generating the code was not.

The Human

Game design & experiment protocol

Requirements & acceptance criteria

Hardware constraints & trade-offs

Testing on real equipment

The AI

Code generation & architecture

Cross-platform implementation

Debugging from error logs

Technical analysis & iteration

Reflection

Two problems, one job. Designing the game meant deciding what to cut and when to stop iterating. Automating the lab meant defining what “correct” looked like before writing a line of code. Both came down to the same thing — knowing the user well enough to write the spec.

Thanks for reading!

Work With MeReturn to Home