Using Steam reviews to build AI playtesting personas

While building Agency Lab, my simulation about running a marketing agency, I wanted to understand what players care about in similar games. Steam reviews give me a starting point: people describing what they enjoyed, what frustrated them and what they wished a game did differently. I wanted to use those observations to shape the personas reviewing my own game.

From reviews to personas

The prototype collects public Steam reviews -- the text, whether the player recommends the game, and how long they had played -- from comparable management games like Game Dev Tycoon, Software Inc. and Two Point Hospital. Those reviews get classified into recurring design concerns: pacing and grind, micromanagement, automation and delegation, late-game depth, UI clarity, onboarding, realism. Praise and frustration become themes I can work with.

I use those themes to define a set of personas for the experiment: an optimizer preoccupied with efficiency, a casual player who cares about onboarding and clarity, a veteran focused on late-game depth, an empire builder chasing growth. Each is a curated character with its own priorities, shaped by what the reviews surfaced rather than invented at random. What I find interesting about this is the connection it creates: a criticism of someone else's game becomes a design question I can ask about mine, and different priorities can turn the same simulated behavior into completely different concerns.

From personas to proposals

The game runs through programmed playstyles, then the AI reviews the summaries with each persona's priorities in mind and proposes changes I could investigate. For example, if reviews of comparable games describe runs losing momentum once the early hustle fades, a persona focused on late-game depth could examine my simulated agency's growth and ask whether an established agency still produces interesting decisions; the proposal might be a pacing change with a checkable goal.

The part of this that excites me is the continuity -- a concern a stranger wrote about a different game can travel through a persona and a simulated run and arrive as a specific, testable suggestion in mine.

Carrying it through to code

Suggestions stay with me to review. When I accept one, it goes to a coding agent that prepares the change and its checks on a separate branch and opens a pull request -- the proposal becomes a starting point I can examine, with the option to reject it at either end.

Some caveats are worth keeping in view. The reviewers are English-language players of other games, an imperfect guide to whoever will eventually play mine, so I treat the patterns as clues. The personas are my curation of those clues, and the critics only ever read summaries -- watching someone actually play still has to happen separately.

Where it leaves me

What I want out of this experiment is a way to carry what players value into the way the game iterates: following a concern from a review of a comparable game through a persona, a simulated run and a proposal, until it becomes something I can test while I am still building.