Why one prompt wasn't enough: splitting PromptMotion into specialist agents

PromptMotion takes a rough idea from a creator and turns it into prompts that AI video models can actually work with. Someone types "a lonely robot finds a flower in a ruined city", and a few seconds later they have a cast, a shot list and a prompt for every clip, written for whichever model they use: Veo 3.1, Kling, Runway, MiniMax or Seedance.

I built it in about five months. It passed 1,500 users in the first two, mostly through organic search and a small amount of Meta ads.

The engine behind it now runs as a small team of agents. It didn't start that way, and I never sat down and planned it like that. Each version came from something going wrong for real users. I'm writing up the whole journey in three parts, and this first one covers the early versions: how one prompt became a team of specialists, and how those specialists learned some film craft.

Throughout the diagrams below, the colours mean the same thing:

V1One prompt doing everything

The first engine was a single LLM call with a long system prompt. Idea in, prompt out. I shipped it fast because I wanted to know whether anyone would use the product before I spent months on it.

People did use it. But the results were all over the place. A few prompts were genuinely good. Most read like a description of a video rather than direction for one.

Once I looked closely, the reason was obvious. That one prompt had to understand the idea, invent a story, plan the shots, write the camera and lighting language, and then format everything for a specific model's syntax. It did a bit of each and none of them well. And when a result came out bad, I had no way to tell which of those jobs had failed.

V2Give each job its own agent

So I split it up. The engine became a Script Architect that coordinates a set of specialist agents, each responsible for one thing. The Concept Analyst works out what the idea is really about emotionally. The Story Architect builds the arc and decides who is in it. The Scene Writer designs each shot. The Output Formatter takes the finished plan and writes a prompt for each video model.

I built it with LangChain and LangGraph. I did look at CrewAI, but I wanted the flow to be an explicit graph with state I could open and inspect at every step, not agents chatting to each other in character. When something breaks in production, I want to see exactly where.

The quality jump was immediate, but the bigger win was debugging. Weak story? Look at the Story Architect. Kling rejecting the syntax? That's the Formatter. Keeping formatting separate also paid off later. When I added MiniMax and Seedance, I changed one agent and nothing else.

V3Knowledge the agents can look up

Splitting the work showed me the next gap. The Scene Writer knew what should happen in a shot, but not how a filmmaker would actually shoot it. Which camera move makes a moment feel hesitant? What kind of light makes a scene lonely instead of just dark?

My first attempt was to write all of that into the agent prompts. They got long, expensive and painful to edit. So I moved it out into a cinematography knowledge base, a set of mappings from emotions to the techniques that create them. The Scene Writer queries it for each shot and only pulls in what's relevant.

This is what sits behind Send to Director in the app, where the AI plans camera, lighting, movement and composition before writing the final prompt.

What I took from this stage

Two things. If one prompt is doing several jobs, split it. You'll get better results, and you'll finally be able to see which part failed. And keep domain knowledge somewhere your agents can look it up, rather than baking it into their prompts where it gets expensive and hard to change.

You can watch the engine run live or read the full PromptMotion case study. If you're building something like this yourself, here's how I approach multi-agent AI development.