AI-Generated User Stories Need Human Judgment

I joined a meeting in which the product owner presented the team with 500 recently written product backlog items for a new product. The goal of the meeting was to see if anything was missing.

No one could tell. With 500 fairly small items, it was extremely hard—perhaps impossible—to spot an important need the backlog had overlooked.

And five hundred items is not even an especially large backlog. Plenty of organizations have backlogs 10 times that size, or even larger.

But imagine how different that conversation would have been if the team had started with 25 or 50 high-level epics. At that level, people could have looked across the product and asked:

Are we missing an important type of user?
Is an essential workflow absent?
Does this reflect how we intend to position the product?
Where are the capabilities that will distinguish us from competitors?
Have we included features merely because similar products have them?

Once those high-level decisions had been made, the team could have decomposed the most important epics into smaller backlog items.

AI makes it very easy to skip the step of identifying and reviewing the product's high-level epics before decomposing them into stories.

Teams can jump immediately to hundreds of detailed backlog items without first considering whether those items collectively describe the right product.

Give AI a description of a product and, within minutes, it can produce hundreds of user stories. Ask again, and it will add acceptance criteria, edge cases, dependencies, and suggested priorities.

The results are often well-written and impressively thorough.

That is exactly what worries me.

A Detailed Backlog Can Create False Confidence

The danger is not that AI will produce obviously terrible backlog items. Those would be easy to spot and reject.

The greater danger is that AI will produce a large number of plausible, polished items that look complete.

The sheer volume of detail can discourage people from questioning the backlog. It is difficult to recognize what is missing when you are looking at hundreds of individual stories. It is also difficult to distinguish an essential capability from something AI included because it seemed reasonable.

A substantial backlog is not necessarily a complete backlog. And a complete-looking backlog is not necessarily a good one.

Teams need to begin at a level where they can still understand the product as a whole.

I prefer to start with a relatively small number of high-level epics identified by humans. AI can then review those epics and suggest important areas the team may have overlooked.

After the team has considered the product at that level, AI can help decompose one or two of the most important epics at a time.

That sequence is much safer than asking AI to create a comprehensive backlog in one pass.

Start with What Will Make the Product Successful

Before asking AI to generate backlog items, describe how the product is intended to succeed.

Is it an all-encompassing product that will compete by offering nearly every capability users might need? Or is it intended for a narrow audience with a few unusually valuable features?

What gap exists in the market? Why might customers choose this product instead of a competitor? Are there financial or timing goals that should affect product decisions? Which user personas or roles are most important?

This context helps AI generate suggestions that fit the intended product rather than a generic version of that product category.

Without that context, AI will often suggest the expected features. Those table-stakes features matter, but they rarely make a product exceptional. AI may miss what the Kano model calls delighters: capabilities users may not explicitly request, but that can distinguish a product from its competitors. That's as true for internal products as commercial ones — an internal tool may not have a commercial competitor, but it still competes with the current process, existing tools, spreadsheets, workarounds, and doing nothing.

Before generating any product backlog, explain:

The outcome the product should create
Who will use it
How those people perform the work today
What is slow, costly, risky, or frustrating about the current approach
What would motivate people to adopt something new
How the team will know the product succeeded
Which constraints and non-goals should limit the proposed solution

For a commercial product, also explain the market opportunity, important competitors, reasons customers might choose this product, and any relevant financial or timing goals.

This strategic context is almost like acceptance criteria for the product itself. It describes what must be true for the product to be considered successful.

Start there. Then use AI to help explore how the product might achieve that success.

A Well-Written Story Can Still Be the Wrong Story

AI does not normally have firsthand evidence from interviews and observations with this product's users. Unless the team provides that evidence, AI is generating plausible features rather than demonstrated needs.

That means teams should ask more than whether an AI-generated story is clear and testable.

They should also ask:

Does this support the way we intend to position the product?
What evidence do we have that users need it?
Is it necessary, competitively differentiating, or merely plausible?
Could we omit it and still succeed?
Do the acceptance criteria describe what matters, or merely what was easy for AI to specify?

That last question is particularly important.

Imagine a team building a receipt-scanning expense application. The product is intended to succeed by making expense submission nearly effortless: a user should be able to photograph a receipt and finish the expense with almost no typing.

Without that context, AI might generate acceptance criteria such as:

Users can upload JPG, PNG, and PDF receipts.
Files larger than 10 megabytes display an error.
Users can enter the merchant, date, amount, and category.
Required fields are validated before submission.
A confirmation appears after the expense is saved.

Those criteria are specific and testable. They also describe a fairly ordinary upload form.

Acceptance criteria that protect the product's intended advantage might instead address:

Extracting the merchant, date, amount, and category from a photograph
Asking users to correct only information the application cannot determine confidently
Detecting duplicate receipts
Suggesting a sensible expense category
Identifying potential policy violations before submission

The first set describes a feature that works. The second helps describe why someone would prefer this product.

But without strategic context and human review, it can easily produce the first: sensible functionality for a generic product.

Humans need to determine what will make this particular product valuable.

AI Should Support Co-Creation, Not Specification Handoffs

Suppose a product owner works alone with AI, creates polished stories and acceptance criteria, and then hands them to the team.

That process may be fast, but it loses many of the benefits of co-creating a solution.

Users understand their goals, frustrations, environment, and current behavior. Customers — who are not always the users — bring purchasing concerns, organizational needs, and their own definitions of value.

Developers, testers, designers, and other team members contribute technical possibilities, risks, constraints, alternative solutions, and ways to test an idea incrementally.

The product owner brings product direction and responsibility for making tradeoffs. The product owner ultimately decides what enters the product backlog, but the best product owners make those decisions in consultation with users, customers, and the team.

These are not isolated responsibilities that should be performed through a sequence of handoffs. They are different types of knowledge brought into the same product conversation.

Co-creation helps teams avoid solving the wrong problem. It surfaces information no one person would have. It creates ownership and trust. It also makes the eventual solution more adaptable because more people understand the reasoning behind it.

AI can contribute to that process. It can propose possibilities, uncover assumptions, identify missing cases, and ask useful questions.

But it has neither accountability for the product nor firsthand knowledge of its particular users and context.

Shared understanding exists among people. AI can help create the conditions for it, but it cannot replace the conversations through which it develops.

Treat AI-Generated Stories as First Drafts

After a product owner and perhaps the team have reviewed an AI-generated story, it can enter the product backlog.

From there, it should be treated like any other first-draft backlog item. It must be prioritized. It may need more discussion and additional detail. It may need to be split, revised, or removed.

I also think it is useful for AI to include initial acceptance criteria when it generates a story. Producing those criteria is inexpensive, and they often help a reader understand the intended behavior.

But they remain first drafts.

As an item rises in priority and gets closer to being brought into a sprint, the team should critique its acceptance criteria during backlog refinement.

AI can help with that. It can look for omissions, overlaps, inconsistencies, and missing examples. It can be especially useful when a story needs concrete examples of calculations or business rules.

Our Story Critic AI skill reviews backlog items and identifies whether they appear adequately detailed for refinement or a sprint.

Even if an AI tool says a story appears ready, the team must still apply its own judgment.

AI can supplement backlog refinement. It cannot eliminate the need for it.

If anything, easy AI-generated detail makes collaborative refinement more important.

Use AI to Expand Your Thinking

"Trust but verify" is good advice for using AI, but it doesn't go far enough.

A team should not merely check whether AI wrote a backlog item correctly. It should determine whether the item deserves to be built at all.

Start with users, product positioning, desired outcomes, and a small number of high-level epics. Ask AI to question, supplement, and expand the team's thinking. Let it propose missing items, decompose selected epics, draft acceptance criteria, and critique stories during refinement.

Then apply human judgment.

Talk with users and customers. Co-create solutions with the whole team. Refine the most important items as they get closer to implementation.

AI can write backlog items.

Only people working and learning together can decide whether those items describe a product worth building.

Last updated September 15th, 2026