Two Product Demos, One Question: Keep the Comparison Fair

One product demonstration shows a polished overview. The other shows a detailed workflow. A discussion then tries to compare the two as though the presenters had answered the same question.

Before ranking either explanation, it helps to notice the mismatch. A shared sample task can make the conversation more concrete without pretending to settle a purchase decision.

Key takeaways

• Compare the task shown, not only the polish of the presentation.

• Use the same harmless fictional scenario for both explanations.

• Avoid wording that tells the presenter which answer you want.

• A demo comparison is not a complete product evaluation.

Step 1: Give the scenario a goal

NN/g recommends realistic, actionable task scenarios that do not give away the route through an interface. For a fictional workshop, the goal might be "find the draft agenda and identify the next session." That is more useful than telling a presenter exactly which menu to open.

The sample needs enough context to be understood, but no real customer data or live transaction. It is an explanatory exercise, not a test of a production system.

Step 2: Keep the follow-up question neutral

"Why is this workflow easier?" assumes it is easier. "How would someone learn this workflow?" invites an explanation without ranking it in advance.

NN/g's guidance on leading questions explains why the wording matters. A question that supplies a conclusion can shape the answer. Keeping the language neutral leaves room for information the audience did not predict.

Step 3: Record the observation before the preference

A fictional note might say "Presenter A used a search field; Presenter B opened a saved view." That is an observation. "I preferred the search approach" is a separate statement about the viewer.

Both can be useful, but neither should masquerade as the other. Keeping them separate makes the discussion clearer when a colleague had a different preference.

The comparison can end with a question rather than a winner. For example: "Which route would make sense to the people who will use this?" Answering that calls for their context, not a conclusion based solely on presentation style.

Sources for the definitions and writing principles discussed here: Nielsen Norman Group, Turn User Goals into Task Scenarios for Usability Testing (January 12, 2014); Nielsen Norman Group, Avoid Leading Questions to Get Better Insights from Participants (December 17, 2017). The examples and suggested discussion formats are illustrative.

Frequently asked questions

Does the same task make two products fully comparable?

No. It makes one part of the discussion easier to compare. Products can differ in scope and intended audience.

Should the scenario use real business material?

It need not. Harmless invented content can explain a workflow without exposing real information.

Is a demo preference enough to choose a vendor?

No. This article concerns clarity of comparison, not purchasing or supplier-selection advice.

A practical next step

Before the next pair of demonstrations, write one ordinary sample task and one neutral follow-up question. Use the same wording for both. Book a Decision-Clarity Call for help making technology work easier to explain.