Third-Party Risk

A Better Vendor Reference Conversation Starts With a Neutral Question

A reference conversation is easier to learn from when the question does not suggest the answer. A small change in wording makes room for a fuller account.

A telephone handset, notebook and pen on a red background.

"You found the product easy to use, right?" sounds friendly. It also supplies the answer before the other person has spoken. A reference conversation can become a string of polite confirmations without anyone intending it.

There is a useful lesson here from user research. Nielsen Norman Group describes leading questions as questions whose wording includes or implies the answer the researcher expects. A reference call is not a usability study, but the wording problem is similar.

The following questions are suggestions for a clearer conversation. They are not a vendor-selection formula, a scorecard or a prediction of how a future relationship will turn out.

Key takeaways

  • A question can suggest its own answer without sounding pushy.

  • Ask for a description before asking for a judgment.

  • Keep the same neutral wording when comparing accounts.

  • Separate what the person said from your interpretation of it.

Step 1: Remove the adjective you hope to hear

Compare "How smooth was the setup?" with "What happened during setup?" The first invites a report about smoothness. The second leaves room for a different account, including one you did not anticipate.

NN/g recommends open-ended questions that do not suggest feelings or conclusions. Applying that idea here means asking what the person experienced before labelling the experience easy, difficult, successful or disappointing.

Step 2: Ask about an example, not a general reputation

A fictional prompt might be: "Think of a time you needed help. What did you do, and what happened next?" The listener can describe a sequence rather than choose a flattering or unflattering adjective.

There may be no relevant example, and that is a valid answer. Do not turn the absence of a story into evidence for or against the product. The question is an invitation to describe, not a demand to produce a particular kind of incident.

Step 3: Keep notes in two separate voices

One column can hold the account: "The team used email, then arranged a screen-sharing call." Another can hold your question: "Would our staff be comfortable with that format?" These statements do different work. The first reports what was said; the second is your interpretation.

Mark a paraphrase as a paraphrase. A memorable phrase should not become a quotation unless it is the speaker's actual wording. Clear notes preserve the distinction between the other person's experience and the listener's conclusions.

Step 4: Leave room for an answer outside the checklist

An ending such as "What have I not asked that would help me understand your experience?" can open space for a different topic. It does not guarantee a revelation. It simply stops the question list from defining everything that may be discussed.

If you speak to several people, the same neutral prompts make the notes easier to compare. Differences may reflect their circumstances rather than a contradiction. Preserve that context instead of forcing every answer into a single rating.

Sources for the definitions and writing principles discussed here: Nielsen Norman Group, Avoid Leading Questions to Get Better Insights from Participants (December 17, 2017). The examples and suggested discussion formats are illustrative.

The related discussion of clearer questions is worth your time alongside this one.

Frequently asked questions

Does neutral wording make the reference independent?

No. The way a reference was selected still matters. Better wording improves the conversation; it does not remove selection bias.

Can a yes-or-no question be useful?

Yes, for a narrow factual point. Follow it with a neutral invitation to explain when the context matters.

Is the approach a way to choose a supplier?

It is a way to ask and record clearer questions. A purchase or supplier decision needs its own context and should not rest on this conversation format alone.

A practical next step

Take one question from the next reference conversation and remove the answer hidden inside it. The aim is a fuller description, not a more flattering one. Book a Decision-Clarity Call for help turning a crowded technology discussion into clearer questions.

Next step

Get Board-Ready on AI and Cyber Risk

A 30-minute call. No charge. You'll leave with a clear read on where your oversight stands and what to do about it.

Stop Guessing. Start Scoring
Tyson Martin

Written by Tyson Martin, NACD.DC

Chief Trust, Security & AI Officer. CISO of record at MIO Partners; former CISO & CTO at Lumber Liquidators and Orvis; advisor to 200+ Fortune 500 boards and C-suites at AWS.

See the track record
Keep reading

More on Third-Party Risk

All CISO Insights
Third-Party Risk

When a Critical Vendor Gets Acquired

A vendor acquisition can change terms, roadmap and support overnight. Use a reassessment trigger list to review contract clauses, exit options and timeline.

Third-Party Risk

The Vendor Offboarding Checklist Nobody Runs

Contracts end but access and data often stay. Four steps to close out a vendor: revoke access, return or delete data, close the contract, record risk.

Third-Party Risk

When the AI Vendor Changes the Terms Mid-Contract

When AI vendor terms change, compare the notice with the signed agreement and deployed use. Check data, price, model behavior and available response options.