Usefulness — Does the AI Recommendation Help Users Take Action?

What Usefulness Measures

Usefulness measures whether a brand mention helps the user make a decision and take action. A high score means the model doesn't just name the brand — it provides specific information: products, pricing, advantages, and selection criteria.

This is a practical value metric: it answers the question "can a user choose my brand based on this response?"

When It Applies

Usefulness is calculated when both conditions are met:

Brand is mentioned in the response
Brand is eligible (Eligibility ≠ Not Eligible)

If the brand isn't mentioned or doesn't fit the query, the metric is not applicable (N/A).

Five Components of Usefulness

Specificity (0–2)

Whether specific products, features, pricing, or characteristics are mentioned:

Score	Description
2	Specific details — product names, prices, specifications
1	Partial specificity — general categories without details
0	Abstract mention without specifics

Actionability (0–2)

Whether the user can act on the response:

Score	Description
2	Clear next steps — what to do, where to go, how to try
1	Partial — direction is given but details are insufficient for action
0	None — mention doesn't help take concrete steps

Decision Criteria (0–2)

Whether the response helps compare options and make a choice:

Score	Description
2	Clear criteria — comparison with alternatives, advantages highlighted
1	Partial criteria — differences mentioned without full comparison
0	No criteria — brand named without context for choosing

Structure & Clarity (0–1)

How clearly and logically the information is presented:

Score	Description
1	Well-structured — lists, sections, logical flow
0	Disorganized — information scattered, hard to identify key points

Tradeoffs & Caveats (0–1)

Whether limitations, nuances, or honest caveats are included:

Score	Description
1	Limitations noted — where the brand may not fit, what the tradeoffs are
0	No caveats — only positive information

Counterintuitively, mentioning limitations increases the score: honest recommendations build more user trust.

Formula

Usefulness = 25 × (specificity / 2) + 25 × (actionability / 2)
           + 25 × (decision_criteria / 2)
           + 12.5 × (structure_clarity / 1) + 12.5 × (tradeoffs_caveats / 1)

Each 0–2 component contributes up to 25 points, each 0–1 component up to 12.5 points. Maximum score: 25 + 25 + 25 + 12.5 + 12.5 = 100.

Examples

Usefulness = 87.5 — High Usefulness

Specific products with pricing (specificity = 2), clear steps (actionability = 2), comparison criteria (decision_criteria = 2), well-structured (structure_clarity = 1), no caveats (tradeoffs_caveats = 0):

Usefulness = 25×(2/2) + 25×(2/2) + 25×(2/2) + 12.5×(1/1) + 12.5×(0/1)
           = 25 + 25 + 25 + 12.5 + 0 = 87.5

Usefulness = 37.5 — Low Usefulness

General mention (specificity = 1), no steps (actionability = 0), partial criteria (decision_criteria = 1), disorganized (structure_clarity = 0), caveats noted (tradeoffs_caveats = 1):

Usefulness = 25×(1/2) + 25×(0/2) + 25×(1/2) + 12.5×(0/1) + 12.5×(1/1)
           = 12.5 + 0 + 12.5 + 0 + 12.5 = 37.5

Dialog analysis panel — Usefulness section

Why Usefulness Matters

Recommendation usefulness bridges visibility and conversion. If the model mentions the brand prominently and relevantly but doesn't give the user enough information to act, the potential customer moves to a competitor that the model described in more detail.

Analyze Usefulness alongside:

Visibility — is the brand visible?
Relevance — is the mention appropriate?
Positioning Match — does the description match positioning?

This combination gives the full picture: the brand is not only visible and relevant but also helps the user choose.

Relevance — query match
Positioning Match — positioning alignment
Metrics System Overview — all metrics and Overall Score

Start evaluating recommendation usefulness

Usefulness — Does the AI Recommendation Help Users Take Action?

On this page