WH Which, Honestly?
Comparison Methods

How to Build a Product Comparison Scorecard

Which, Honestly? · Updated 2026-07-19 · 4 min read
How to Build a Product Comparison Scorecard
SummaryBuild a product comparison scorecard by defining the product’s job, setting non-negotiable pass-or-fail requirements, and choosing five to eight clearly defined criteria. Assign weights that total 100 before researching, create an anchored scoring scale, and record the source, date, exact model, and evidence type behind every score. On a 0-to-5 scale, calculate each weighted contribution as score divided by 5 times the criterion weight, then add the results. Keep unknowns visible, preserve unrounded values until the end, and stress-test whether different reasonable weights or removed low-confidence claims change the winner.
Contents

A scorecard makes priorities visible, not objective

A product scorecard converts needs into criteria, evidence, weights, and a result. It does not remove judgment. The person choosing criteria and weights shapes the winner, so publish or record those choices instead of presenting a decimal as natural law.

Use scorecards for comparable products doing the same core job. Do not combine safety-critical, medical, legal, or financial suitability with ordinary preference unless qualified guidance defines non-negotiable requirements first.

Write the job and exclusions

Complete: “I need this product to ___ for ___ under ___ conditions.” Then state exclusions: incompatible platform, inaccessible control, prohibited material, size limit, safety certification, privacy requirement, or firm budget.

Treat a non-negotiable as pass/fail, not a low weighted score. A product that cannot fit the space does not become suitable by having excellent packaging.

Choose five to eight criteria

Useful criteria may include required function, total cost, dimensions, weight, repairability, warranty terms, accessibility, privacy, compatibility, maintenance, noise, energy use, portability, and support. Choose those relevant to the job.

Define each criterion before researching. “Easy to use” might mean setup time, readable controls, one-handed operation, or compatibility with assistive technology. Without a definition, the score measures atmosphere.

Browse Comparison Methods for more evidence checks.

Set weights that total 100

Assign importance as percentages that sum exactly to 100. Example:

Check the arithmetic: 30 + 25 + 20 + 15 + 10 = 100.

Do this before reading reviews, when possible. Otherwise a product you already fancy may mysteriously receive a judging system tailored like a small ceremonial suit.

Define the scoring scale

Use a simple scale such as 0 to 5 and anchor it:

Describe 2 and 4 as intermediate. Mark unknown as unknown, not zero and not three. Missing evidence should lower confidence and trigger research, demonstration, or a question.

Build an evidence column

For every score, record the source, date checked, exact model or version, and whether the information is a manufacturer specification, seller term, independent measured test, personal observation, owner report, or editorial judgment.

Open the original source and confirm the claim. Current model pages, manuals, warranty documents, privacy terms, accessibility documentation, and safety notices can change. A real link does not support a number it never states.

Never label borrowed review impressions as hands-on experience. If you did not test the product, say so. If a reviewer received a sample or uses affiliate links, note the disclosure when it affects your evidence assessment.

Calculate weighted scores transparently

For a 0-to-5 scale, compute each contribution:

weighted contribution = score ÷ 5 × criterion weight

If a product scores 4 on a criterion weighted 25:

4 ÷ 5 × 25 = 20

Add contributions for a total out of 100. Keep unrounded values until the final total. If criteria are unknown, either leave the total incomplete or show a confidence warning; do not quietly treat unknowns as average.

Stress-test the winner

Change the most debatable weight and see whether the result flips. Remove a low-confidence claim. Recalculate for another user’s priorities. A close result means the products are close under this model, not that one is universally best.

Try the method on paper planners versus task apps. End with who each option fits, major drawbacks, unknowns, and the date checked. “Best overall” is rarely a fact. It is usually a scorecard with its name tag hidden.

FAQ

How many criteria should a product scorecard use?

Five to eight usually keeps an ordinary decision understandable, but the right number depends on the product and risks. Combine overlapping criteria and define each one precisely. Non-negotiable safety, compatibility, access, or size requirements should be pass-or-fail rather than diluted among preferences. More criteria do not automatically create more rigor; they can count the same concern repeatedly.

Should product-comparison weights add to 100?

Yes, percentages totaling 100 make relative importance and weighted totals easy to audit. Check the arithmetic before scoring. Another valid method can use points or pairwise rankings, but explain it. Set weights before seeing results when possible, and run an alternative weighting to reveal whether the conclusion is robust or depends on one debatable priority.

How do I score a criterion with missing information?

Mark it unknown and investigate through current primary documentation, a demonstration, a direct seller or manufacturer question, or reliable independent testing. Do not turn unknown into zero unless the rule explicitly treats absence of evidence as failure, and do not assign an average. Show incomplete totals or a confidence warning so missing facts remain visible in the conclusion.

Can manufacturer specifications be trusted in a comparison?

Use current manufacturer documents for stated dimensions, compatibility, included parts, instructions, and warranty terms, but read the exact model and region and note the date. Marketing language may not answer real performance questions. Pair specifications with appropriate independent measured testing or clearly labeled observation. Never stretch a document to support a claim it does not make or imply personal testing you did not perform.