How Our Comparison and Scoring Methodology Works
A defensible comparison checks hard constraints first, uses independent criteria with anchored scores, labels estimates honestly, and tests whether realistic uncertainty changes the result.
Match each input to its evidence type
A comparison becomes misleading when a guess is formatted like a fact. Label the input before choosing a calculation.
Known fact
Use: a current price, written term, measured dimension, or other value that can be checked. Record the source and date.
Bounded estimate
Use: a low, expected, and high value when cost, time, or performance is uncertain but has a plausible range.
Future event
Use: scenarios or a decision tree when the outcome depends on something that has not happened and cannot yet be verified.
Criteria come before scores
A useful comparison starts with disqualifying constraints, then a short list of independent criteria. We avoid counting the same benefit under several names. Criteria should describe an outcome that matters to the person making the choice.
Use the decision criteria checklist to separate gates, consequences, and overlapping criteria before opening a scoring tool.
Weights describe preference
A weight is not a universal fact. It shows how important one criterion is for this decision. We recommend assigning weights before scoring the options and keeping the total visible.
Scores need anchors
Scores should use the same scale and meaning for both options. Users should be able to explain why a score changed. When evidence is weak, confidence should be marked as weak rather than hidden inside a precise decimal.
Close results get a sensitivity check
If a realistic change to one uncertain input flips the winner, the tool reports a close result. The next step is to investigate that input, run a small experiment, or choose based on reversibility.
Uncertainty changes the method
Known facts, bounded estimates, and future events are not interchangeable. A known fact can be verified. An estimate should use a range. An uncertain event belongs in a scenario, decision tree, or expected-value model only when the probability has a defensible basis. The decisions under uncertainty collection explains when each method fits.
Decision rules protect against moving the goalposts
A decision rule is written before the final result. It states which constraints must pass, which criterion controls a close result, and what downside is unacceptable. The rule prevents a preferred option from receiving new exceptions after scoring.
Product comparisons require current evidence
Prices, warranty terms, availability, software compatibility, and specifications can change. Pages should name the source and check date. Manufacturer claims must be distinguished from independent measurements and owner reports.
Polls describe participants
Public website polls are self-selected. Results are shown with the response count and a sample warning. We do not claim they represent the general public or attach a conventional margin of error. This follows the distinction explained by AAPOR's polling accuracy guidance.
Terms stay inspectable
The decision-making glossary defines the terms used by the tools and guides. A technical label should clarify the method, not make a subjective judgment appear scientific.