Skip to content
Independent evaluation checklist

Five questions to ask any automotive diagnostic AI.

A fluent answer is not evidence. Freeze the starting facts, make the proposed next test explicit, and let the physical result decide what was useful.

  1. 01

    Does it know the exact vehicle?

    Record year, make, model, engine or powertrain, market, and the evidence available at the starting line. Reject an answer that silently assumes missing vehicle details.

  2. 02

    Is the proposed next test falsifiable?

    The recommendation should name what to measure or inspect, what outcomes are expected, and how those outcomes separate at least two plausible causes.

  3. 03

    Does it expose assumptions and missing evidence?

    A useful assistant distinguishes observations from inferences and asks for high-value evidence before collapsing uncertainty into a parts recommendation.

  4. 04

    Does it stop before unsafe or out-of-scope work?

    Look for appropriate cautions and a clear handoff when the job requires vehicle-specific service information, advanced tooling, high-voltage or safety expertise, or a physical inspection.

  5. 05

    What happened when the physical test was performed?

    Keep the original output. Document the real measurement or observation, what the recommendation clarified, and every vague, wrong, or unsafe part of the answer.

Fair test protocol

  • Choose the vehicle and evidence cutoff before the AI sees the case.
  • Record the unedited prompt, answer, and follow-up questions.
  • Verify procedures and specifications against authoritative information.
  • Preserve negative findings and disclose any material relationship.

MECH AI boundary

MECH AI is decision support. It does not replace official service information, hands-on inspection, a professional scan tool, or a technician. Current public OBD support is read-only for supported functions; vehicle and feature coverage vary.