Coding The Brains · Published 8 September 2026
Data-quality audit before a predictive model
Before commissioning a predictive model, check whether your data describes the decision you want to make, is consistent across sources, and can be evaluated without information from the future. A model can produce a precise-looking number from a dataset that does not support a useful decision.
Our published used-car price prediction project connects data collection, cleaning, a K-nearest-neighbors model and a web application. This walkthrough uses a separate, deliberately flawed synthetic dataset to make the data checks reproducible. It does not expose project data or report that project's model accuracy.
1. Agree on what one row and one prediction mean
For a used-car example, a row might represent a listing observed on a particular date. An asking price is not a completed sale price. If the business needs an estimate of eventual transaction value, training against advertised prices answers a different question. Write down the target, prediction time, currency, mileage unit, source and intended user before collecting more records.
Ask the same question for other projects: is a customer record a person, an account or a subscription? Does “revenue” include refunds? Disagreement over these definitions will survive a dashboard redesign.
2. Reconcile sources before judging a model
Start with row counts, identifiers, missing values, units, plausible ranges and timestamps. Keep rejected records with reasons rather than silently dropping them. Duplicate detection should reflect the business key: two observations of the same vehicle on different dates are not necessarily duplicates.
| Check | Why it matters | Decision to document |
|---|---|---|
| Missing target | A missing price cannot serve as a known training label. | Exclude from supervised training; retain separately where useful. |
| Duplicate identity | Repeated records can overweight a listing or cross evaluation boundaries. | Choose a source and observation-time rule. |
| Units and currency | Miles and kilometers are not interchangeable. | Preserve original values and record every conversion. |
| Observation time | Later information can leak into an earlier prediction. | Define what was available at prediction time. |
| Coverage | A clean sample can still exclude the market you intend to serve. | Compare relevant segments and expose unsupported cases. |
3. Run the small, inspectable example
Download the standalone JavaScript audit. It contains eight synthetic records, uses no packages, makes no network requests and can be read before execution. Run it locally with Node.js:
node used-car-data-audit.mjsThe included fixture produces: 8 rows, 1 duplicate ID, 1 missing price, 1 invalid price, 1 invalid mileage value, 1 invalid date and 3 accepted rows. Counts may overlap on other inputs. “Accepted” means only that these elementary checks passed; it does not mean a record is representative or suitable for a model.
This is a teaching example, not a general CSV importer or a production validation library. Adapt the key, date rules and ranges to your sources. Do not send customer records to an unfamiliar online validator.
4. Design evaluation before feature engineering
Keep a genuinely held-out evaluation set. Where records repeat across entities, consider grouping related observations. Where you are predicting future outcomes, consider a time-based split. Fit transformations on training data, then apply them to evaluation data; scikit-learn's common pitfalls guide explains why fitting preprocessing on the full dataset leaks information.
Compare a candidate model with a simple baseline that matches the decision, such as a segment median for the illustrative pricing task. Inspect errors by relevant segment and set an abstention rule for unsupported inputs. The strongest average score may conceal errors that make a particular workflow unusable.
To inspect that tradeoff, run our predictive-model baseline example. It calculates aggregate and segment-level errors from explicitly synthetic predictions and shows what an evaluation handoff should contain.
Check evaluation leakage with a second runnable fixture
Updated 5 October 2026: added the split-audit fixture and blank handoff sheet.
A row can pass elementary cleaning checks and still make an evaluation misleading. Download the prediction-split audit and run node prediction-split-audit.mjs. This separate four-record synthetic fixture reports one entity shared by training and evaluation (vehicle-A) and one feature available after its prediction timestamp (eval-2). It does not fit a model or measure accuracy.
| Signal | Question to resolve | What not to assume |
|---|---|---|
| Shared entity | Will production predict unseen entities, or future observations of known entities? | Overlap is not automatically leakage. The split must match the intended deployment question. |
| Feature arrives later | Was this exact value available when the prediction would have been made? | A field existing in today's database does not establish historical availability. |
| Invalid timestamp | Can the observation and availability times be parsed and reconciled? | An unparseable date must not silently pass the time check. |
| Learned preprocessing | Were imputation, scaling and feature selection fitted only on training observations? | A clean split cannot fix a transformation fitted on the full dataset. |
The script flags overlap, future features and invalid timestamps. It does not inspect your preprocessing pipeline, label provenance, permissions or distribution shift. Its function accepts explicit training and evaluation arrays; adapt the contract locally and never paste private records into an unfamiliar service. For Python implementations, consult scikit-learn's cross-validation guide and preprocessing leakage examples.
5. Commission an audit with a concrete handoff
Download the 16-check data-audit handoff sheet. Evidence, owner, status and decision are deliberately blank. Complete those fields for your actual dataset; downloading a template is not a passed audit.
Build, narrow the scope, or collect better data?
Proceed to a bounded baseline when the target and prediction moment are defined, the necessary features exist at that moment, and a representative evaluation is possible. Narrow the scope when evidence supports only specific segments or a smaller decision. Improve collection first when labels, permissions or observation history cannot support the requested outcome. These are evidence-based decisions, not universal minimum row counts: there is no single dataset size that guarantees a useful model.
Ask for the rejected-row reasons, the split rule, a simple baseline and the unsupported-input policy before comparing complex models. A useful handoff explains unresolved risks as clearly as the accepted records.
A useful data audit should leave you with source definitions, a repeatable validation script, rejected-row reasons, coverage gaps, an evaluation plan and a recommendation: build, narrow the scope, or improve collection first. Agree on an owner and refresh schedule for these checks.