The Engineering Cost of AI Defaults

Reviewing the output of a coding assistant does not make it safe. Review examines the existing declarations; the highest-leverage decisions inside a generated pipeline are the declarations that were never produced. Four lines of scikit-learn that import, fit, predict and return the right shape have already settled how much randomness each tree sees, how hard the coefficients are penalised, how the validation folds were cut, and what happens to a column holding no data. Every one of those settlements is absent from the diff and present in the result.

What an omitted argument commits you to

Python supplies an omitted parameter with the value declared on the class, and that value encodes a statistical or operational choice. The library documents it. The run does not report it. No alert triggers, no warning appears, and the output looks plausible, which is why the choice reaches production unexamined. Towards Data Science describes the mechanism by which "an underspecified request becomes an executable program", and notes that a request for code is not a conversation about assumptions: an assistant will examine defaults when prompted, and will not raise them unprompted. That last point is the one to hold on to. The tooling can answer. It does not volunteer.

The estimator defaults that decide the model

from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import LogisticRegression
from sklearn.cluster import KMeans

RandomForestRegressor()  # max_features=1.0
LogisticRegression()     # penalty="l2", C=1.0
KMeans()                 # init="k-means++", n_init="auto"

RandomForestRegressor defaults to max_features=1.0, making every feature available at every split, while RandomForestClassifier defaults to "sqrt". The gap removes the feature subsampling that distinguishes a random forest from plain bagging. The row bootstrap remains, so the trees stay random in the rows they receive, but no rule forces them apart in the splits they choose. One dominant predictor, price or discount in a sales model, keeps winning the early splits across every bootstrap sample, and trees that share structure share variance that averaging cannot remove. Setting max_features=0.33, roughly a third of the features per split, restores the decorrelation. The question for whoever generated the code is why a model chosen for feature-level randomness has that randomness disabled (Towards Data Science).

LogisticRegression defaults to an L2 penalty with C=1.0, so the fitted model is regularised whether that was the request or not, and 1.0 encodes no knowledge of the problem. The penalty is not scale-free, which is the part usually missed. Income expressed in euros with a coefficient of 0.0001 and the same income expressed in thousands of euros with a coefficient of 0.1 give identical predictions, because 50,000 × 0.0001 and 50 × 0.1 are both 5. The penalty squares the coefficient, so the second representation enters it a million times more strongly, and the same C now buys a different trade-off against the data. Standardise continuous features inside a Pipeline, so the scaler is fitted within each training fold, then tune C against validation. Otherwise the unit of measurement is helping to decide how much regularisation each feature receives.

KMeans defaults to init="k-means++" with n_init="auto", and under that initialisation the automatic setting means one complete run. One run is one basin of attraction. The algorithm places centres, assigns, updates and converges, and it can settle where further iterations never escape. n_init=10 repeats the whole procedure from ten starting positions and keeps the lowest inertia, the sum of squared distances from each point to its assigned centre. max_iter does not substitute for restarts, because it only permits more updates inside the run already chosen, and random_state=42 makes that single outcome reproducible without testing any alternative.

The defaults that decide the experiment

cross_val_score(model, X, y, cv=5) fixes the number of folds and leaves the construction of them to the library. Regression defaults to KFold with shuffle=False, so validation sets are consecutive row blocks in the order the dataframe already has. Sort by date, customer or region and that ordering has become part of the evaluation design. One hundred weeks of sales under cv=5 trains on weeks 21 to 100 and evaluates on weeks 1 to 20, using later observations to predict earlier ones: valid code, wrong experiment. shuffle=True mixes past and future, so the splitter must follow the specific meaning of unseen data in this context. Independent rows in arbitrary order take shuffled KFold. Forecasting takes a time-based split that trains on earlier observations and evaluates on later ones. Questions about new customers take a grouped splitter that keeps each customer entirely on one side of the boundary. Classification substitutes StratifiedKFold to hold class proportions, without shuffling and without respecting groups or time.

SimpleImputer defaults to strategy="mean" and removes any feature that was entirely missing during fitting, because a column with no values has no mean to compute. Twenty features in, nineteen out, no exception raised: The model sees nineteen at training and nineteen at prediction, so no downstream component notices. When the empty column later arrives populated, the fitted imputer still drops it and the new measurements never reach the model. keep_empty_features=True keeps the column and fills it with zero under the mean strategy, which preserves the shape but cannot teach a relationship the training fold never contained. Inspect the output feature names and shape after preprocessing.

Why the bill arrives after the throughput gain

Unit prices are falling and total spend is not. Benchmark price-performance for frontier models has improved by a factor of roughly 5 to 10 per year, and after controlling for hardware the MIT study published on arXiv attributes about 3 times per year of that to algorithmic efficiency. Cheaper tokens make a generated pipeline cheaper to accept and no cheaper to audit per line. Flat-rate assistant pricing has given way to token metering in the meantime, and the Forbes Technology Council reports that enterprise agentic coding can reach $2,000 per engineer per month, and that 62% of organisations in a 396-company survey recorded unexpected AI cost spikes that changed strategic decisions.

Quality tracks the other way. Cortex surveyed more than 50 engineering leaders and analysed development metrics for its 2026 benchmark report: pull requests per author up 20% year on year, incidents per pull request up 23.5%, change failure rates up about 30%, and formal AI usage policies in only 45% of organisations. DX measured more than 400 companies over fifteen months and found AI tool usage up 65% against a median pull request throughput gain of 8%, with most organisations in the 5 to 15% band and coding accounting for about 14% of a developer's day. Those are the figures a mis-specified validation fold corrupts, because a model trained on the wrong split returns a number that walks straight into the reporting.

Where human review has to attach

Not one of these defaults is a defect, and every library has reasons to ship them. The failure is in the process that accepts the code. The Home Office engineering standard for AI use explains the process requirement directly: AI-assisted outputs must be reviewed and approved by a human before production, AI-assisted changes must be traceable through commits, pull requests and reviews, and AI-assisted code must meet the same security expectations as code written without it. Reviewing a diff satisfies none of those on its own, because a default never appears in a diff. Traceability does. A commit or pull request marked as AI-assisted invites the reviewer to ask which arguments were omitted and what each omission decided.

Keeping a default on purpose

The remedy is one question asked before acceptance. Which arguments did the generator leave out, and what does each omission commit this pipeline to? Then three checks. Move the scaler inside the Pipeline, so units stop voting on regularisation strength. Pick the splitter from the generalisation target and state which of future rows, held-out customers or arbitrary rows the model must survive. Print the transformed feature names and shape after each preprocessing step; this lets a dropped column surface in development. Keep every default that fits the problem, and record why it fits.

Accepting generated code is not the act of confirming that it runs. It is the act of deciding the questions the code left open.

Sources

Source: Your AI Assistant Wrote the Code. Who Checked the Defaults?, Towards Data Science