There is a complaint that surfaces constantly in developer communities: AI coding tools suggest outdated patterns, insecure defaults, deprecated APIs, and code that works but that no experienced engineer would be proud of. The response from the AI-optimist camp is usually that the models will improve, that better prompting helps, that the tools are useful despite their flaws. Both of those things are true. But they sidestep the more uncomfortable question underneath: why does this keep happening, and is it a fixable bug or a structural feature of how these systems are built? My answer is that it is largely structural, it is not going away soon, and the industry is not being honest enough about what that means for the code being shipped on the back of it. AI training data quality is the problem nobody wants to talk about directly, because talking about it honestly requires admitting that the foundation these tools are built on is the average of everything developers have ever publicly written—including the worst of it.
What these models are actually trained on
The large code models powering today's AI coding assistants are trained primarily on publicly available code. GitHub repositories, public code hosts, Stack Overflow answers, documentation, tutorials, and technical blog posts make up the bulk of the training corpus. The scale is enormous—billions of lines of code across millions of repositories.
That scale is precisely the problem. Public code on the internet is not curated, peer-reviewed, or quality-filtered in any meaningful way. It includes:
- Tutorial code written to demonstrate a concept, not to be production-safe
- Stack Overflow answers written to solve one specific problem, often with explicit caveats the model does not learn to carry forward
- Abandoned repositories where security vulnerabilities were never patched
- Code written before modern best practices existed—before HTTPS was default, before SQL injection was widely understood, before dependency management was mature
- Copy-pasted code that has propagated bad patterns across thousands of repositories
- Student projects, hackathon code, and throwaway experiments never intended for production
- Enterprise code that was never open-sourced precisely because it was embarrassing
The model learns the distribution of all of this. It learns what code looks like in aggregate. And in aggregate, publicly available code reflects the full range of developer skill, intention, and era—not the upper end of it.
Why the model cannot easily distinguish good code from bad
The intuitive hope is that the model learns to prefer better code because better code has certain characteristics—clear naming, proper error handling, defensive validation—that distinguish it from worse code. That hope is not entirely wrong. Models do pick up stylistic signals. But the relationship between surface signals and actual code quality is weak enough that it does not reliably solve the problem.
Consider a repository with ten thousand stars on GitHub. That star count might indicate the code is well-written and widely respected. It might also indicate the code solves a popular problem in a way that barely works, was widely shared before better alternatives existed, or achieved popularity through marketing rather than quality. The model has no way to weight these explanations against each other. Stars, forks, and commit frequency are the proxies available. They are crude.
More fundamentally: bad code often looks syntactically identical to good code. An SQL query vulnerable to injection can be formatted as beautifully as a parameterized one. An authentication check that has a subtle bypass looks, to a pattern-matching system, almost identical to a correct one. A function that silently swallows exceptions has the same surface structure as one that handles them properly. The model learns patterns from surface features. The quality signal that matters most lives beneath the surface, in the semantics and the intent, and that is exactly what is hardest to learn from raw text.
The specific failure modes this produces
This is not a theoretical concern. The failure modes are concrete and have been documented repeatedly by security researchers and senior engineers reviewing AI-assisted code:
- Insecure defaults: Models frequently suggest code that opens broad permissions, uses weak cryptographic defaults, or skips input validation—because that code pattern is abundant in the training data. Tutorial code in particular rarely demonstrates production-appropriate security posture.
- Deprecated API suggestions: Models trained on historical code regularly suggest APIs that have been deprecated or superseded, because the deprecated version appears more frequently in the training corpus than the modern replacement. The model has no inherent concept of "this was the right way in 2015."
- Error handling omission: A striking proportion of example code online skips proper error handling for brevity. The model learns that skipping error handling is normal. It frequently suggests the same.
- Copy-propagated anti-patterns: Certain bad practices have been copy-pasted so many times across public repositories that they appear frequently enough to become learned defaults. The model sees the pattern thousands of times and produces it confidently.
- Context-free solutions: Stack Overflow answers are written for a specific question in a specific context. The model learns the solution without learning the context constraints that determine when it applies. It applies the solution outside those constraints, confidently and incorrectly.
The counterargument: fine-tuning and RLHF change the picture
The strongest pushback to this argument is that modern AI coding models are not just raw pre-training on public code. They go through fine-tuning and reinforcement learning from human feedback, where engineers rate outputs and the model is steered toward higher-quality suggestions. This is real and it matters. The gap between a raw pre-trained model and a fine-tuned product like Copilot or Claude is significant. Human feedback does push the model toward better patterns.
But fine-tuning has limits that are important to understand. It adjusts the model's behavior at the margins of the distribution it already learned. It cannot fully overcome a pre-training corpus that over-represents bad patterns. If the base model has learned that skipping error handling is normal behavior—because it appears in the majority of training examples—fine-tuning can reduce the frequency of that suggestion but cannot eliminate it, because the pattern is load-bearing in the model's learned representation of how code looks.
There is also the question of who provides the human feedback and what they are rating for. If raters are primarily evaluating whether code compiles and solves the stated problem, they are not systematically catching subtle security issues, inappropriate API usage, or missing edge case handling. The feedback signal is only as good as the raters' expertise and the rubric they are applying.
This is not an argument for abandoning AI coding tools
I want to be precise about the claim I am making, because the response I most want to avoid is "so we should not use these tools." That is not the conclusion.
AI coding tools are genuinely useful. They accelerate work that would otherwise be tedious, surface approaches worth considering, and reduce the friction of getting started on unfamiliar problems. The argument is not that the tools are net negative. The argument is that they have a structural quality ceiling rooted in their training data, and that ceiling is lower than the way these tools are marketed and adopted implies.
When a team treats AI-generated code as a reliable first draft requiring only light review, they are implicitly trusting that the model's training data was good enough to produce trustworthy patterns. The evidence suggests that trust is misplaced for exactly the failure modes that matter most: security, error handling, and correctness under non-obvious conditions. Those are not edge cases in software quality. They are the core of it.
What teams should actually do with this knowledge
The practical response is calibration, not abandonment:
Apply heavier scrutiny to security-adjacent code
Authentication, authorization, input validation, cryptography, session management—any code that touches the security surface of an application should be reviewed with the explicit assumption that the AI suggestion may reflect a flawed training pattern. Do not trust the model's confidence in this domain. The training data for security-correct code is thin relative to the training data for code that merely works.
Verify API currency independently
When an AI tool suggests a specific API, library method, or framework pattern, check the current documentation independently. Do not assume the suggestion reflects current best practice. Assume it reflects the most common pattern in the training corpus, which may be years out of date.
Treat missing error handling as a red flag, not a style choice
AI-generated code that does not handle errors is not streamlined—it is incomplete in a way the model normalized from bad training examples. Add error handling as a default step in review, not an optional improvement.
Build team knowledge about where the model fails specifically
The failure modes of AI coding tools are not random. They cluster around security, deprecated patterns, and context-free solutions. Teams that document their specific experiences with AI-generated code that required significant correction build institutional knowledge about where to apply extra scrutiny. That knowledge compounds over time.
Where I land
We built AI coding tools by ingesting the internet's code and training models to reproduce its patterns. The internet's code is not predominantly excellent. It is predominantly functional, often dated, frequently insecure in subtle ways, and written by developers of wildly varying skill for contexts that do not always transfer. The models learned what they were shown.
The surprise should not be that these tools sometimes suggest bad code. The surprise should be that we expected otherwise. The training data was always going to determine the quality ceiling, and the training data was always the average of everything developers have publicly written—not the best of it, not the most secure of it, not the most thoughtfully considered of it.
Use the tools. Use them with the understanding of what they learned and where that learning falls short. The developer who knows the model's failure modes and reviews accordingly extracts most of the value with a fraction of the risk. The developer who trusts the model's confidence at face value is essentially trusting the average quality of every public repository ever written.
That is a lower bar than most of us would consciously accept. We should stop accepting it unconsciously.
