AI in the SDLC: Security Considerations When Developers Use Coding Assistants

Developer adoption of AI coding assistants has moved faster than most organisations' policies have kept up with. A survey of development teams in mid-2024 found that the majority of developers in organisations that had not formally adopted a coding assistant were using one anyway. The productivity gains are real and the tools are genuinely useful. The security risks are also real, and they are not the same as the risks introduced by developers making mistakes in their own code.

Security teams that have not yet addressed AI use in the software development life cycle are behind the curve. This is not about prohibiting a useful class of tool; it is about understanding the specific failure modes that AI-assisted code generation introduces and putting controls in place that are appropriate to those failure modes rather than carrying over controls designed for a different threat model.

AI-generated code inherits the insecure patterns present in training data

Coding assistants learn from large corpora of existing code. That code includes insecure patterns: SQL injection vulnerabilities, missing input validation, hardcoded credentials, insecure cryptographic implementations, and race conditions. When a developer accepts a code suggestion that includes one of these patterns, the result is a vulnerability in the codebase that is functionally identical to one the developer might have introduced themselves, but with an important difference: developers often have some awareness of the vulnerability classes they are prone to, whereas the AI suggestion may carry no visible signal that something is wrong.

Research on AI-generated code security has consistently found elevated rates of certain vulnerability classes compared to equivalent human-written code. The categories most commonly affected include memory safety issues, injection vulnerabilities, and insecure cryptographic choices. Static analysis tools catch some of these, but not all. The implication for security teams is that AI-generated code should be treated as requiring the same security review as any other code contribution, and that training for developers should address recognising insecure patterns in AI suggestions, not just avoiding them in original code.

Context leakage is a risk that coding assistants introduce at the IDE level

Most coding assistants work by sending context from the developer's current file, and often from related files in the project, to an external inference endpoint. That context may include secrets that have been written into configuration files, API keys present in the current file, internal system names, proprietary business logic, or personally identifiable information present in test data. The developer sending the request may not realise that adjacent file contents are being included in the context window.

Organisations need to assess what data is potentially in scope when developers use their chosen coding assistant, and whether the terms of service and data handling practices of that tool are acceptable given the sensitivity of the codebase. This assessment needs to include not just the code itself but the development environment: test fixtures containing production data, configuration files that reference internal infrastructure, and documentation that describes system architecture are all potential sources of sensitive context. At minimum, development workstations used for work on sensitive systems should have clear policies about which coding assistants may be used and under what conditions.

Hallucinated package names create supply chain risk

When a developer asks an AI coding assistant to suggest a library or dependency for a particular task, the assistant may suggest a package that does not exist. Developers who follow the suggestion and attempt to install the recommended package may find that an attacker has registered that package name and is serving malicious code to anyone who installs it. This attack pattern, sometimes called dependency confusion or package hallucination, is a direct consequence of the same hallucination behaviour that affects language models in other contexts.

Defending against this requires developer awareness and process controls. Developers should verify that a suggested package exists, is actively maintained, and has a credible provenance before adding it as a dependency. Software composition analysis tools that flag new dependencies for review before they enter the build pipeline add an additional layer of detection. The control is not complex, but it needs to be explicitly addressed in policy and developer training, because the natural flow of using a coding assistant is to accept suggestions and move on.

What a policy for AI-assisted development should cover

A policy governing AI coding assistant use in the development team does not need to be lengthy, but it does need to address the specific risks that these tools introduce. It should cover:

• Which coding assistants are approved for use, and on what categories of codebase
• Data handling requirements: what information may be sent to external inference endpoints
• The treatment of AI-generated code in code review: it is not exempt from security review because it was AI-generated
• Dependency verification requirements when a coding assistant suggests a new library
• Training data and IP considerations: the risk that proprietary code sent to a coding assistant may be used to improve the model
• Incident reporting obligations if a developer believes sensitive information has been exposed through a coding assistant session

If you want to assess your current SDLC security posture against the risks that AI-assisted development introduces, or if you need help developing a policy that development teams will actually follow, contact us at info@cyberlinx.com.au.