AI Co-Authored Code Increases Issues and Risks in Software Development

The research conducted by CodeRabbit focused on 470 GitHub pull requests, comparing AI co-authored submissions with those authored by humans. Out of these, 320 were AI-assisted, while 150 were human-written. The findings showed that AI-generated pull requests had an average of 10.83 issues, significantly higher than the 6.45 issues found in human submissions. This discrepancy indicates a heavier review load for teams, as AI-assisted changes often require more scrutiny before merging.

Notably, the severity of issues also increased with AI-assisted code. Critical issues rose from 240 in human code to 341 in AI co-authored code, marking a 40% increase. Major issues surged by approximately 70%, from 257 to 447. This escalation in both volume and severity necessitates closer inspection of AI-assisted changes to prevent defects from reaching production systems.

Logic and correctness issues were the most significant category of concern, with AI-generated code producing about 75% more findings in this area. Common problems included business logic errors and misconfigurations, which can lead to serious operational risks. Additionally, issues related to error handling and control flow were nearly doubled in AI submissions.

Readability and maintainability also suffered in AI co-authored code, with readability issues occurring more than three times as often. This can slow down review processes and complicate future maintenance, as formatting problems and naming inconsistencies were found at nearly triple the rate compared to human code.

Security-related findings were consistently higher in AI-assisted code, with overall security issues about 1.5 times more frequent. Improper password handling and insecure object references were particularly prevalent, increasing the risk of vulnerabilities in production environments. These findings underscore the importance of thorough reviews to catch potential security flaws.

While performance issues were less frequent, certain patterns in AI-generated code, such as excessive input-output operations, were observed to occur nearly eight times more often. This can lead to degraded system performance under load, complicating capacity planning for organizations.

Interestingly, some categories still showed higher rates in human-authored code, such as spelling errors and testability issues, reflecting the ongoing challenges faced by software teams regardless of the coding method.

Implications for Software Development Teams

The findings from CodeRabbit’s research reveal critical insights for software development teams utilizing AI coding tools. The increased volume and severity of issues in AI-assisted code necessitate enhanced review practices to mitigate risks. Organizations should consider implementing more rigorous testing and validation processes for AI-generated code to ensure reliability and security.

Furthermore, teams may need to invest in training and resources to better understand the unique challenges posed by AI-assisted development. By addressing these predictable weaknesses, organizations can improve their software quality and reduce the likelihood of defects impacting production systems.