11 Code Review
Acknowledgements
Today we’ll be discussing best practices for reviewing AI-generated code. The guidelines here come mainly from the following sources:
Gemini Code Assist was also active during the development of these materials, and some text was generated with its assistance.
I’ve included some suggested prompts to help you evaluate AI-generated code. These are not exhaustive, but they can be a helpful starting point for your code reviewing. You can modify these prompts or come up with your own based on the specific context of the code you’re reviewing, but remember that you are the one who is accountable for the review.
Assumptions
Code generated by AI often looks good on the surface; assume it has issues or is otherwise incomplete
Edge cases are often not handled properly
AI doesn’t understand the intent behind the code, so it may not align with your goals
AI doesn’t understand security implications, so be cautious about vulnerabilities
AI explanations of what the code does can be helpful, but they may not be accurate or complete
Maintain accountability; take ownership and be sure you understand the code
Start with functional checks
- Does the code run without errors?
- Does it produce the expected output for typical inputs?
- Does it handle edge cases appropriately?
- Do the tests cover a wide range of inputs, including edge cases and potential failure points?
- Ask: does this code solve the right problem?
- Do you have a plan? If so, does the code align with your plan?
- “What functional tests to validate this code change do not exist or are missing?”
- “What possible vulnerabilities or security issues could this code introduce?”
Code quality
- Is the code readable and maintainable?
- Avoid accepting code that is difficult to understand, even if it works
- Does it use clear naming conventions?
- Is it well-organized and modular? (avoid monolithic functions)
Consistency and dependencies
- Does it fit in with the rest of the codebase?
- Avoid one-off solutions that don’t integrate well
- Avoid code that reinvents the wheel instead of using existing libraries or functions
- Does it use packages you are unfamiliar with? If so, do some research to understand them before accepting the code
Code comments
- Ensure the comments are accurate and helpful
- The code should be readable enough that it doesn’t require comments to understand what it does, but comments can provide valuable context and explanations
- Do comments describe what the code does, or why it does it that way? Both can be helpful, but the latter is often more valuable for maintainability
Optimization and performance
- Check for unneccesary recomputation or inefficient algorithms
- Check for any obvious security issues or vulnerabilities
- Check for any obvious performance issues
- Avoid premature optimization; focus on correctness and readability first
- “Examine this code in context with the provided plan. What assumptions about business logic, design preferences, or user behaviors have been made?”
- “How could this code be broken down into smaller, more testable units that follow the Single Responsibility Principle?”
Dependencies
- Check for any new dependencies introduced by the code
- Are they necessary? Could the same functionality be achieved with existing dependencies?
- Are they well-maintained and widely used, or are they obscure and potentially risky to rely on?
- Are there any potential conflicts with your project’s license?
- “Analyze and list all with their respective licenses.”
- “Are each of the dependencies actively maintained (that is, not archived and have recent maintainer activity)?”
AI-specific pitfalls
Hallucinated API’s and packages
AI training data may be outdated, so it may suggest deprecated functions or libraries
Watch for “slopsquatting” where the AI suggests a package with a similar name to a popular one, but it’s actually malicious or low-quality
AI will sometimes “fix” a failed test by deleting the test
AI doesn’t understand the difference between front-end and back-end code - pay close attention to interactions between the two
Linear logic is easy, but real code often has branching and edge cases that AI may not handle well
Complex requirements might get lost or misunderstood by the AI, leading to incomplete or incorrect code
- “Identify every API call, library method, and external parameter used in this code. For each one, verify it exists in the [Language/Framework] documentation for version [X]. List any that seem ‘custom’ or potentially non-existent.”
- “Examine the interaction between the frontend and backend. Identify where authentication happens. Does the backend re-verify the user’s role, or is it assuming the frontend already did the check?”
- “What critical edge cases or negative test scenarios (inputs that should fail) are currently missing from the generated tests? Write three new test cases that would likely break this implementation.”
Final thoughts
- Reviewing with a friend can be helpful for catching issues you might miss on your own
- Automate what you can, but don’t rely solely on automated tools for code review
- Keep learning and improving
- Be aware of your assumptions and remember that AI does not think like you do
Review Questions
- What should be your default assumption when reviewing code that appears functional on the surface?
- Beyond checking if the code runs, what are two specific functional checks you should perform regarding inputs?
- How should we handle the discovery of a new, unfamiliar package or dependency introduced by the AI agent?
- What is “slopsquatting,” and why is it a unique risk when using AI-suggested packages?
- When evaluating code comments, what type of information is generally considered more valuable for long-term maintainability than just “what” the code does?