Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cookbook for insecure code detection #1112

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

mhbuehler
Copy link

@mhbuehler mhbuehler commented Mar 19, 2024

Summary

This PR adds a cookbook and dataset for vulnerability detection proposed in #1100

Motivation

LLMs like GPT4 have shown proficiency in classifying code as secure or insecure. This notebook demonstrates prompts that improve classification accuracy from 67% to 80%, potentially helping developers with their secure coding practices. For more details, refer to this blog post. This use case demonstrates:

  • Using LLMs to identify and/or correct software vulnerabilities
  • Experimenting with multiple prompt techniques (zero-shot, few-shot, KNN few-shot) and measuring performance impacts
  • Using the OpenAI API to perform binary classification

For new content

  • I have added a new entry in registry.yaml (and, optionally, in authors.yaml) so that my content renders on the cookbook website.
  • I have conducted a self-review of my content based on the contribution guidelines:
    • Relevance: This content is related to building with OpenAI technologies and is useful to others.
    • Uniqueness: I have searched for related examples in the OpenAI Cookbook, and verified that my content offers new insights or unique information compared to existing documentation.
    • Spelling and Grammar: I have checked for spelling or grammatical mistakes.
    • Clarity: I have done a final read-through and verified that my submission is well-organized and easy to understand.
    • Correctness: The information I include is correct and all of my code executes successfully.
    • Completeness: I have explained everything fully, including all necessary references and citations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant