Skip to content

RedFlag uses AI to identify high-risk code changes. Run it in batch mode for release candidate testing or in CI pipelines to flag PRs and add reviewers. RedFlag's flexible configuration makes it valuable for any team.

License

Notifications You must be signed in to change notification settings

Addepar/RedFlag

Repository files navigation

RedFlag

Python Contributors MIT License LinkedIn
Forks Stargazers Issues

RedFlag leverages AI to determine high-risk code changes. Run it in batch mode to scope manual security testing of release candidates, or run it in your CI pipelines to flag PRs and add the appropriate reviewers. Despite being a security tool, RedFlag can be leveraged for almost any team as it's configuration makes it infinitely flexible.
Read the blog post »

View Sample Report · Report Bug · Request Feature

Table of Contents

Batch Mode

RedFlag is able to analyze a large number of commits in a single run. These commits can be specified using commit hashes, branch names, or tags. This is useful for scoping manual security testing of logical groups of code, such as release candidates.

Workflow

Batch Workflow

Getting Started

Installation

Use a Virtual Environment
  1. Create a virtual environment:

    python -m venv redflag-venv
    source redflag-venv/bin/activate
  2. Install RedFlag:

    pip install addepar-redflag

Alternatively, if you'd like to use Poetry, clone the repo and use poetry install and then poetry run redflag.

Setup Credentials

Credentials can be set using:

  1. Environment variables
  2. A .env file
  3. CLI parameters
  4. Configuration file (This is not recommended for security reasons!)
AWS Credentials

RedFlag uses Boto3-Compatible Credentials using Profiles or Env Vars. Ensure that your AWS IAM policy has InvokeModel and InvokeModelWithResponseStream permissions to Amazon Bedrock. Lastly, make sure you've requested the necessary Claude models!

GitHub PAT

Use a Personal Access Token with repo permissions. Set the token as an environment variable:

export RF_GITHUB_TOKEN=your-token-here
Jira API Token (Optional)

First, set a Jira URL (https://your-org.atlassian.net) in the configuration file (jira_url), as a CLI parameter (--jira-url), or as an environment variable (RF_JIRA_URL).

Then create a Jira API Token and set it as an environment variable:

export RF_JIRA_USER=your-username-here
export RF_JIRA_TOKEN=your-token-here
Slack Token (Optional)
  1. Create a Slack App
  2. Request required scopes
  3. Install and authorize the App
  4. Add environment variables or configuration entries when running RedFlag
export RF_SLACK_TOKEN=xoxb-slack-token-here
export RF_SLACK_CHANNEL=C0123456789

Don't forget to invite the bot to the channel to avoid a channel_not_found error.

Usage

Note: When running RedFlag, output messages will indicate it is retrieving and evaluating commits from Pull Requests (PRs). Typically, commits are made via PRs if you have branch protection or repository rulesets in place. Consequently, once a PR is squashed and merged, it appears as a single commit in the target branch. The terminology used may vary depending on how you choose to run RedFlag, as you can specify --from and --to commit SHAs or branches.

Here are some examples on how to run RedFlag in batch mode:

# Using branch names:
redflag --repo YourOrg/SomeRepo --from main --to dev
# Using commit hashes:
redflag --repo YouOrg/SomeRepo --from a1b2c3 --to d4e5f6
# With a custom configuration file:
redflag --config custom-config.yml

Report Output

By default, RedFlag produces an HTML report that can be opened in a browser.

(back to top)


CI Mode

RedFlag can be run in CI pipelines to flag PRs and add the appropriate reviewers. This mode uses GitHub Actions to run RedFlag on every PR and post a comment if the PR requires a review. Additionally, CI Mode is best suited for Slack alerting.

CI Mode

(back to top)


Evaluation Mode

RedFlag can be run in evaluation mode to evaluate the performance of the AI model using your own custom dataset. This mode is useful for understanding how the model and prompts perform on your codebase and aids in security risk evaluation.

Evaluation Mode

(back to top)


Advanced Configuration

Order of Precedence

  1. CLI Parameters
  2. Environment Variables
  3. Configuration File
  4. Default Values

On each execution, RedFlag will load the configuration in the order of precedence above and then output a table that shows the final configuration and where each parameter was set.

Configuration Options and Defaults

The following table shows configuration options for each parameter:

General Settings

Parameter CLI Param Env Var Config File Default
Configuration File --config - - -
Repository --repo RF_REPO repo -
Branch/Commit From --from RF_FROM from -
Branch/Commit To --to RF_TO to -

Integration Settings

Parameter CLI Param Env Var Config File Default
GitHub Token --github-token RF_GITHUB_TOKEN github_token -
Jira URL --jira-url RF_JIRA_URL jira.url -
Jira Username --jira-user RF_JIRA_USER jira.user -
Jira Token --jira-token RF_JIRA_TOKEN jira.token -
Slack Token --slack-token RF_SLACK_TOKEN slack.token -
Slack Channel (ID) --slack-channel RF_SLACK_CHANNEL slack.channel -
Slack Message Headline --slack-headline RF_SLACK_HEADLINE slack.headline -

LLM Settings

Parameter CLI Param Env Var Config File Default
Debug LLM --debug-llm - - False
Bedrock Model ID --bedrock-model-id RF_BEDROCK_MODEL_ID bedrock.model_id anthropic.claude-3-sonnet-20240229-v1:0
Bedrock Profile --bedrock-profile RF_BEDROCK_PROFILE bedrock.profile -
Bedrock Region --bedrock-region RF_BEDROCK_REGION bedrock.region -
Review Prompt (Role) - - prompts.review.role Security review (see sample.config.yaml)
Review Prompt (Question) - - prompts.review.question Security review (see sample.config.yaml)
Test Plan Prompt (Role) - - prompts.test_plan.role Security review (see sample.config.yaml)
Test Plan Prompt (Question) - - prompts.test_plan.question Security review (see sample.config.yaml)

Input/Output Settings

Parameter CLI Param Env Var Config File Default
Output Directory --output-dir RF_OUTPUT_DIR output_dir results
Maximum Commits --max-commits RF_MAX_COMMITS max_commits 0 (∞)
Don't Output HTML --no-output-html - - -
Don't Output JSON --no-output-json - - -
Don't Show Progress Bar --no-progress-bar - - -
Don't Strip HTML Comments --no-strip-html-comments - - -
Filter Commit Titles - - filter_commits.title -
Filter Commit Users - - filter_commits.user -
Strip Description Lines - - strip_description_lines -

Evaluation Parameters (eval Command)

Parameter CLI Param Env Var Config File Default
Evaluation Dataset --dataset RF_DATASET dataset -

(back to top)


Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

(back to top)


License

Distributed under the MIT License. See LICENSE.md for more information.

(back to top)


Contact

Say hi to Addepar Security Engineering at security-engineering@addepar.com.

(back to top)

About

RedFlag uses AI to identify high-risk code changes. Run it in batch mode for release candidate testing or in CI pipelines to flag PRs and add reviewers. RedFlag's flexible configuration makes it valuable for any team.

Topics

Resources

License

Stars

Watchers

Forks