Skip to content

Add Quark Script Agent in Document#664

Merged
haeter525 merged 4 commits into
ev-flow:masterfrom
pulorsok:master
Jul 27, 2024
Merged

Add Quark Script Agent in Document#664
haeter525 merged 4 commits into
ev-flow:masterfrom
pulorsok:master

Conversation

@pulorsok

@pulorsok pulorsok commented Jul 27, 2024

Copy link
Copy Markdown
Member

Quark Script Agent

Introducing Quark's new member, the Quark Script Agent, the first AI assistant in the Quark team. This agent enables users to perform analyses using natural language, without the need for programming or scripting expertise, making the process simple and user-friendly.

The Quark Script Agent integrates with LangChain, which utilizes OpenAI's large language models to act as a bridge between natural language and the Quark Script API. LangChain defines the Quark Script API as a tool that large language models can understand and use. This means that users can easily call new analysis APIs using natural language commands by simply adding new tools as needed.

Showcase: Detecting CWE-798 with Quark Script Agent

Here's an example of using the Quark Script Agent with the quarkScriptAgent.py. This agent can currently detect CWE-798 vulnerability in the ovaa.apk. See the details below.

Quick Start

  1. clone the repository:
git clone https://github.com/quark-engine/quark-script.git
  1. Install the required packages:
pip install -r requirements.txt
  1. Run the script:
python quarkScriptAgent.py
  1. Result:
截圖 2024-07-26 下午3 39 12

Decode the Prompts

Here are two prompts, each for executing different analysis processes.

1st Prompt: Initialize the rule instance with the rule path set to "rule.json"

Used Quark Script APIs/Tools that LLM used: loadRule

2nd Prompt: Run Quark Analysis using the rule instance on the apk sample "ovaa.apk", 
            and Check if the parameters are hard-coded. If yes, display the hard-coded values.

Used Quark Script APIs/Tools that LLM used: runQuarkAnalysis, getBehaviorOccurList, getParameterValues and isHardCoded

The loadRule, runQuarkAnalysis, getBehaviorOccurList, getParameterValues, and isHardCoded functions are treated as tools within LangChain, enabling them to be invoked through the gpt-4o model to analyze and identify CWE-798 vulnerabilities in the ovaa.apk sample.

截圖 2024-07-27 下午8 45 25
  • Notes:
    1. Since LangChain currently does not support passing Python instances between tools, we are temporarily using global variables to pass parameters between tools in quarkScriptAgent.py.
    2. Place the rules, samples, and quarkScriptAgent.py in the same folder; the LLM will automatically find files with matching names.
    3. A web GUI is under construction, please stay tuned!

@haeter525 haeter525 self-assigned this Jul 27, 2024
@haeter525 haeter525 self-requested a review July 27, 2024 12:08
@haeter525 haeter525 added documentation Improvements or additions to documentation pr-processing-state-05 labels Jul 27, 2024
Comment thread docs/source/quark_script_agent.rst Outdated
Comment thread docs/source/quark_script_agent.rst Outdated
Comment thread docs/source/quark_script_agent.rst Outdated
@codecov

codecov Bot commented Jul 27, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.73%. Comparing base (44c9973) to head (a66c67d).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #664   +/-   ##
=======================================
  Coverage   78.73%   78.73%           
=======================================
  Files          64       64           
  Lines        5324     5324           
=======================================
  Hits         4192     4192           
  Misses       1132     1132           
Flag Coverage Δ
unittests 78.73% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@haeter525 haeter525 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation pr-processing-state-05

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants