Skip to content

Conversation

spoiicy
Copy link
Member

@spoiicy spoiicy commented Jun 19, 2025

closes #2768

Description

This PR aims to support fetching details from CommandSequenceAPI from Greedybear, inside existing greedybear analyzer.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue).
  • New feature (non-breaking change which adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist

  • I have read and understood the rules about how to Contribute to this project
  • The pull request is for the branch develop
  • A new plugin (analyzer, connector, visualizer, playbook, pivot or ingestor) was added or changed, in which case:
    • I strictly followed the documentation "How to create a Plugin"
    • Usage file was updated. A link to the PR to the docs repo has been added as a comment here.
    • Advanced-Usage was updated (in case the plugin provides additional optional configuration). A link to the PR to the docs repo has been added as a comment here.
    • I have dumped the configuration from Django Admin using the dumpplugin command and added it in the project as a data migration. ("How to share a plugin with the community")
    • If a File analyzer was added and it supports a mimetype which is not already supported, you added a sample of that type inside the archive test_files.zip and you added the default tests for that mimetype in test_classes.py.
    • If you created a new analyzer and it is free (does not require any API key), please add it in the FREE_TO_USE_ANALYZERS playbook by following this guide.
    • Check if it could make sense to add that analyzer/connector to other freely available playbooks.
    • I have provided the resulting raw JSON of a finished analysis and a screenshot of the results.
    • If the plugin interacts with an external service, I have created an attribute called precisely url that contains this information. This is required for Health Checks (HEAD HTTP requests).
    • If the plugin requires mocked testing, _monkeypatch() was used in its class to apply the necessary decorators.
    • I have added that raw JSON sample to the MockUpResponse of the _monkeypatch() method. This serves us to provide a valid sample for testing.
    • I have created the corresponding DataModel for the new analyzer following the documentation
  • I have inserted the copyright banner at the start of the file: # This file is a part of IntelOwl https://github.com/intelowlproject/IntelOwl # See the file 'LICENSE' for copying permission.
  • Please avoid adding new libraries as requirements whenever it is possible. Use new libraries only if strictly needed to solve the issue you are working for. In case of doubt, ask a maintainer permission to use a specific library.
  • If external libraries/packages with restrictive licenses were added, they were added in the Legal Notice section.
  • Linters (Black, Flake, Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
  • I have added tests for the feature/bug I solved (see tests folder). All the tests (new and old ones) gave 0 errors.
  • If the GUI has been modified:
    • I have a provided a screenshot of the result in the PR.
    • I have created new frontend tests for the new component or updated existing ones.
  • After you had submitted the PR, if DeepSource, Django Doctors or other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.

Important Rules

  • If you miss to compile the Checklist properly, your PR won't be reviewed by the maintainers.
  • Everytime you make changes to the PR and you think the work is done, you should explicitly ask for a review by using GitHub's reviewing system detailed here.

@spoiicy spoiicy marked this pull request as ready for review June 19, 2025 12:55
@spoiicy spoiicy requested a review from fgibertoni June 19, 2025 12:55
@spoiicy
Copy link
Member Author

spoiicy commented Jun 19, 2025

JSON Results

{
  "report": {
    "enrichment_results": {
      "ioc": {
        "id": 675259,
        "asn": null,
        "name": "170.64.171.45",
        "type": "ip",
        "log4j": false,
        "cowrie": true,
        "scanner": true,
        "days_seen": [
          "2025-04-22",
          "2025-04-26",
          "2025-04-28",
          "2025-04-30",
          "2025-05-01",
          "2025-05-10",
          "2025-06-08",
          "2025-06-19"
        ],
        "last_seen": "2025-06-19T03:44:10.391000",
        "first_seen": "2025-04-22T21:14:59.247000",
        "related_ioc": [],
        "attack_count": 80,
        "ip_reputation": "",
        "login_attempts": 1292,
        "payload_request": false,
        "general_honeypot": [
          "Suricata"
        ],
        "destination_ports": [
          22
        ],
        "interaction_count": 7889,
        "number_of_days_seen": 8,
        "expected_interactions": 262.13378441837506,
        "recurrence_probability": 0.312159644782353
      },
      "found": true,
      "query": "170.64.171.45"
    },
    "command_sequence_results": {
      "detail": "No command sequences found for IP: 170.64.171.45"
    }
  },
  "data_model": null,
  "errors": [],
  "parameters": {
    "url": "https://greedybear.honeynet.org",
    "same_cluster_commands": false,
    "command_sequence_toggle": true
  }
}

@spoiicy spoiicy removed the request for review from fgibertoni June 19, 2025 12:56
@spoiicy spoiicy marked this pull request as draft June 19, 2025 13:41
@spoiicy spoiicy force-pushed the greedybear_update branch 2 times, most recently from 88de46a to 9644795 Compare June 23, 2025 08:12
@spoiicy spoiicy requested a review from mlodic June 23, 2025 13:54
@spoiicy spoiicy marked this pull request as ready for review June 23, 2025 13:54
@mlodic
Copy link
Member

mlodic commented Jun 23, 2025

could you do a screenshot with an example of an IP with a real command sequence? I dont remember whether you already have an account on the honeynet GB to catch this info

@spoiicy
Copy link
Member Author

spoiicy commented Jun 23, 2025

could you do a screenshot with an example of an IP with a real command sequence? I dont remember whether you already have an account on the honeynet GB to catch this info

I tried a dozen of IPs with commandSequenceAPI and i was just getting the same result : #2902 (comment), where you can see the
"command_sequence_results": { "detail": "No command sequences found for IP: 170.64.171.45" }

What do you think, should I try some more? Let me know

@spoiicy spoiicy requested a review from mlodic June 24, 2025 11:18
Copy link
Member

@mlodic mlodic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last tweaks and we are ready. I'll find an observable that you can test fully

command_sequence_uri = "/api/command_sequence"

result = {}
if get_hash_type(self.observable_name) == "sha-256":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I notice another problem. If an user analyze an md5 it would go through the else while, on the contrary, it should get an error message saying it is not supported. Please check the obs classification first and then check the has type

Copy link
Member Author

@spoiicy spoiicy Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier, I had handled the case, but then I realised, wouldn't this be taken care of by

run_hash=True,
run_hash_type="sha256",

in analyzer config, so I dropped the idea of handling it.

Let me know what you think

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

about that, that's another unecessary addition. run_hash and run_hash_type must be set only in case we want to run a file analyzer as an observable analyzer (case when we want to check the hash of a file).
So, no, this does not solve the problem. Also, please remove that configuration from the analyzer cause it is not needed.
You can try to analyze both an md5 and a sha256 to test this

enrichment_response = requests.get(
self.url + enrichment_uri, params=params_, headers=headers
)
result["enrichment_results"] = enrichment_response.json()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also about retrocomoatibility. Please keep old results in the same json hierarchy as they were before. Create a new hierarchy only for the new data

@spoiicy
Copy link
Member Author

spoiicy commented Jun 25, 2025

Successful CommandSequence Response with sha-256 hash

image

Unsuccessful Response when unsupported hash type like md5 is provided

image

@spoiicy
Copy link
Member Author

spoiicy commented Jun 25, 2025

@mlodic, I hope the implementation is fine according to you. Let me know if we have to write seperate test for this because i guess hash of another type is being passed which is raising the error and as a result default Test fails.

@spoiicy spoiicy requested a review from mlodic June 25, 2025 20:25
@mlodic
Copy link
Member

mlodic commented Jun 27, 2025

great!

description="Scan an IP, domain or a command sequence hash against the [GreedyBear](https://www.honeynet.org/2021/12/27/new-project-available-greedybear/) service",
observable_supported=["ip", "domain", "hash"],
run_hash=True,
run_hash_type="sha256",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned these values are not needed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll remove them.


if (
self.observable_classification == Classification.HASH
and get_hash_type(self.observable_name) != "sha-256"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this way you are running the get hash method 2 times. please do that once

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll make the intended changes.

and get_hash_type(self.observable_name) != "sha-256"
):
raise AnalyzerRunException(
"GreedyBear does not support hashes other than SHA-256."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

considering the tests are being reworked, for this case dont raise an analyzer exception for now, make the analyzer end with a success but add an error message to the output. in this way the test will work. not the recommended way but it is fine for now

Copy link
Member Author

@spoiicy spoiicy Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I pass the error in success response. When the refactoring of the testing suite is complete, I'll add the tests for this analzyer.

@spoiicy spoiicy requested a review from mlodic June 27, 2025 13:43
Copy link
Member

@mlodic mlodic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! just wait for the start of the next week so that I can provide an IP address to test

@mlodic mlodic linked an issue Jul 2, 2025 that may be closed by this pull request
@spoiicy
Copy link
Member Author

spoiicy commented Jul 5, 2025

Analyzer Result when command_sequence_toggle is set to True

{
  "report": {
    "enrichment_results": {
      "ioc": {
        "id": 602367,
        "asn": null,
        "name": "103.181.143.181",
        "type": "ip",
        "log4j": false,
        "cowrie": true,
        "scanner": true,
        "days_seen": [
          "2025-03-23",
          "2025-03-24"
        ],
        "last_seen": "2025-03-24T00:21:17.441000",
        "first_seen": "2025-03-23T23:31:00.217000",
        "related_ioc": [],
        "attack_count": 11,
        "ip_reputation": "",
        "login_attempts": 24,
        "payload_request": false,
        "general_honeypot": [
          "Suricata"
        ],
        "destination_ports": [
          22
        ],
        "interaction_count": 317,
        "number_of_days_seen": 2,
        "expected_interactions": 0,
        "recurrence_probability": 0
      },
      "found": true,
      "query": "103.181.143.181"
    },
    "command_sequence_results": {
      "license": "https://github.com/honeynet/GreedyBear/blob/main/FEEDS_LICENSE.md",
      "executed_by": [],
      "executed_commands": [
        {
          "time": "2025-03-23T23:59:55.697538",
          "command_sequence": "crontab -l\nw\nuname -m\ncat /proc/cpuinfo | grep model | grep name | wc -l\ntop\nuname\nuname -a\nwhoami\nlscpu | grep Model\ndf -h | head -n 2 | awk 'FNR == 2 {print $2;}'",
          "command_sequence_hash": null
        }
      ]
    }
  },
  "data_model": null,
  "errors": [],
  "parameters": {
    "url": "https://greedybear.honeynet.org",
    "command_sequence_toggle": true
  }
}

@spoiicy spoiicy force-pushed the greedybear_update branch 2 times, most recently from 1a581ba to ffd557d Compare July 8, 2025 08:54
@spoiicy spoiicy force-pushed the greedybear_update branch from ffd557d to 326d201 Compare July 13, 2025 13:43
@spoiicy spoiicy requested a review from mlodic July 13, 2025 14:50
@mlodic mlodic merged commit 72f3da4 into develop Jul 13, 2025
11 checks passed
@mlodic
Copy link
Member

mlodic commented Jul 13, 2025

great!

@fgibertoni fgibertoni deleted the greedybear_update branch July 14, 2025 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Greedybear Analyzer Commands Sequence API
2 participants