Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ailtyposquatting #2341

Merged
merged 20 commits into from
May 31, 2024
Merged

ailtyposquatting #2341

merged 20 commits into from
May 31, 2024

Conversation

g4ze
Copy link
Contributor

@g4ze g4ze commented May 26, 2024

closes #1545

Description

Please include a summary of the change and link to the related issue.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue).
  • New feature (non-breaking change which adds functionality).
  • Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist

  • I have read and understood the rules about how to Contribute to this project
  • The pull request is for the branch develop
  • A new plugin (analyzer, connector, visualizer, playbook, pivot or ingestor) was added or changed, in which case:
    • I strictly followed the documentation "How to create a Plugin"
    • Usage file was updated.
    • Advanced-Usage was updated (in case the plugin provides additional optional configuration).
    • I have dumped the configuration from Django Admin using the dumpplugin command and added it in the project as a data migration. ("How to share a plugin with the community")
    • If a File analyzer was added and it supports a mimetype which is not already supported, you added a sample of that type inside the archive test_files.zip and you added the default tests for that mimetype in test_classes.py.
    • If you created a new analyzer and it is free (does not require any API key), please add it in the FREE_TO_USE_ANALYZERS playbook by following this guide.
    • Check if it could make sense to add that analyzer/connector to other freely available playbooks.
    • I have provided the resulting raw JSON of a finished analysis and a screenshot of the results.
    • If the plugin interacts with an external service, I have created an attribute called precisely url that contains this information. This is required for Health Checks.
    • If the plugin requires mocked testing, _monkeypatch() was used in its class to apply the necessary decorators.
    • I have added that raw JSON sample to the MockUpResponse of the _monkeypatch() method. This serves us to provide a valid sample for testing.
  • If external libraries/packages with restrictive licenses were used, they were added in the Legal Notice section.
  • Linters (Black, Flake, Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
  • I have added tests for the feature/bug I solved (see tests folder). All the tests (new and old ones) gave 0 errors.
  • If changes were made to an existing model/serializer/view, the docs were updated and regenerated (check CONTRIBUTE.md).
  • If the GUI has been modified:
    • I have a provided a screenshot of the result in the PR.
    • I have created new frontend tests for the new component or updated existing ones.
  • After you had submitted the PR, if DeepSource, Django Doctors or other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.

Important Rules

  • If you miss to compile the Checklist properly, your PR won't be reviewed by the maintainers.
  • Everytime you make changes to the PR and you think the work is done, you should explicitly ask for a review. After being reviewed and received a "change request", you should explicitly ask for a review again once you have made the requested changes.
    image
    "report": {
    "addDash": [
    "g-oogle.com",
    "go-ogle.com",
    "goo-gle.com",
    "goog-le.com",
    "googl-e.com"
    ],
    "omission": [
    "oogle.com",
    "gogle.com",
    "goole.com",
    "googe.com",
    "googl.com"
    ],
    "subdomain": [
    "g.oogle.com",
    "go.ogle.com",
    "goo.gle.com",
    "goog.le.com",
    "googl.e.com"
    ]
    },}

@g4ze
Copy link
Contributor Author

g4ze commented May 26, 2024

migrations can be changed if other analyzers are merged before. kindly review the wrapper and associated params...
@mlodic @0ssigeno

@g4ze g4ze requested review from mlodic and 0ssigeno May 26, 2024 01:03
@g4ze g4ze linked an issue May 26, 2024 that may be closed by this pull request
@g4ze
Copy link
Contributor Author

g4ze commented May 27, 2024

image
i'm getting permission error for file management, am I missing something here?

@mlodic
Copy link
Member

mlodic commented May 27, 2024

is there a way to avoid writing into the disk? I mean, you already get the results back from the functions you call, right? If you set the path to the written to file to None what happens?

@g4ze
Copy link
Contributor Author

g4ze commented May 27, 2024

tried that. works now :p

@g4ze g4ze requested a review from mlodic May 27, 2024 16:25
@g4ze
Copy link
Contributor Author

g4ze commented May 27, 2024

image

formatoutput="yara",
pathOutput=None,
)
if self._job.tlp == "CLEAR" and self.dns_resolving:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the enum for the compare? self._job.TLP.CLEAR.value

and dns resolving {self.dns_resolving}"""
)
resultList = []
response["algorithms"] = runAll(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please show the results from this? cause "formatoutput"="yara" I don't think it is the right option

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Member

@mlodic mlodic May 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I mean it does not make sense to me cause I should have expected "Yara" code output. But ok, nevermind, the important thing is that it works. Maybe Yara would be used in the file written to the disk be but we are not generating it.
So, it could make sense to set it as None too to avoid confusion to the next reader

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

says : known format: None. Will use text format instead
committing with text only.

pathOutput=None,
)
if self._job.tlp == "CLEAR" and self.dns_resolving:
response["dnsResolving"] = dnsResolving(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also results from this plz

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
give nothing here...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a problem cause I would have expected a populated answer. For instance, xx.com does exist.

The output should be like this: https://github.com/typosquatter/ail-typo-squatting?tab=readme-ov-file#dns-output Can you check the library closely pls?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Figuring out the problem took quite some time. seems to work fine now. It just takes awfully lot of time.
attching a part of the output


{'k-com.xn--ogbpf8fl': {'NotExist': True}, 'k-com.xn--otu796d': {'NotExist': True}, 'k-com.xn--p1acf': {'NotExist': True}, 'k-com.xn--p1ai': {'NotExist': True}, 'k-com.xn--pgbs0dh': {'NotExist': True}, 'k-com.xn--pssy2u': {'NotExist': True}, 'k-com.xn--q7ce6a': {'NotExist': True}, 'k-com.xn--q9jyb4c': {'NotExist': True}, 'k-com.xn--qcka1pmc': {'NotExist': True}, 'k-com.xn--qxa6a': {'NotExist': True}, 'k-com.xn--qxam': {'NotExist': True}, 'k-com.xn--rhqv96g': {'NotExist': True}, 'k-com.xn--rovu88b': {'NotExist': True}, 'k-com.xn--rvc1e0am3e': {'NotExist': True}, 'k-com.xn--s9brj9c': {'NotExist': True}, 'k-com.xn--ses554g': {'NotExist': True}, 'k-com.xn--t60b56a': {'NotExist': True}, 'k-com.xn--tckwe': {'NotExist': True}, 'k-com.xn--tiq49xqyj': {'NotExist': True}, 'k-com.xn--unup4y': {'NotExist': True}, 'k-com.xn--vermgensberater-ctb': {'NotExist': True}, 'k-com.xn--vermgensberatung-pwb': {'NotExist': True}, 'k-com.xn--vhquv': {'NotExist': True}, 'k-com.xn--vuq861b': {'NotExist': True}, 'k-com.xn--w4r85el8fhu5dnra': {'NotExist': True}, 'k-com.xn--w4rs40l': {'NotExist': True}, 'k-com.xn--wgbh1c': {'NotExist': True}, 'k-com.xn--wgbl6a': {'NotExist': True}, 'k-com.xn--xhq521b': {'NotExist': True}, 'k-com.xn--xkc2al3hye2a': {'NotExist': True}, 'k-com.xn--xkc2dl3a5ee0h': {'NotExist': True}, 'k-com.xn--y9a3aq': {'NotExist': True}, 'k-com.xn--yfro4i67o': {'NotExist': True}, 'k-com.xn--ygbi2ammx': {'NotExist': True}, 'k-com.xn--zfr164b': {'NotExist': True}, 'k-com.xxx': {'NotExist': True}, 'k-com.xyz': {'NotExist': True}, 'k-com.yachts': {'NotExist': True}, 'k-com.yahoo': {'NotExist': True}, 'k-com.yamaxun': {'NotExist': True}, 'k-com.yandex': {'NotExist': True}, 'k-com.ye': {'NotExist': True}, 'k-com.yodobashi': {'NotExist': True}, 'k-com.yoga': {'NotExist': True}, 'k-com.yokohama': {'NotExist': True}, 'k-com.you': {'NotExist': True}, 'k-com.youtube': {'NotExist': True}, 'k-com.yt': {'NotExist': True}, 'k-com.yun': {'NotExist': True}, 'k-com.za': {'NotExist': True}, 'k-com.zappos': {'NotExist': True}, 'k-com.zara': {'NotExist': True}, 'k-com.zero': {'NotExist': True}, 'k-com.zip': {'NotExist': True}, 'k-com.zm': {'NotExist': True}, 'k-com.zone': {'NotExist': True}, 'k-com.zuerich': {'NotExist': True}, 'k-com.zw': {'NotExist': True}, 'k-com': {'NotExist': True}}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool man! :)

@g4ze g4ze requested a review from mlodic May 30, 2024 09:13
@g4ze g4ze requested a review from 0ssigeno May 30, 2024 09:13
@mlodic
Copy link
Member

mlodic commented May 31, 2024

great!

@mlodic mlodic merged commit 0bac4e6 into intelowlproject:develop May 31, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Analyzer] AILTypoSquatting
3 participants