Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Entities Processing Module #84

Closed
briandelmsft opened this issue Oct 27, 2021 · 14 comments · Fixed by #87
Closed

Entities Processing Module #84

briandelmsft opened this issue Oct 27, 2021 · 14 comments · Fixed by #87
Assignees
@briandelmsft
Labels
new module New automation module
Milestone
Logic/Code - Preview

Comments

@briandelmsft
Copy link
Owner

Build a shared module to handle entity processing to provide consistent results. This is particularly important for the account entity where UPN and/or Azure AD Id are not always included in the entity, but are needed by much of the automation.

Proposed sample return:

{
    "AccountsAllValid": false,
    "AccountsInput": 4,
    "AccountsValid": 3,
    "Accounts": [
        {
            "AADId": "",
            "DistinguishedName": "",
            "objectSID": "",
            "UserPrincipalName": ""
        }
    ],
    "IPsAllValid": true,
    "IPsInput": 1,
    "IPsValid": 1,
    "IPs": [
        {
            "Address": ""
        }
    ],
    "HostsAllValid": true,
    "HostsInput": 0,
    "HostsValid": 0,
    "Hosts": []
}
@briandelmsft briandelmsft added the new module New automation module label Oct 27, 2021
@briandelmsft briandelmsft self-assigned this Oct 27, 2021
@piaudonn
Copy link
Collaborator

🤯Less than 3 sec exec time for a 5 entities array, that's neet!!

@piaudonn
Copy link
Collaborator

if(not(empty(item()['properties']?['upnSuffix'])), 'UPN', if(not(empty(item()['properties']?['aadUserId'])), 'AADId', if(not(empty(item()['properties']?['sid'])), 'ObjectSid', if(not(empty(item()['properties']?['ntDomain'])), 'SamAccountName', if(contains(item()['properties']?['accountName'], ',DC='), 'DistinguishedName', 'Unknown'))))) 👍

What about an optional path in the IP side to geo locate the IP :) or even more about the IP? That way, all other modules could leverage the data as opposed as quering additional services or even the TI table on their side. Or is that too much out of scope? Maybe another module that does just that? Geo locate, reputation score and add all that to tags or comments? Well that maybe already exists... It's late I do a lot of free associations...

@briandelmsft
Copy link
Owner Author

I love it, I think the geo location data in particular would be very helpful, it's easy to get and it is truly an enrichment of the entity.

The threat intel I think would be best in its own module since we want to make sure those insights flow all the way back to the initial playbook... because if initial playbook -calls-> module -calls-> enrichment then which module would bring the TI back to the initial playbook to make a decision based on the TI? I think we'd still need a TI one to do this.

@briandelmsft
Copy link
Owner Author

one other thing about that if(), at first I did it in a variable inside a loop and then did the switch on the variable value but that resulted in needing to enable concurrency control to 1. So i changed it to put it in the select array before the loop, now 'switch'ing on the property in the array so I could remove the concurrency control and it sped up the execution

@briandelmsft
Copy link
Owner Author

@piaudonn the latest commit to the enrich_entities branch now adds Geo data to the IP entities. This introduces 2 dependencies

  • Incident ARM Id needs to be sent to the Enrich-Entities Playbook
  • Enrich-Entities Playbook needs Azure Sentinel Responder (actually, probably even Sentinel Reader would be fine but I didn't test it)

There is a limit of 100 queries/hr/account to the API which i think will be sufficient for most cases

It also removes the requirement that the host entities have an dnsDomain included... now the entity will succeed and provide the host name regardless.

The new return now looks like this:

{
  "Accounts": [
    {
      "userPrincipalName": "victim1@sentinellab.xyz",
      "id": "128bde25-5ba5-4532-9e56-81c42bc64c59",
      "onPremisesSecurityIdentifier": "S-1-5-21-565363340-1337343146-2447627351-32655",
      "onPremisesDistinguishedName": "CN=victim1,OU=Lab,OU=Org,DC=ad,DC=briandel,DC=ca",
      "onPremisesDomainName": "ad.briandel.ca",
      "onPremisesSamAccountName": "victim1"
    }
  ],
  "AccountsAllValid": false,
  "AccountsInput": 2,
  "AccountsValid": 1,
  "Hosts": [
    {
      "DnsDomain": "",
      "FQDN": "host02.",
      "Hostname": "host02"
    },
    {
      "DnsDomain": "contoso.com",
      "FQDN": "host1.contoso.com",
      "Hostname": "host1"
    }
  ],
  "HostsAllValid": true,
  "HostsInput": 2,
  "HostsValid": 2,
  "IPs": [
    {
      "Address": "24.150.222.222",
      "GeoData": {
        "asn": "7992",
        "carrier": "cogeco cable",
        "city": "oakville",
        "cityCf": 90,
        "continent": "north america",
        "country": "canada",
        "countryCf": 99,
        "ipAddr": "24.150.222.222",
        "ipRoutingType": "fixed",
        "latitude": "43.43135",
        "longitude": "-79.76624",
        "organization": "cogeco cable canada inc.",
        "organizationType": "Internet Service Provider",
        "region": "central canada",
        "state": "ontario",
        "stateCf": 95,
        "stateCode": "on"
      }
    }
  ],
  "IPsAllValid": true,
  "IPsInput": 1,
  "IPsValid": 1
}

@briandelmsft
Copy link
Owner Author

To reduce the risk of hitting the API limit, I added a property to trigger to prompt for the Geo Data enrichment, so for modules that don't need it we can do everything else in the module minus pulling the geo data.

@briandelmsft
Copy link
Owner Author

Ok, one more thing 😄... building on the idea of enriching the IP, I added (c47522c) additional user enrichments (title, department, onPremisesSyncEnabled, etc) Since we're in AAD anyways, might as well get some extra properties.

We can add more if needed and may be able to pull enough data that we can remove the need for User.Read.All on the modules that presently require it... not that it's a significant right, just may become redundant.

@piaudonn
Copy link
Collaborator

piaudonn commented Oct 29, 2021 •
edited

Shall we share the vault then (like you suspected earlier) to use the Geo API?

I guess we could also store the lookup result into a Storage Account table to avoid looking up the same IP multiple times in a certain amount of time. Or is that adding to much complexity?

And what about getting an IsAADAdmin flag or something of the sort to denote a (permanent or not) privileged user?

🐇 🕳 ❓

@briandelmsft
Copy link
Owner Author

No APi key is needed.

We could cache results, would be fairly easy with table storage or similar but I think we can hold off on that for now

@briandelmsft
Copy link
Owner Author

I'll add the admin one for sure

@briandelmsft
Copy link
Owner Author

@piaudonn am i missing something or in the graph is there no way to pull admin roles by user? you have to do it by role?

Also, haven't committed yet but I've added the users' managed into the enrichment

@piaudonn
Copy link
Collaborator

Indeed, in my scripts doing these I first export the list of /roleManagement/directory/roleAssignments?`$filter=roleDefinitionId eq '62e90394-69f5-4237-9190-012177145e10' etc... and check if the user ID is in it.

@briandelmsft
Copy link
Owner Author

/roleManagement/directory/roleAssignments?$filter=principalId%20eq%20'0963f2f3-b647-43fd-b6be-8dd00f39419c'&$expand=roleDefinition

Drop all the returned data except displayName?

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#roleManagement/directory/roleAssignments(roleDefinition())",
  "value": [
    {
      "id": "4-PYiFWPHkqVOpuYmLiHa_PyYwlHtv1Dtr6N0A85QZw-1",
      "principalId": "0963f2f3-b647-43fd-b6be-8dd00f39419c",
      "directoryScopeId": "/",
      "roleDefinitionId": "88d8e3e3-8f55-4a1e-953a-9b9898b8876b",
      "roleDefinition": {
        "id": "88d8e3e3-8f55-4a1e-953a-9b9898b8876b",
        "description": "Can read basic directory information. Commonly used to grant directory read access to applications and guests.",
        "displayName": "Directory Readers",
        "isBuiltIn": true,
        "isEnabled": true,
        "resourceScopes": [
          "/"
        ],
        "templateId": "88d8e3e3-8f55-4a1e-953a-9b9898b8876b",
        "version": "1",
        "rolePermissions": [
          {
            "allowedResourceActions": [
              "microsoft.directory/administrativeUnits/standard/read",
              "microsoft.directory/administrativeUnits/members/read",
              "microsoft.directory/applications/standard/read",
              "microsoft.directory/applications/owners/read",
              "microsoft.directory/applications/policies/read",
              "microsoft.directory/contacts/standard/read",
              "microsoft.directory/contacts/memberOf/read",
              "microsoft.directory/contracts/standard/read",
              "microsoft.directory/devices/standard/read",
              "microsoft.directory/devices/memberOf/read",
              "microsoft.directory/devices/registeredOwners/read",
              "microsoft.directory/devices/registeredUsers/read",
              "microsoft.directory/directoryRoles/standard/read",
              "microsoft.directory/directoryRoles/eligibleMembers/read",
              "microsoft.directory/directoryRoles/members/read",
              "microsoft.directory/domains/standard/read",
              "microsoft.directory/groups/standard/read",
              "microsoft.directory/groups/appRoleAssignments/read",
              "microsoft.directory/groups/memberOf/read",
              "microsoft.directory/groups/members/read",
              "microsoft.directory/groups/owners/read",
              "microsoft.directory/groups/settings/read",
              "microsoft.directory/groupSettings/standard/read",
              "microsoft.directory/groupSettingTemplates/standard/read",
              "microsoft.directory/oAuth2PermissionGrants/standard/read",
              "microsoft.directory/organization/standard/read",
              "microsoft.directory/organization/trustedCAsForPasswordlessAuth/read",
              "microsoft.directory/applicationPolicies/standard/read",
              "microsoft.directory/roleAssignments/standard/read",
              "microsoft.directory/roleDefinitions/standard/read",
              "microsoft.directory/servicePrincipals/appRoleAssignedTo/read",
              "microsoft.directory/servicePrincipals/appRoleAssignments/read",
              "microsoft.directory/servicePrincipals/standard/read",
              "microsoft.directory/servicePrincipals/memberOf/read",
              "microsoft.directory/servicePrincipals/oAuth2PermissionGrants/read",
              "microsoft.directory/servicePrincipals/owners/read",
              "microsoft.directory/servicePrincipals/ownedObjects/read",
              "microsoft.directory/servicePrincipals/policies/read",
              "microsoft.directory/subscribedSkus/standard/read",
              "microsoft.directory/users/standard/read",
              "microsoft.directory/users/appRoleAssignments/read",
              "microsoft.directory/users/deviceForResourceAccount/read",
              "microsoft.directory/users/directReports/read",
              "microsoft.directory/users/licenseDetails/read",
              "microsoft.directory/users/manager/read",
              "microsoft.directory/users/memberOf/read",
              "microsoft.directory/users/oAuth2PermissionGrants/read",
              "microsoft.directory/users/ownedDevices/read",
              "microsoft.directory/users/ownedObjects/read",
              "microsoft.directory/users/photo/read",
              "microsoft.directory/users/registeredDevices/read",
              "microsoft.directory/users/scopedRoleMemberOf/read"
            ],
            "condition": null
          }
        ]
      }
    }
  ]
}

@briandelmsft
Copy link
Owner Author

@piaudonn ok, I'm merging the first version into main. additional api permission is required for the roles, it's in the powershell script and readme. I'm sure there's still work to do here but I think it's stable enough to start porting modules over to it. File an issues you see

@briandelmsft briandelmsft mentioned this issue Oct 30, 2021
Merged
@briandelmsft briandelmsft added this to the Logic/Code - Preview milestone Nov 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@briandelmsft briandelmsft

Labels
new module New automation module
Projects
None yet
Milestone
Logic/Code - Preview
Development

Successfully merging a pull request may close this issue.

Enrich entities briandelmsft/SentinelAutomationModules
2 participants
@piaudonn @briandelmsft