Facts and requirements not aligning

[timbrigham-oc]

Describe the bug

This might be more of the same issue I reported last year, might be something new.

I'm not able to concurrently use variable injection #{fact.source.name} and a requirement plugins.stockpile.app.parsers.basic hasAuth that should be able to work together.

To Reproduce
Steps to reproduce the behavior:

1. Create a custom adversary.
2. Create a "OC - TEST Echo Remote Host FQDN" using command echo #{preseed.remote.host.fqdn}, parser plugins.stockpile.app.parsers.basic and output source remote.host.fqdn. I'm doing this as a quick and dirty emulation of finding additional hosts without doing a scan.
3. Create a "OC - Net use FQDN" ability, command like net use \\#{preseed.remote.host.fqdn}\c$ /user:#{preseed.domain.user.name} #{preseed.domain.user.password} parser plugins.stockpile.app.parsers.basic and output source hasAuth
4. Create a "OC - Copy Sandcat SMB" ability, command line like $drive = \\#{remote.host.fqdn}\C$; Copy-Item -Path .\sandcat.go-windows -Destination $drive"\Windows\IMECache\s4ndc4t.exe" -Verbose; requirement like plugins.stockpile.app.requirements.basic, source hasAuth
5. Create a "OC - Copy Sandcat SMB TEST" with a command line like $drive = \\my.hardcoded.name\C$; Copy-Item -Path .\sandcat.go-windows -Destination $drive"\Windows\IMECache\s4ndc4t.exe" -Verbose; requirement like plugins.stockpile.app.requirements.basic, source hasAuth. This can optionally also have a plugins.stockpile.app.requirements.basic, source remote.host.fqdn added without changing the behavior.

Behavior

1. "OC - TEST Echo Remote Host FQDN" successfully sets remote.host.fqdn
2. "OC - Net Use FQDN" successfully finishes and creates hasAuth fact
3. "OC - Copy Sandcat SMB" is not executed, with a "Fact dependency not fulfilled" shown in the audit logs. The only fact in "Copy Sandcat SMB" is remote.host.fqdn, which is for sure created above by "OC - TEST Echo Remote Host FQDN"
4. "OC - Copy Sandcat SMB TEST" executes without an issue since the same value has been hard coded.

Expected behavior
It should be possible to use variable substitution in conjunction with requirement definitions.

[timbrigham-oc]

Here are some sample abilities I'm testing with that are relevant here.

They assume a few facts to be included in the original fact source passed in. Those are denoted with a preseed suffix where applicable.

=====================================
- tactic: build-capabilities
  technique_name: Abuse Elevation Control Mechanism
  technique_id: T1001.002
  name: OC - TEST Echo Remote Host FQDN
  description: ''
  executors:
  - name: psh
    platform: windows
    command: 'echo #{preseed.remote.host.fqdn}'
    code: null
    language: null
    build_target: null
    payloads: []
    uploads: []
    timeout: 60
    parsers:
    - module: plugins.stockpile.app.parsers.basic
      parserconfigs:
      - source: remote.host.fqdn
        edge: ''
        target: ''
        custom_parser_vals: {}
    cleanup: []
    variations: []
    additional_info: {}
  requirements: []
  privilege: ''
  repeatable: false
  buckets:
  - build-capabilities
  additional_info: {}
  access: {}
  singleton: false
  plugin: ''
  delete_payload: true
  id: 1159ecea-de16-453a-b378-66ae91601061

=====================================
- tactic: build-capabilities
  technique_name: Abuse Elevation Control Mechanism
  technique_id: T1001.002
  name: OC - TEST Echo Remote Host FQDN
  description: ''
  executors:
  - name: psh
    platform: windows
    command: 'echo #{preseed.remote.host.fqdn}'
    code: null
    language: null
    build_target: null
    payloads: []
    uploads: []
    timeout: 60
    parsers:
    - module: plugins.stockpile.app.parsers.basic
      parserconfigs:
      - source: remote.host.fqdn
        edge: ''
        target: ''
        custom_parser_vals: {}
    cleanup: []
    variations: []
    additional_info: {}
  requirements: []
  privilege: ''
  repeatable: false
  buckets:
  - build-capabilities
  additional_info: {}
  access: {}
  singleton: false
  plugin: ''
  delete_payload: true
  id: 1159ecea-de16-453a-b378-66ae91601061

=====================================
- tactic: lateral-movement
  technique_name: 'Remote Services: SMB/Windows Admin Shares'
  technique_id: T1021.002
  name: OC - Net use FQDN
  description: "Mounts a network file share on a target computer. \nWorks against\
    \ FQDN. Expects OC- DNS Match completion before use."
  executors:
  - name: psh
    platform: windows
    command: 'net use \\#{preseed.remote.host.fqdn}\c$ /user:#{preseed.domain.user.name}
      #{preseed.domain.user.password}'
    code: null
    language: null
    build_target: null
    payloads: []
    uploads: []
    timeout: 60
    parsers:
    - module: plugins.stockpile.app.parsers.basic
      parserconfigs:
      - source: hasAuth
        edge: ''
        target: ''
        custom_parser_vals: {}
    cleanup:
    - net use \\#{preseed.remote.host.fqdn}\c$ /delete | Out-Null
    variations: []
    additional_info: {}
  requirements: []
  privilege: ''
  repeatable: false
  buckets:
  - lateral-movement
  additional_info: {}
  access: {}
  singleton: false
  plugin: ''
  delete_payload: false
  id: 3867cf4a-bee3-4019-b4a8-0c50c2d481ac

=====================================
- tactic: lateral-movement
  technique_name: 'Remote Services: SMB/Windows Admin Shares'
  technique_id: T1021.002
  name: OC - Copy Sandcat SMB
  description: auto-generated
  executors:
  - name: psh
    platform: windows
    command: " $drive = \\\\#{remote.host.fqdn}\\C$;\n Copy-Item -Path .\\sandcat.go-windows\
      \ -Destination $drive\"\\Windows\\IMECache\\s4ndc4t.exe\" -Verbose;"
    code: null
    language: null
    build_target: null
    payloads:
    - sandcat.go-windows
    uploads: []
    timeout: 60
    parsers: []
    cleanup:
    - '$drive = \\#{remote.host.fqdn}\C$;

      Remove-Item $drive"\Windows\IMECache\s4ndc4t.exe" -Force;

      '
    variations: []
    additional_info: {}
  requirements:
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: hasAuth
      edge: ''
      target: ''
  privilege: ''
  repeatable: false
  buckets:
  - lateral-movement
  additional_info: {}
  access: {}
  singleton: false
  plugin: ''
  delete_payload: false
  id: 6a2a3fb4-d6a3-4949-9356-c0660ba82f61

=====================================
- tactic: build-capabilities
  technique_name: Abuse Elevation Control Mechanism
  technique_id: T1001.002
  name: OC - Copy Sandcat SMB TEST
  description: ''
  executors:
  - name: psh
    platform: windows
    command: '$drive = ''\\myhardcodedfqdn\C$'';

      Copy-Item -Path .\sandcat.go-windows -Destination $drive"\Windows\IMECache                                                                                                                                                             \s4ndc4t.exe"
      -Verbose;

      '
    code: null
    language: null
    build_target: null
    payloads: []
    uploads: []
    timeout: 60
    parsers: []
    cleanup:
    - '$drive = ''\\myhardcodedfqdn\C$'';

      Remove-Item $drive"\Windows\IMECache\s4ndc4t.exe" -Force;

      '
    variations: []
    additional_info: {}
  requirements:
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: hasAuth
      edge: ''
      target: ''
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: remote.host.fqdn
      edge: ''
      target: ''
  privilege: ''
  repeatable: false
  buckets:
  - build-capabilities
  additional_info: {}
  access: {}
  singleton: false
  plugin: ''
  delete_payload: true
  id: 2de5b614-d28a-42ec-ac0f-f60d5cb5b2df


=====================================
report json output: 

    "skipped_abilities": [
        {
            "dcalbt": [
                {
                    "reason": "Fact dependency not fulfilled",
                    "reason_id": 3,
                    "ability_id": "6a2a3fb4-d6a3-4949-9356-c0660ba82f61",
                    "ability_name": "OC - Copy Sandcat SMB"
                }
            ]
        }
    ]

[timbrigham-oc]

The thing that really buggers me is the difference between the variable substitution in the ability body and the requirements definition.

Specifically, in the Test instance it works with a check to see if the value is available, but if I add it to the command body it fails every time. How can I be able to use the same variable in one part of the UI and logic, but not the other?

  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: remote.host.fqdn
      edge: ''
      target: ''

[uruwhy]

Fact requirements are typically used to check a condition for one or more facts - for instance, the paw_provenance requirement checks that the given agent was the one who discovered a particular fact, which is useful for cases like making sure the agent uses a hostname fact for its actual host and not that of another agent. The basic requirement is typically used to ensure that two facts have a specific relationship - a common use case is ensuring that a username fact corresponds to a password fact.

Suppose you have an ability that outputs username and password facts (e.g. cred dump), and a second ability that wants to use valid username/password combinations for authentication or other purposes. Without relationships, the default behavior would be to try out every possible username/password combination, which is noisy and would fail most of the time. Instead, your first ability can use a parser that will link the username and password facts when generating them - this is typically represented as an "edge" that specifies the relationship between the two. Here's an example from a Mimikatz ability from stockpile:

parsers:
          plugins.stockpile.app.parsers.katz:
          - source: domain.user.name
            edge: has_password
            target: domain.user.password

The custom parser will ensure that every valid username/password fact combination is connected with the has_password edge. A subsequent ability can take these username and password facts and make sure they only use the valid combinations by invoking the basic requirement to check that domain.user.name is connected to domain.user.password with the has_password edge. Here's an example from another ability in Stockpile:

requirements:
    - plugins.stockpile.app.requirements.paw_provenance:
      - source: host.user.name
    - plugins.stockpile.app.requirements.basic:
      - source: host.user.name
        edge: has_password
        target: host.user.password

Looking at your ability YML files, you aren't specifying any edge when using the basic requirement, so CALDERA has no way of knowing what the desired condition is. I suspect this is why you are receiving the "Fact dependency not fulfilled" audit message.

I found a similar ability in stockpile that mounts a network share and outputs a fact, which uses a custom parser and to link the remote.host.fqdn fact with edge has_share:

parsers:
          plugins.stockpile.app.parsers.share_mounted:
            - source: remote.host.fqdn
              edge: has_share

No target fact is needed since you're just saying that the FQDN fact has a specific condition. You can do something similar in your case and just rename to edge to hasAuth - note that this would be an edge rather than a completely new source fact. And then in your ability that uses the remote.host.fqdn fact with the mounted share, you would specify the basic requirement with the remote.host.fqdn as the source fact and hasAuth as the edge - no target fact needed. See this example ability for reference:

- plugins.stockpile.app.requirements.basic:
    - source: remote.host.fqdn
      edge: has_share

[timbrigham-oc]

Thanks @uruwhy , we've been digging into this as time permits.

I included the stock rules for Mount Share and Copy 54ndc47 (SMB). I also adjusted how I'm working with my facts (emulating them being learned by echoing the values and capturing, versus reading from the on disk fact store).

My test looks like the following -

I get why the default Mount Share and Copy 54andc47 (SMB) rules aren't firring since there are dependencies that aren't met.

Mount Share needs domain.user.name -> has_password -> domain.user.password which doesn't exist since there isn't an edge between the two in my testing.

Copy 54ndc47 (SMB) is going to fail on remote.host.fqdn -> has_54ndc47_copy since I'm succeeding on setting that OC - TEST Imported Copy 54ndc47 (SMB) at a minimum.

It still seems like there's something wrong.. Steps 7 and 8 here are practically identical. One is my custom written rule, the other is a clone of Copy 54ndc47 (SMB) where I started pulling out some of the requirements trying to get to basics.

I tried running this through diffchecker, just to see what I might be missing.
It's obvious that the tactic, technique, and command bodies are different, as well as the UUIDs.

Here's what really surprised me though. On my non working rule, OC - TEST Copy Sandcat SMB, in the requirements definition, there is a target line, with a blank value. My gut reaction is this is a problem with a null versus empty comparison somewhere upstream

This ability is firing.

- tactic: lateral-movement
  technique_name: 'Remote Services: SMB/Windows Admin Shares'
  technique_id: T1021.002
  name: OC - TEST Imported Copy 54ndc47 (SMB)
  description: OC - TEST Copy 54ndc47 to remote host (SMB)
  executors:
  - name: psh
    platform: windows
    command: '$path = "sandcat.go-windows";

      $drive = "\\#{remote.host.fqdn}\C$";

      Copy-Item -v -Path $path -Destination $drive"\Users\Public\s4ndc4t.exe";'
    code: null
    language: null
    build_target: null
    payloads:
    - sandcat.go-windows
    uploads: []
    timeout: 60
    parsers:
    - module: plugins.stockpile.app.parsers.54ndc47_remote_copy
      parserconfigs:
      - source: remote.host.fqdn
        edge: has_54ndc47_copy
        target: ''
        custom_parser_vals: {}
    cleanup:
    - '$drive = "\\#{remote.host.fqdn}\C$";

      Remove-Item -Path $drive"\Users\Public\s4ndc4t.exe" -Force;'
    variations: []
    additional_info: {}
  requirements:
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: remote.host.fqdn
      edge: has_share
  privilege: ''
  repeatable: false
  buckets:
  - lateral-movement
  additional_info: {}
  access: {}
  singleton: true
  plugin: ''
  delete_payload: true
  id: 66666666-f7ca-49d3-9410-10813e472b30

- tactic: build-capabilities
  technique_name: Abuse Elevation Control Mechanism
  technique_id: T1001.002
  name: OC - TEST Copy Sandcat SMB
  description: auto-generated
  executors:
  - name: psh
    platform: windows
    command: '$drive = ''\\#{remote.host.fqdn}\C$'';

      Copy-Item -Path .\sandcat.go-windows -Destination $drive"\Windows\IMECache\s4ndc4t.exe"
      -Verbose;

      '
    code: null
    language: null
    build_target: null
    payloads:
    - sandcat.go-windows
    uploads: []
    timeout: 60
    parsers: []
    cleanup:
    - '$drive = ''\\#{remote.host.fqdn}\C$'';

      Remove-Item $drive"\Windows\IMECache\s4ndc4t.exe" -Force;

      '
    variations: []
    additional_info: {}
  requirements:
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: remote.host.fqdn
      edge: has_share
      target: ''
  privilege: ''
  repeatable: false
  buckets:
  - build-capabilities
  additional_info: {}
  access: {}
  singleton: true
  plugin: ''
  delete_payload: true
  id: 2de5b614-d28a-42ec-ac0f-f60d5cb5b2df

[timbrigham-oc]

Not a great fan of replying to myself, but I'm pretty sure I'm right here -

I just updated my test adversary with a new line item created by cloning OC - TEST Copy Sandcat SMB and doing nothing more then ripping out the target line.

- tactic: build-capabilities
  technique_name: Abuse Elevation Control Mechanism
  technique_id: T1001.002
  name: OC - TEST Copy Sandcat SMB local edit
  description: auto-generated
  executors:
  - name: psh
    platform: windows
    command: '$drive = ''\\#{remote.host.fqdn}\C$'';

      Copy-Item -Path .\sandcat.go-windows -Destination $drive"\Windows\IMECache\s4ndc4t.exe"
      -Verbose;

      '
    code: null
    language: null
    build_target: null
    payloads:
    - sandcat.go-windows
    uploads: []
    timeout: 60
    parsers: []
    cleanup:
    - '$drive = ''\\#{remote.host.fqdn}\C$'';

      Remove-Item $drive"\Windows\IMECache\s4ndc4t.exe" -Force;

      '
    variations: []
    additional_info: {}
  requirements:
  - module: plugins.stockpile.app.requirements.basic
    relationship_match:
    - source: remote.host.fqdn
      edge: has_share
  privilege: ''
  repeatable: false
  buckets:
  - build-capabilities
  additional_info: {}
  access: {}
  singleton: true
  plugin: ''
  delete_payload: true
  id: 66666666-d28a-42ec-ac0f-f60d5cb5b2df

This runs as I would expect it to.

[uruwhy]

Glad you got it working! Your additional findings made me realize that I should've been more explicit in my earlier description by saying that you shouldn't add a target line at all in the requirements since it does look like the requirements module handles a non-existent vs "empty string" target differently, as you expected: https://github.com/mitre/stockpile/blob/master/app/requirements/base_requirement.py#L15-L19

I may put in a PR shortly to handle empty targets the same way as if they weren't specified at all, just so other users don't run into this same problem. Thanks for bringing this to our attention!

[timbrigham-oc]

Thanks @uruwhy I love this community!

If I were crafting the YML by hand it wouldn't have been there for sure, but it's being included by default if you create it via the 'Create an Ability' UI in the web console.

I'll keep an eye out for your PR on this for sure! I started digging into the facts and requirements here a bit, this is one of the files I was looking at. Python isn't my language of choice but run into that same null or empty conditional logic issue pretty regularly in PowerShell. Would the fix look something like this, out of curiosity? I'm assuming it'd be right before iterating the keys.

        if not relationship.target or str(relationship.target).strip() == "":
            return False

[uruwhy]

Gotcha, I wasn't aware the ability YAML files created by the UI always had that line by default. I normally create my abilities by hand and rarely use the UI for that, so that's good to know (and also means this fix will definitely be helpful).
As for the PR, my plan is to essentially turn this line into the following:

if self.enforcements.get('target', None):

Which will resolve to false if your relationship_match dictionary under your ability yaml requirements has no target key or if the target value is an empty string.

I'll just have to test this and get the PR reviewed, but in the meantime I can give you the test branch if you want to run with it

[timbrigham-oc]

Gotcha, I figured there was probably something better then doing a string comparison. Not familiar at all with these internal object types.

I already dumped a few extra lines of logging into that file trying to trace which state it needed to be in. I can go ahead and try making that edit locally, see if it has the desired impact.

[uruwhy]

If you want to play around with the dev branch with the pending fix (note that this is in the stockpile plugin and not in the core codebase)

cd caldera/plugins/stockpile
git checkout master
git pull
git fetch origin issue-3177:issue-3177
git checkout issue-3177

Then restart your caldera server

[uruwhy]

Gotcha, I figured there was probably something better then doing a string comparison. Not familiar at all with these internal object types.

I already dumped a few extra lines of logging into that file trying to trace which state it needed to be in. I can go ahead and try making that edit locally, see if it has the desired impact.

Sure, go for it. It'll be a few days before someone else on the team is able to take a look at the PR, with the holiday weekend in the US and all.