Add retry logic to DNS that attempts to retry against a public resolver. Ensure that report indicates level of confidence in the correctness of the result.

[schrolla]

Agency 2 assessment showed a "FAIL" but when looking into the domains flagged by the assessment script, many of which were sub-domains to ones configured in the DNS records hosted by the agency's domain. Agency 2 also mentioned that they noticed two domains that should be approved.

Ethan did check some domains, one did provide a SFT and one provided a “null” output. Ethan also investigated and it could be some weird DNS lookup failures on our end. Maybe we just try to do more testing against tenants with many (think Agency 2 has 60) domains.

[nanda-katikaneni]

Did EXO assessment on a E3 tenant (Test Tenant#5)- most policies worked fine but had an intermittent issue with Policy 2.2. In some of the tests, the policy errored with GetSpfRecord command failure. This maybe DNS lookup failures on the test environment - but I could not consistently reproduce the issue. The tenant had 12 domains - may need some additional testing and improved error handling to identify transient issues.

[adhilto]

I recommend we create an issue to enhance the DNS request code for SPF, DKIM, and DMARC. Transient issues are to be expected with DNS and our code currently isn't built to handle that appropriately. We should add a re-try mechanism so that for any failed query (other than for an NXDOMAIN) the query gets re-tried.

Also in favor of additional testing.

[schrolla]

Recommendation is to add an issue to create or re-create issue to look at adding a DNS retry to the current automation to account for potential intermittent DNS failures.

[ethanb-cisa]

I believe the issue is related to where the agency runs the script. Querying a domain from internal may hit an internal DNS server that returns a different result than what is served to the public for that domain. This is the case inside CISA. Resolve-DnsName -Type TXT -Name cisa.dhs.gov | fl returns no SPF but running from an Azure VM does.

Some orgs will block DNS requests to non-agency DNS servers, so it's not like we could set a public DNS server as the recursive resolver (1.1.1.1, etc). I believe we even require them to prevent this to prevent DNS filtering bypass.

This likely also impacts DKIM/DMARC.

So figuring that out will be...interesting.

[adhilto]

You're probably right. I tested the Get-ScubaSpfRecords against one of the domains that was failing in yesterday's review, it worked just fine. That said, when we overhauled EXO's error handling, we did improve the DNS code. Now if the queries fail due to a network issue, the report will display the error rather than just saying there were no SPF/DKIM/DMARC records. I'd be curious to see what the output from these agencies looks like on the latest code.

[gdasher]

I rescoped #38 to be about the retries. Its possible those issues are also split horizon issues, though it surprises me a bit. But it is possible they fully claim their own zones and don't think about things like txt records that only matter for inbound traffic. We should discuss in team meeting how we want to rescope these issues and move forward.

[nanda-katikaneni]

Now if the queries fail due to a network issue, the report will display the error rather than just saying there were no SPF/DKIM/DMARC records. I'd be curious to see what the output from these agencies looks like on the latest code.

Yes, this is the behavior I saw in my tests on E3 tenant. The new error handling is correctly flagging when spf records are missing; when the dns queries are failed due to network issues, it raised the error on the terminal output with a corresponding note in the report as below.

[schrolla]

Details of potential fix from duplicate issue.

💡 Summary

Enhance DNS-related code by adding a retry over DoH option.

Motivation and context

We've seen some inconsistent results for tests that explicitly make DNS requests (SPF, DKIM, and DMARC checks), possibly due to networking issues. Adding DoH as a fallback should the traditional DNS request fail improves the odds of successfully making a request.

Implementation notes

Example of making a DoH call with Powershell:

$(Invoke-WebRequest -H @{"accept"="application/dns-json"} -Uri "https://1.1.1.1/dns-query?name=dhs.cisa.gov&type=txt").RawContent