docs: Policy Audit Mode improvements

[kaworu]

• Scale down the deathstar Deployment to one so that it's easier to observe ingoing flows (as there is only one backend to the service and thus one node to observe from) and enable Policy Audit Mode for a specific endpoint.
• Check that Policy Audit Mode is enabled/disabled using cilium endpoint get.
• Use Hubble to observe instead of cilium monitor, and check the verdict once Policy Audit Mode is disabled.
• Add the empire-default-deny CNP to the cleanup section.
• Misc improvements / rewording on the way.

[qmonnet]

Looks good to me. It's surprisingly complex to run the command on all Pods from the Deployment (but I don't know how to shorten it). Although, I'm somewhat disappointed to see there's no awk involved here :).

[kaworu]

It's surprisingly complex to run the command on all Pods from the Deployment (but I don't know how to shorten it).

For this particular guide, I think we should scale the deathstar svc to 1 Pod. Then, we wouldn't need to loop through the deathstar endpoints to enable/disable Policy Audit Mode and also we wouldn't need to worry about which cilium Pod to exec into for hubble observe. What do you think? @qmonnet

[qmonnet]

Yes, I think that's a good idea

[kaworu]

@qmonnet made many improvements on the way, please take another look and let me know if you think the commits should be squashed 🙏

[qmonnet]

@qmonnet made many improvements on the way, please take another look and let me know if you think the commits should be squashed 🙏

Looks good! Yes, ideally squashing or reorganising the commits would be good, so we don't add this for loop to remove it two commits later