-
Notifications
You must be signed in to change notification settings - Fork 61
Specify external DNS IPs in PlanningInput instead of deriving them from the parent blueprint
#9291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This is part of #9238: we'd like to pull `BlueprintResourceAllocator`'s weird construction out of `BlueprintBuilder`, and this removes _one_ use of it. When adding an internal DNS zone, the caller must now specify the `DnsSubet` instead of `BlueprintBuilder` choosing internally. To assist with this, adds `BlueprintBuilder::available_internal_dns_subnets()`, which returns an iterator of available subnets. I'm not sure this is the right interface; will leave a comment inline with some other ideas.
| // This can't be `#[cfg(test)]` because it's used by the `ExampleSystem` | ||
| // helper (which itself is used by reconfigurator-cli and friends). We give | ||
| // it a scary name instead. | ||
| pub(crate) fn inject_untracked_external_dns_ip( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a solid cleanup. I have a few questions, but they're mostly small. I like where we're going on this.
nexus/db-queries/src/db/datastore/deployment/external_networking.rs
Outdated
Show resolved
Hide resolved
nexus/db-queries/src/db/datastore/deployment/external_networking.rs
Outdated
Show resolved
Hide resolved
| &self, | ||
| opctx: &OpContext, | ||
| ) -> Result<BTreeSet<IpAddr>, Error> { | ||
| // We can _implicitly_ determine the set of external DNS IPs provied |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // We can _implicitly_ determine the set of external DNS IPs provied | |
| // We can _implicitly_ determine the set of external DNS IPs provided |
| loaded collections: f45ba181-4b56-42cc-a762-874d90184a43, eb0796d5-ab8a-4f7b-a884-b4aeacb8ab51, 61f451b3-2121-4ed6-91c7-a550054f6c21 | ||
| loaded blueprints: 184f10b3-61cb-41ef-9b93-3489b2bac559, dbcbd3d6-41ff-48ae-ac0b-1becc9b2fd21, 8da82a8e-bf97-4fbd-8ddd-9f6462732cf1, 58d5e830-0884-47d8-a7cd-b2b3751adeb4 | ||
| loaded service IP pool ranges: [V4(Ipv4Range { first: 192.0.2.2, last: 192.0.2.20 })] | ||
| loaded external IP policy: ExternalIpPolicy { service_pool_ipv4_ranges: [Ipv4Range { first: 192.0.2.2, last: 192.0.2.20 }, Ipv4Range { first: 198.51.100.1, last: 198.51.100.30 }], service_pool_ipv6_ranges: [], external_dns_ips: {198.51.100.1, 198.51.100.2, 198.51.100.3} } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where did this extra IPv4 range come from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our ExampleSystemBuilder uses them for the external DNS external IPs. On main they're added as "untracked" (i.e., not present in a service pool):
omicron/nexus/reconfigurator/planning/src/example.rs
Lines 532 to 548 in 3ca8ec6
| // Add as many external IPs as is necessary for external DNS zones. We | |
| // pick addresses in the TEST-NET-2 (RFC 5737) range. | |
| for i in 0..self.external_dns_count.0 { | |
| builder | |
| .inject_untracked_external_dns_ip(IpAddr::V4(Ipv4Addr::new( | |
| 198, | |
| 51, | |
| 100, | |
| (i + 1) | |
| .try_into() | |
| .expect("external_dns_count is always <= 30"), | |
| ))) | |
| .expect( | |
| "this shouldn't error because provided external IPs \ | |
| are all unique", | |
| ); | |
| } |
This PR gets rid of inject_untracked_external_dns_ip(), so I had to add a range to the service pool that covered them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's still one typo here but this looks good to me
This is built on top of #9250, and is related to both #9238 and #8949:
BlueprintResourceAllocatorinternalsExternalIpPolicytype; today it holds a couple of IP pools and a set of external DNS IPs, but it gives us a cleaner place to integrate all the IP pool work that @bnaecker is doing with ReconfiguratorSince we don't actually have external DNS IPs in a policy anywhere, this adds
DataStore::external_dns_external_ips_specified_by_rack_setup()(which is implemented as "load the current target blueprint and scan it for external DNS zones"; i.e., exactly what the builder was doing before). We should eventually delete this method once it's possible for an operator to reconfigure the external DNS IPs, but this lets the planning input build up the policy as though we already had this information available, more or less. Other than that new method, almost all the changes in this PR are to tests; the non-test changes are pretty minimal.