Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions omicron-common/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ http = "0.2.0"
hyper = "0.14"
libc = "0.2.98"
propolis-server = { git = "https://github.com/oxidecomputer/propolis", rev = "b6da043d" }
postgres-protocol = "0.6.1"
rayon = "1.5"
reqwest = { version = "0.11", default-features = false, features = ["rustls-tls"] }
ring = "0.16"
Expand Down Expand Up @@ -41,6 +42,14 @@ features = [ "serde" ]
git = "https://github.com/oxidecomputer/dropshot"
branch = "main"

[dependencies.ipnet]
version = "2.3.1"
features = [ "serde" ]

[dependencies.macaddr]
version = "1.0.1"
features = [ "serde_std" ]

[dependencies.schemars]
version = "0.8"
features = [ "chrono", "uuid" ]
Expand Down
216 changes: 215 additions & 1 deletion omicron-common/src/api/model.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ use std::fmt::Debug;
use std::fmt::Display;
use std::fmt::Formatter;
use std::fmt::Result as FormatResult;
use std::net::SocketAddr;
use std::net::{IpAddr, SocketAddr};
use std::num::NonZeroU32;
use std::time::Duration;
use uuid::Uuid;
Expand Down Expand Up @@ -1313,6 +1313,220 @@ impl From<steno::SagaStateView> for SagaStateView {
}
}

/// A Virtual Private Cloud (VPC) object.
#[derive(Clone, Debug)]
pub struct VPC {
/** common identifying metadata */
pub identity: IdentityMetadata,
/** id for the project containing this Instance */
pub project_id: Uuid,
}

/// An `Ipv4Net` represents a IPv4 subnetwork, including the address and network mask.
#[derive(Clone, Copy, Debug, Deserialize, PartialEq, Serialize)]
pub struct Ipv4Net(pub ipnet::Ipv4Net);

impl std::ops::Deref for Ipv4Net {
type Target = ipnet::Ipv4Net;
fn deref(&self) -> &Self::Target {
&self.0
}
}

impl std::fmt::Display for Ipv4Net {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}

impl JsonSchema for Ipv4Net {
fn schema_name() -> String {
"Ipv4Net".to_string()
}

fn json_schema(
_: &mut schemars::gen::SchemaGenerator,
) -> schemars::schema::Schema {
schemars::schema::Schema::Object(
schemars::schema::SchemaObject {
metadata: Some(Box::new(schemars::schema::Metadata {
title: Some("An IPv4 subnet".to_string()),
description: Some("An IPv4 subnet, including prefix and subnet mask".to_string()),
examples: vec!["192.168.1.0/24".into()],
..Default::default()
})),
instance_type: Some(schemars::schema::SingleOrVec::Single(Box::new(schemars::schema::InstanceType::String))),
string: Some(Box::new(schemars::schema::StringValidation {
// Fully-specified IPv4 address. Up to 15 chars for address, plus slash and up to 2 subnet digits.
max_length: Some(18),
min_length: None,
// Addresses must be from an RFC 1918 private address space
pattern: Some(
concat!(
// 10.x.x.x/8
r#"^(10\.(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9]\.){2}(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9])/(1[0-9]|2[0-8]|[8-9]))$"#,
// 172.16.x.x/12
r#"^(172\.16\.(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9])\.(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9])/(1[2-9]|2[0-8]))$"#,
// 192.168.x.x/16
r#"^(192\.168\.(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9])\.(25[0-5]|[1-2][0-4][0-9]|[1-9][0-9]|[0-9])/(1[6-9]|2[0-8]))$"#,
).to_string(),
),
})),
..Default::default()
}
)
}
}

/// An `Ipv6Net` represents a IPv6 subnetwork, including the address and network mask.
#[derive(Clone, Copy, Debug, Deserialize, PartialEq, Serialize)]
pub struct Ipv6Net(pub ipnet::Ipv6Net);

impl std::ops::Deref for Ipv6Net {
type Target = ipnet::Ipv6Net;
fn deref(&self) -> &Self::Target {
&self.0
}
}

impl std::fmt::Display for Ipv6Net {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}

impl JsonSchema for Ipv6Net {
fn schema_name() -> String {
"Ipv6Net".to_string()
}

fn json_schema(
_: &mut schemars::gen::SchemaGenerator,
) -> schemars::schema::Schema {
schemars::schema::Schema::Object(
schemars::schema::SchemaObject {
metadata: Some(Box::new(schemars::schema::Metadata {
title: Some("An IPv6 subnet".to_string()),
description: Some("An IPv6 subnet, including prefix and subnet mask".to_string()),
examples: vec!["fd12:3456::/64".into()],
..Default::default()
})),
instance_type: Some(schemars::schema::SingleOrVec::Single(Box::new(schemars::schema::InstanceType::String))),
string: Some(Box::new(schemars::schema::StringValidation {
// Fully-specified IPv6 address. 4 hex chars per segment, 8 segments, 7
// ":"-separators, slash and up to 3 subnet digits
max_length: Some(43),
min_length: None,
pattern: Some(
// Conforming to unique local addressing scheme, `fd00::/8`
concat!(
r#"^(fd|FD)00:((([0-8a-fA-F]{1,4}\:){6}[0-8a-fA-F]{1,4})|(([0-8a-fA-F]{1,4}:){1,6}:))/(6[4-9]|[7-9][0-9]|1[0-1][0-9]|12[0-6])$"#,
).to_string(),
),
})),
..Default::default()
}
)
}
}

/// A VPC subnet represents a logical grouping for instances that allows network traffic between
/// them, within a IPv4 subnetwork or optionall an IPv6 subnetwork.
#[derive(Clone, Debug)]
pub struct VPCSubnet {
/** common identifying metadata */
pub identity: IdentityMetadata,

/** The VPC to which the subnet belongs. */
pub vpc_id: Uuid,

// TODO-design: RFD 21 says that V4 subnets are currently required, and V6 are optional. If a
// V6 address is _not_ specified, one is created with a prefix that depends on the VPC and a
// unique subnet-specific portion of the prefix (40 and 16 bits for each, respectively).
//
// We're leaving out the "view" types here for the external HTTP API for now, so it's not clear
// how to do the validation of user-specified CIDR blocks, or how to create a block if one is
// not given.
/** The IPv4 subnet CIDR block. */
pub ipv4_block: Option<Ipv4Net>,

/** The IPv6 subnet CIDR block. */
pub ipv6_block: Option<Ipv6Net>,
}

/// The `MacAddr` represents a Media Access Control (MAC) address, used to uniquely identify
/// hardware devices on a network.
// NOTE: We're using the `macaddr` crate for the internal representation. But as with the `ipnet`,
// this crate does not implement `JsonSchema`, nor the the SQL conversion traits `FromSql` and
// `ToSql`.
#[derive(Clone, Copy, Debug, Deserialize, PartialEq, Serialize)]
pub struct MacAddr(pub macaddr::MacAddr6);

impl std::ops::Deref for MacAddr {
type Target = macaddr::MacAddr6;
fn deref(&self) -> &Self::Target {
&self.0
}
}

impl std::fmt::Display for MacAddr {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}

impl JsonSchema for MacAddr {
fn schema_name() -> String {
"MacAddr".to_string()
}

fn json_schema(
_: &mut schemars::gen::SchemaGenerator,
) -> schemars::schema::Schema {
schemars::schema::Schema::Object(schemars::schema::SchemaObject {
metadata: Some(Box::new(schemars::schema::Metadata {
title: Some("A MAC address".to_string()),
description: Some(
"A Media Access Control address, in EUI-48 format"
.to_string(),
),
examples: vec!["ff:ff:ff:ff:ff:ff".into()],
..Default::default()
})),
instance_type: Some(schemars::schema::SingleOrVec::Single(
Box::new(schemars::schema::InstanceType::String),
)),
string: Some(Box::new(schemars::schema::StringValidation {
max_length: Some(17), // 12 hex characters and 5 ":"-separators
min_length: Some(17),
pattern: Some(
r#"^([0-8a-fA-F]{2}:){5}[0-8a-fA-F]{2}$"#.to_string(),
),
})),
..Default::default()
})
}
}

/// A `NetworkInterface` represents a virtual network interface device.
#[derive(Clone, Debug, Deserialize, JsonSchema, Serialize)]
pub struct NetworkInterface {
/** common identifying metadata */
pub identity: IdentityMetadata,

/** The VPC to which the interface belongs. */
pub vpc_id: Uuid,

/** The subnet to which the interface belongs. */
pub subnet_id: Uuid,

/** The MAC address assigned to this interface. */
pub mac: MacAddr,

/** The IP address assigned to this interface. */
pub ip: IpAddr,
Comment on lines +1526 to +1527
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After seeing @Nieuwejaar 's presentation in the control plane sync today, IMO we should probably be explicit this is an "internal" IP address.

Sounds like Nexus is also going to be responsible for assigning + telling the switch/OPTE about the external IP address too, so we can do effective translation.

Should we add an optional (I assume it would be optional - seems possible to have internal-only NICs) "external IP address" field here too?

(also FYI @rzezeski for feedback)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After seeing @Nieuwejaar 's presentation in the control plane sync today, IMO we should probably be explicit this is an "internal" IP address.

More precisely, I think this is an IP address from the associated VPC Subnet. I'm not sure it's always "internal", depending on how you mean that: a customer may eventually have two VPC Subnets with routes between them, and this address would be visible on that other network, right?

There may be two layers of abstraction here: the VNIC that we store in the database presumably knows about VPC Subnets and knows this is an IP address from that subnet. By the time we pass this information to Sled Agent, I'm not sure it needs to know that -- it's just an IP address, wherever it came from.

Should we add an optional (I assume it would be optional - seems possible to have internal-only NICs) "external IP address" field here too?

Hmm. I've been assuming that when we implement Floating IPs and Ephemeral IPs, these would have separate records in the database for those. That is, we'd create a row in a FloatingIp table that contains the VnicId and the public IP address. This makes it easy to see which have been allocated, attached where, etc. And it makes it easy to have different kinds of public IPs on the VNIC or even more than one of them (e.g., more than one Floating IP). I'm not sure if the latter is something we intend to support in v1 but it seems reasonable. Thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More precisely, I think this is an IP address from the associated VPC Subnet. I'm not sure it's always "internal", depending on how you mean that: a customer may eventually have two VPC Subnets with routes between them, and this address would be visible on that other network, right?

Yeah, you're right - I came at this with the framing of "internal to the VPC networks" vs "external to the outside world" mindset, but I agree that those frames are relative. Perhaps just clarification that "IP address allocated within the VPC subnet" would be enough to distinguish it from "the other one". (this admittedly may be inferred from context, but I like being as explicit as we can be, since there so many dang layers at play)

There may be two layers of abstraction here: the VNIC that we store in the database presumably knows about VPC Subnets and knows this is an IP address from that subnet. By the time we pass this information to Sled Agent, I'm not sure it needs to know that -- it's just an IP address, wherever it came from.

I'm not so certain - from the point of view of Propolis, yes, it can just act on the VPC subnet, but there may be information that needs ferrying to OPTE to transform packets, which would presumably be proxied through the sled agent.

Hmm. I've been assuming that when we implement Floating IPs and Ephemeral IPs, these would have separate records in the database for those. That is, we'd create a row in a FloatingIp table that contains the VnicId and the public IP address. This makes it easy to see which have been allocated, attached where, etc. And it makes it easy to have different kinds of public IPs on the VNIC or even more than one of them (e.g., more than one Floating IP). I'm not sure if the latter is something we intend to support in v1 but it seems reasonable. Thoughts?

I think this may be another case where "db representation" != "internal API representation".

From a storage perspective I absolutely agree with you, that makes sense - use a different object, reference this NIC by UUID, do querying that way, etc.

From an API perspective, I think that public IP would also need transferring to OPTE running on the Sled - IMO it makes sense to put it in the structure being passed as a part of the "instance_ensure" invocation.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More precisely, I think this is an IP address from the associated VPC Subnet. I'm not sure it's always "internal", depending on how you mean that: a customer may eventually have two VPC Subnets with routes between them, and this address would be visible on that other network, right?

Yeah, you're right - I came at this with the framing of "internal to the VPC networks" vs "external to the outside world" mindset, but I agree that those frames are relative. Perhaps just clarification that "IP address allocated within the VPC subnet" would be enough to distinguish it from "the other one". (this admittedly may be inferred from context, but I like being as explicit as we can be, since there so many dang layers at play)

I think the way I would frame this distinction is actually what addresses are on objects that are visible from inside of the VM on IP interfaces. That is, does a user see this when they run ifconfig or ip addr or ipconfig /all.

Otherwise all the other addresses we've talked about, ephemeral and floating, are not seen on interfaces inside of the guest. The reason I like phrasing it this way is that pretty much that's the thing that really impacts a bit of who and what is receiving traffic and what rules are required where.

There may be two layers of abstraction here: the VNIC that we store in the database presumably knows about VPC Subnets and knows this is an IP address from that subnet. By the time we pass this information to Sled Agent, I'm not sure it needs to know that -- it's just an IP address, wherever it came from.

I'm not so certain - from the point of view of Propolis, yes, it can just act on the VPC subnet, but there may be information that needs ferrying to OPTE to transform packets, which would presumably be proxied through the sled agent.

Information may be proxied through sled-agent, but I'd probably be careful to distinguish between sled-agent being basically just being a transport or it actually needing to know a bunch of stuff.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way I would frame this distinction is actually what addresses are on objects that are visible from inside of the VM on IP interfaces. That is, does a user see this when they run ifconfig or ip addr or ipconfig /all.

Otherwise all the other addresses we've talked about, ephemeral and floating, are not seen on interfaces inside of the guest. The reason I like phrasing it this way is that pretty much that's the thing that really impacts a bit of who and what is receiving traffic and what rules are required where.

Ack - IMO it would still be handy to clarify this is the IP address "inside the VM'.

Information may be proxied through sled-agent, but I'd probably be careful to distinguish between sled-agent being basically just being a transport or it actually needing to know a bunch of stuff.

I think we're on the same page here - it just needs to get to OPTE somehow, but sled-agent is doing little else with the info.

}

/*
* Internal Control Plane API objects
*/
Expand Down
Loading