Enable retrieving and modifying a resource by `id`

[just-be-dev]

Currently we have no way of retrieving or acting on a resource if we have its id. To remedy that we should implement a suite of endpoints to act on resources given the id. Having an id is unique in that you don't have to do a path resolution to find the resource (given that ids are globally unique to a resource).

As originally suggested by @davepacheco, we've agreed on /by_id/<resource>/<id> as the general shape of the API. It makes sense to have GET, PUT, and DELETE actions for each of these resources.

Previous approach

When calling an API endpoint we should allow name or id to be used interchangeably. We've had several discussions about this in the past. @davepacheco mentioned using id:<uuid> as a pattern for specifying ids in the route.

I'd personally like to add a restriction to names such that a name can't be a valid UUID in order to avoid confusion there. We could go further by creating a wrapping enum Identifier type which looks like

enum Identifier {
  Id(Uuid),
  Name(String)
}

which would be type that the api expects as a resource identifier. That would then be converted from the api callsite down into an id to be passed to the rest of the internal calls.

Regardless of the implementation I do feel strongly we should have some mechanisms for being able to make requests by id especially given that we have no mechanism for tracing resource renames.

[david-crespo]

The relevant meeting recording is here: Omicron + Oso + Review (2021-11-22 at 10:02 GMT-8)

The part where we get into the above questions starts at 1:00:10. The discussion is kind of jumpy but here are some notes I wrote down afterward:

---

While going over Dave's second authz PR, we realized that the (org id, project id, disk id) needed to authorize actions on a disk is basically the same as what you produce when just trying to look up a disk by the names. We also realized you need all three IDs even when you’re doing a lookup with a UUID because you need the parent IDs for authz.

Because you need all these IDs no matter what, you might as well do the lookups that produce those IDs as early as possible in the call chain. That way you can deal with a universal resource identifier thing as much of the time as possible. So, rather than passing around models (e.g., to and from the nexus.rs functions), you pass around what I’m thinking we should call a Resource, which looks something like

struct DiskResource {
  disk: db::models::Disk,
  identifier: (org_id, project_id, disk_id)
}

Regardless of whether you’re looking up with (org_name, project_name, disk_name) or just disk_id, you do the lookups up front that produce a DiskResource, and after that point you’re all good, you have a universal resource identifier that will work anywhere.

[davepacheco]

I like this general idea. I think the biggest reason to support lookups by id is when API consumers are following pointers in the API, ids make a lot of sense. The consumer might be looking at a VPC object returned by the API that has its associated project_id. But without a way to get a Project by id, it's annoying to fetch the corresponding Project information -- they'd have to, like, list all Projects in all Organizations that they can see and find one with the matching id, I guess? (We could put the project name into the VPC object instead of its id, but it doesn't really help: they might fetch that name and now it's a different Project and they have to do the whole search again.) This also helps a lot if consumers are building any of their own infrastructure atop our API -- they'll want to use ids, since they're more stable.

(This example is maybe contrived because if you have a VPC object, you would have had to request it by knowing which Project you were in already. But I don't think that's generally true -- you could be looking at a billing report or metrics output or maybe the parent Project was renamed (and replaced) since you fetched the VPC resource.)

Part of the idea with the id: prefix was that's already not allowed in a name. I also like that a human can tell which one they're looking at, and can't be easily confused (which you could be if you, say, parsed it as a uuid first, then tried as a name -- someone could confuse a person with something that looked like a uuid, but isn't quite). There are probably other ways to do this.

I wonder too if we'd be better off with a separate set of top-level endpoints. One advantage is that you wouldn't have to know what parent container your resource is in. I feel like we discussed another important advantage for correctness but I can't remember it now -- sorry!

As @david-crespo was getting at, this will overlap quite a bit with the ongoing authz work. I think this might actually be a lot easier after we've built up internal APIs for doing an authz-aware lookup.

[david-crespo]

I also want to note the picture I gave fits perfectly with what @zephraph described here

That would then be converted from the api callsite down into an id to be passed to the rest of the internal calls.

With the qualification that the ID to be passed down is actually the chain of parent IDs down to the resource. I represented that as a tuple for simplicity but it doesn't have to be one.

[davepacheco]

Another consideration came up today. To use a concrete example, suppose a client has an Instance id and wants to reboot the Instance.

If we implement /organizations/:orgIdOrName/projects/:projectIdOrName/instances/:instanceIdOrName, this won't help them. They'd need to have saved the parent project id or name and the organization project id or name. That's weird by itself, since the uuid in principle should be sufficient. But what if the Instance has been moved between Projects, or the Project has been moved between Organizations? They're out of luck again -- they don't have the information they need to look up the Instance by id without doing an exhaustive search through all Organizations and Projects.

This suggests that we implement /by_id/instances/:instance_id, which lets you GET an instance by its id. But if we don't implement anything else there, then you'll still need to use the by-name hierarchy to reboot it. So you'll need to also get /by_id/projects/:project_id to get its parent project id, and similar for the parent Org, and then issue /organizations/:orgName/projects/:projectName/instances/:instanceName/reboot with an ETag-conditional request that it's the same Instance.

My takeaway is that for this to be useful (i.e., if we want people to be able to operate on an Instance with its id at least as easily as they can by name), we probably should mirror all the routes in the by-name hierarchy in the by-id hierarchy too. (Another option would be to have the by-name hierarchy 301 you over to the by-id one, but that seems like as much work and a lot less convenient for users.)

[davepacheco]

I'm going to try prototyping a new interface for resource lookup that I think will make it a lot easier to implement this (among many other things).

[david-crespo]

When Justin and I talked about this when we were in town, we landed in the same place, re:

a) /by_id/instances/:instance_id and
b) we need PUT and other endpoints there too

[just-be-dev]

Yes, sorry, I'll update this issue to match that context.

[davepacheco]

This could be enabled by #798.

[just-be-dev]

Yeah, I've just been reading through that. I'm incredibly hyped! That's a great piece of work.