Satellite UI needs to use different edge service endpoints by project default placement for access key generation

[jtolio]

Background

Users, projects, buckets, and segments can all have a "default placement" set in the database. The placement is an integer, but maps to a configured set of rules on the satellite.

Example for this ticket (newlines inserted for readability):

placement: 
    10:country(US); 
    11:annotated(placement(10),annotation("testAnnotation","testValue")); 
    12:placement(11) || country("DE"); 
    0:exclude(placement(10)); 

The snippet above is not a real configuration on a satellite. However, if a satellite had the above placement configuration, the following would be true:

• a placement of 10 on a project means that all segments/pieces should be geofenced to the US
• a placement of 0 means that segments/pieces should be geofenced outside the US
• a placement of 11 means what is in placement 10 (geofence to US), as well as containing additional arbitrary metadata testAnnotation=testValue
• a placement of 12 means what is in placement 11, plus geofencing in Germany. In other words, pieces/segments with a placement of 12 can be in either the US or Germany (DE), and there will also be the metadata testAnnotation=testValue on that placement

Again, this is not a real-life example. The placement numbers above do not correspond with the actual placements that are configured in production. To see what is actually in production, see an example here in the infra repo.

For this ticket, we want to be able to add annotation to placement rules to override the default edge services URLs for projects with those placements.

So for example:
If the satellite has a placement rule configured as

10:annotated(placement(9),annotation("auth-url","https://custom.authservice.url")

And a project has default_placement set to 10, when registering credentials on authservice within the UI, we should use the custom URL, https://custom.authservice.url, from the placement configuration instead of the default, https://auth.storjsatelliteshare.io.

Acceptance Criteria

• make it possible to get overridden authservice/public linksharing/private linksharing URLs from a project's placement in the UI. In the satellite placement config, the keys or these should be auth-url, public-linksharing-url, and internal-linksharing-url.
  • if it is possible to get annotations from the default placement of the project, we may be able to get this information in the satellite UI using the existing "get project details" endpoint
  • otherwise, we probably need to add a new endpoint to get edge service overrides for a particular project
    • this endpoint would need to cross-reference the project's default placement with the placement rules defined on the satellite, and return the annotated auth URL and linksharing URLs if they are found for the particular placement. Projects without a default placement can be disregarded (return an empty result, or simply return the global defaults in these cases)
• In the event that a project has custom edge URLs defined according to its placement, use these URLs instead of the global defaults within the satellite UI
  • it should not be possible for the satellite UI to use the global edge URL defaults on such a project. S3 credential generation and object browser actions should always use the overridden URLs if they exist
• Ensure that our content security policy properly handles any custom URLs that we may configure in placement rules
  • this does not necessarily need to be dynamic/automatic. If we can add some additional hardcoded subdomains that cover any custom annotations, that should be sufficient
• Out of scope: updating storj/infra production placement config to include these new annotations

Old issue text:

Depends on #6190

Looking through the code, it appears we will need to have the FrontendConfig values of GatewayCredentialsRequestURL, LinksharingURL, and PublicLinksharingURL all set dynamically by project. Once #6190 has a defined way of determining a project's settings for these values, we will need to make sure that either:

1. it is okay for FrontendConfig to be dynamic and be different per project or
2. change how these values are delivered to the frontend so that they can reliably be set per project and not have accidental cross-project caching.

[onionjake]

If we add a new endpoint that gets "ProjectConfig" then that could also be used in #6189 ?

[ferristocrat]

Assuming we are able to use placements (see #6190 (comment)) I think this is still relevant but we need a way to fetch the placement (which I assume we already have?)

[mobyvb]

see these changes to understand more about how placement rules work:

• https://review.dev.storj.io/c/storj/storj/+/10505
• https://review.dev.storj.io/c/storj/storj/+/10507
• https://review.dev.storj.io/c/storj/storj/+/10508

[ferristocrat]

Summary

Something called "placements" are used for geofencing and Storj Select. Placements can be applied at the account, project or bucket level and can have additional metadata (annotations). We plan to use this metadata to inform the Satellite UI of the correct edge related URLs to use for auth service and linksharing (both the internal use URL and the public URL).

Acceptance Criteria (WIP)

• If no placements are set the default URLs are used
  • These are the URLs that are in production today
• If placements are set at the account or project level the following "annotations" are used:
  • Generated linkshare URLs use the public-linksharing-url
  • Satellite UI's use of linksharing uses the internal-linksharing-url
  • Generated S3 credentials use the auth-url provided, which should return the necessary gateway URL to show customers
• Content security policy should allow the new URLs

Assumptions

All new placements can contain the following (optional) placements:

• gateway-url (maybe this isn't necessary since the auth service returns the gateway URL that the Satellite UI uses?)
• auth-url
• public-linksharing-url
• internal-linksharing-url

Open Questions

1. What is the expected behavior with regard to account, project and bucket level placements?
   • Does it behave like bucket placement overrides project placement overrides account placement?
2. How can we add new placements for edge services URLs? Where and when will they be sent?
3. Will Satellite UI have access to the placements already via the project?
4. What about BUCKET LEVEL placements? Since S3 credentials can cover multiple buckets, this may get weird?

[ferristocrat]

Some context

• Placements started just for Geofencing, but have evolved to be mroe general purpose -- placement rules are attached to a segment (because satellite doesn't know anything about buckets)

bucket level placements -- drive new objects created within a bucket (applied at the segment level)
user (account) level placement - determines placement for any new projects created
project level placement - determines placement for any new objects created
object level placement - determines placement for all segments that the object is made up of

"annotations" are abitrary key:value pairs that can be part of a placement script... thus we COULD add annotations for the URLs needed.

config.yaml in infra repo for US1 satellite -- add annotations to the 11th placement

[littleskunk]

Just for fun I want to point out one small mistake in the example placement. It doesn't matter for this ticket but there is a trap in the system that you should be ware of.

11:annotated(placement(10),annotation("testAnnotation","testValue"));
12:placement(11) || country("DE");
[...]
a placement of 12 means what is in placement 11, plus geofencing in Germany. In other words, pieces/segments with a placement of 12 can be in either the US or Germany (DE), and there will also be the metadata testAnnotation=testValue on that placement

The last and condition is wrong. The example placement is US only + the annotation OR DE nodes regardless of the additional annotation. So be careful with the actual syntax.

To avoid this trap we also have a copy of the production placement rules inside a unit test. With that unit test we can double check if our final placement rule would be (US OR DE) AND annotation vs US AND annotation OR DE. The unit test will give us the proof. So please keep the unit tests updated to make sure we never trigger this little hidden bomb in production.

[storj-gerrit]

Change satellite/console: update CSP to include storjapi.io mentions this issue.

[storj-gerrit]

Change satellite/console: return edge URL overrides in project info responses mentions this issue.

[storj-gerrit]

Change web/satellite: use edge service URL overrides mentions this issue.