Support base path proxy rewrites

[newswangerd]

Is your feature request related to a problem? Please describe.

For AAP 2.5 we're looking at re-architecting the platform to use an API gateway. Our end goal is to have a single host that serves the APIs for multiple AAP services (AWX, EDA, Galaxy NG) via a proxy. The proxy needs to be able to support multiple instances of galaxy ng from different API base paths. Ex:

• /proxy/v1/galaxy1/ proxies to galaxy1.example.com/api/galaxy/
• /proxy/v1/galaxy2/ proxies to galaxy2.example.com/api/galaxy/

This is a huge problem for galaxy because all of the pulp HREFs are based on the pulp API base path. With a setup like this, clients will get HREFs that start with /api/galaxy that don't work on the proxy.

Since we're looking to support existing galaxy ng services using this new API gateway, we don't want to force admins to change their API base paths to match the path on the proxy server.

Describe the solution you'd like

At the moment we're solving this issue by using regex to replace the HREFs for every response. This is an inefficient and error prone process. I would really like it if the could be handled on the pulp side. It would be great if the pulp base path could be set for a sepcific request using a configurable header such as X-Forwarded-Prefix.

Alternatives

We've considered using headers to direct traffic on the AAP proxy instead of using base paths (so all galaxy requests would be served from /api/galaxy on the proxy, with a header to select specific galaxy instances), but this is a very sub optimal solution that we would like to avoid.

[mdellweg]

As long as we use hrefs as task reserved resources we need to be very careful here.

[mdellweg]

Idea from pulpcore meeting:
If we used the independent part of the href (not including the domain ; starting from /api/v3) we can probably fill in the prefix from the request to the api. That way the database would store settings independent data.

[newswangerd]

For context, this is the Lua script that we have to run on envoy on every request and response that goes the proxy:

function envoy_on_response(handle)
    -- This script rewrites urls in the response by replacing the service base path
    -- in Location headers and in the request body.
    -- ex: translates /service-path/galaxy/ to /gateway-path/hub/ in service response bodies
    handle:logDebug("Running lua script to rewrite response body.")

    local location = handle:headers():get("Location")
    local prefix_rewrite = handle:metadata():get("prefix")
    local prefix = handle:metadata():get("prefix_rewrite")

    if location then
        
        local new_location = string.gsub(location, prefix, prefix_rewrite)
        handle:headers():replace("Location", new_location)
    end
    local body = handle:body()
    if body then
        body_bytes = body:getBytes(0, body:length())
        local modified_body = string.gsub(body_bytes, prefix, prefix_rewrite)

        local content_length = handle:body():setBytes(modified_body)
        handle:headers():replace("content-length", content_length)
    end
end

function envoy_on_request(handle)
    -- Translate urls in the request body back to the service's base path.
    -- ex: translates /gateway-path/hub/ to /service-path/galaxy/ in service request bodies
    handle:logDebug("Running lua script to rewrite request body.")

    local prefix_rewrite = handle:metadata():get("prefix")
    local prefix = handle:metadata():get("prefix_rewrite")

    local path = handle:headers():get(":path")
    if string.match(path, "=") then
        -- need to escape "-"
        local match = "%=" .. string.gsub(prefix_rewrite, "%-", "%%-")
        local replace = "%=" .. string.gsub(prefix, "%-", "%%-")

        handle:logInfo(match)
        handle:logInfo(replace)

        local new_path = string.gsub(path, match, replace)

        -- if the query param is urlencoded, replace those instead
        new_path = string.gsub(new_path, string.gsub(match, "/", "%%%%2F"), string.gsub(replace, "/", "%%%%2F"))

        handle:headers():replace(":path", new_path)
    end

    local body = handle:body()
    if body then
        body_bytes = body:getBytes(0, body:length())
        local modified_body = string.gsub(body_bytes, prefix_rewrite, prefix)

        local content_length = handle:body():setBytes(modified_body)
        handle:headers():replace("content-length", content_length)
    end
end

It's adding a significant amount of latency to the proxy.