Running as part of another server

[hmhealey]

Instead of running an image proxy as a standalone process, I was looking at having it run as part of another server (in my case, Mattermost) so that the other server handles the HTTP requests and then calls out to the image proxy to actually get the image. Since the only public method of the Proxy type is ServeHTTP, this isn't possible right now without possibly some weird logic to fake a request.

Would you accept a PR that makes the changes required for that? I haven't tested these changes yet, but it seems like it would require changes to:

1. Make Proxy.serveImage public and have it take an image URL as the second parameter instead of an http.Request.
2. Similarly, make NewRequest take an image URL and remove the Request.Original field since it is no longer passed down
3. Move the referrer check out of Proxy.allowed and into Proxy.ServeHTTP so that it can still be checked when we have the original request

Alternatively, to avoid some of those changes (in particular, changing the public fields of the Request), we could

1. Make Proxy.serveImage public and have it take an explicit image URL in addition to the request
2. Have NewRequest use the explicit URL if it exists (ServeHTTP would pass nil) or otherwise use the original request URL as it does now
3. Have Proxy.allowed skip the referrer check if there is no original request provided

[willnorris]

I'm certainly open to the idea in theory. Some time ago, I tried doing the same thing for my own website... combining imageproxy and other similar go services into a single binary for easier management.

When I tried to do it then, I think the problem I ran into was how URLs were being canonicalized, but I think that problem may have long since been handled. I'm curious what Mattermost is doing, and why the existing ServeHTTP method doesn't work. Is Matter most doing some kind of additional processing on the request, rather than just passing it off to imageproxy?

My only hesitation with these kinds of changes is that I want to try and keep the exported API as minimal as possible. But embedding imageproxy behind/within another server is a totally valid use-case that I'd love to be able to support, I just want to be really thoughtful in how we do it.

[hmhealey]

The few issues I've noted so far with just using ServeHTTP are that:

1. Mattermost already has an API endpoint (/api/v4/image) that it uses for serving images from an external proxy, so we'd need to support that route for an internal image proxy. Mattermost can also be hosted at a subpath which may change that route slightly.
2. Similarly, the existing API takes the URL as a query parameter instead of as part of the path.
3. There's a few places we're going to be requesting proxied images to gather some metadata about them before a client requests them, so those cases won't even have an HTTP request.

The more I think about this, there may be a few other changes that might be needed like passing options separately from the URL, but I'd need to think more about if those changes are needed and how that could be done. Maybe to support that case, ServeHTTP could call NewRequest and pass the results to ServeImage so that anyone else could construct the Request itself with whatever options and URL that it wants.

If you're open to the idea though, I can take a shot at implementing any required changes to see how they work in practice. I'm hoping to find some time in the next week or so for that.

[willnorris]

Are you hoping to use imageproxy as a drop-in replacement of the existing proxy? Same path, same query parameters, etc? Just trying to better understand what additional options would be necessary.

For passing the remote URL as a query parameter rather than the path, see #66 and (more broadly) https://github.com/willnorris/imageproxy/blob/plugins/doc/plugin-design.md.

I haven't done much more thinking about changing to a plugin-based approach since I wrote that doc back in February, but that's still the direction I would want to go rather than complicating the core system. Unfortunately, I still don't have any immediate plans or time to implement that, so that doesn't really help you much for Mattermost. :-\

[willnorris]

The other thing you could do is to provide a thin layer that accepts request using whatever API you want (pulling options off query params, potentially even with different parameter names), and then construct a new http.Request that matches what imageproxy expects. That should be doable with no modifications to imageproxy at all, and that thin layer could slowly phase out as imageproxy is updated over time to support the additional options you would need.

[hmhealey]

Yeah, we're hoping to use it as a drop-in replacement, but all we do with the existing proxy is send it image requests without any special options. The main goal of what I'm working on right now is just to make it so that the Mattermost server can spin up its own proxy for people that don't want to set up their own, so we don't need any of those additional options yet.

Since Mattermost's API would have to stay the same (for compatibility and because some people may still want an external proxy), I'm just planning on adding some extra handling to let it forward requests directly to the imageproxy instance that it controls instead of only being able to contact an external proxy.

I haven't decided on exactly how Mattermost would interface with imageproxy yet outside of having Mattermost control the proxy instance. What I was mentioning above with having it call serveImage directly would probably keep that connecting logic simpler, but it would require changes to imageproxy like the ones I mentioned above. Doing some sort of translation layer like you proposed is also something I'm considering, although it's a bit less "nice" since it would be faking that request and such.

[hmhealey]

By the way, creating an instance of the proxy and then faking an HTTP request to it using ServeHTTP seems to work. If you're interested in seeing the hopefully-final code that does that, you can find it here