Preserving request data across a redirect

[gmc444-b]

Hello,

I'm trying to use morepath with some legacy services within my company. Their http protocol uses a single URL for a service (the service name) and everything else is in the payload in JSON. For example, the JSON for a request for a customer record would look like:
{ 'GetCustomer' : { 'id' : 123456 } }
and the JSON for submitting a customer record would look like:
{ 'SetCustomer' : { 'id' : 234567, 'name' : 'FavoredCustomer' } }

I'd like to handle this by redirecting the request to a path that handles the named type. The problem I encounter is that the redirect causes a new request to be formulated, formatted as a GET, and the payload is lost. I could formulate the redirect to contain all the data as variables in the URI, but the payloads could be large, with a complex structure that would be difficult to translate.

How would I go about performing the redirect in a way that preserves the body of the request?

Thanks,

Greg

[href]

I'm not quite sure what you are trying to accomplish, so I maybe you can give me some more information or correct my understanding:

So an API client is sending your original JSON to a single URL, like example.org/old-api and you would like to redirect the client to example.org/new-api/api-method with the same or some modified payload.

In this case, the original client is instructed by you to do a redirect with a certain payload. Your Morepath application would basically act as some kind of translation service.

Am I understanding this right? If not, can you tell me who is the client and who is the server in this HTTP transation? Because so far this sounds to me like it is a HTTP protocol question, not necessarily a Morepath one. That is not to say you shouldn't ask your question, it's just how I currently understand it.

[gmc444-b]

Your understanding is better than mine, actually. I did not realize that the redirect was going back to the client until I read your description and verified that this was happening.

I would like to dispatch the request to the appropriate control logic using the information in the POST, entirely on the server side. For example, if I have:

@App.path(model=GetCustomer, path='getcustomer')
def get_customer():
     ...

I'd like the framework to take care of making the connection and perform the rerouting based on the JSON data.

Does that make sense?

[href]

Ah, so your web application written in Morepath would perform the request to the legacy api, without the client knowing anything about it?

In this case you could simply do another request inside your JSON view, maybe using the requests library. For example:

import requests

@App.path(model=GetCustomer, path='getcustomer')
def get_customer():
     return GetCustomer()

@App.json(model=GetCustomer, request_method='POST')
def view_customer(self, request):
     post = translate_request(request.POST)
     response = requests.post("example.org/legacy-api/get", post)
     return translate_response(response)

You'd basically be acting as a proxy.

The translate_request function would take the POST data as you get it and translate it into the POST data required by your legacy api (if necessary).

The translate_response function would take the response of your legacy api and either return a JSON or return an error response. This would probably involve looking at the status_code of the response and translating it into a webob exception, if the status_code is not 200.

The two translation functions could also be implemented on the model. Say in an execute function that you call for all your models:

class Proxy(object):
    def execute(self, request):
        # do a generic translation

class GetCustomer(Proxy):
     # add something specific for the translation

@App.path(model=GetCustomer, path='getcustomer')
def get_customer():
     return GetCustomer()

@App.json(model=Proxy, request_method='POST')
def view_customer(self, request):
    return self.execute(request)

The Proxy class doesn't need to live on a path as long as its descendants do. That way you don't need to bother much about writing views and can just keep your logic contained in your models.

Does this answer your question?

[gmc444-b]

I'll give this a try. Thanks for your help!

[href]

You're welcome! Let me know how it goes. If I don't hear back from you I'll close this ticket eventually.

[gmc444-b]

This is closer to what I want, but I'm still not quite there. What I'd like is a substitution for the older protocol (everything coming in on one URI, routing off of the posted JSON data) done with morepath. What this sketches out seems more like a protocol adapter between morepath and an existing API/service. Part of my problem is a large set of clients that only understand the old protocol, and a need to develop new services based on the same protocol.

Or maybe I'm missing something in my ignorance?

Thanks,

Greg

[href]

Okay at this point I'm not sure anymore what you need :)

Can you give me a somewhat complete example of how an interaction with your clients and two apis looks like? Like a simple diagram, flow-chart or something that shows who sends what to whom including the request data and the resulting responses.

It seems to me like you don't yet know what you need in terms of this request/response flow. Your old clients need to send requests somewhere (presumably to your old api or to a new service that speaks the old protocol) and your services need to respond with something. Ultimately that's all there is with HTTP, right?

[gmc444-b]

Hi,

The protocol my company has locked onto looks more like RPC than rest. Clients are written to use a single URI for all requests, and provide a JSON payload that describes the desired action. The type of the request is the name of the root at in the JSON payload. So everything is a POST. Most services are written using generated and hand-coded C++. I'm trying to get Python established as an alternative for the server side.

I'd like to use morepath routing to be able to handle the request using the JSON data.
The flow I currently have is like this:

                             Client sends:
                             GET /customers 
                             headers
                             data:
                             { getCustomer : { id : 123456 } }
<---------------------------

Server:
obj_name = list(req.json_body.keys())[0]
vars = req.json_body[name]
if obj_name == "getCustomer":
return App.redirect(req.class_link(GetCustomer,
variables = vars))

Server sends:
REDIRECT /GetCustomer?id=123456
headers
data: (None)
---------------------------->
Client
Sends back the rerouted request:
GET /GetCustomer?id=123456 1.1
headers
data: (None)
<---------------------------

Server does:
@App.path(model=GetCustomer, path='GetCustomer')
def get_customer(id=0):
look up customer
return customer

RESPONSE
headers
data:
{ Customer : {
id : 123456,
name : "John Doe" } }
----------------------->

I'd like to extend Morepath routing so that instead of
routing on the URI, it can route on that root key in
the JSON data. Or at least do the rerouting in such a
way that it avoids the return trip to the client.

So if my service writers could say something like:

@App.json_path(GetCustomer, 'GetCustomer')
@def get_customer(get_customer_obj):
return Customers.get(get_customer_obj.id)

And avoid the redirect, while leveraging the existing
code for handling the routing, I'd be very happy. I'm
just hung up on how to extend the routing in this way.

[href]

Ok let me know if I'm on to something:

All you requests are handled by your server. The clients will hit it on exactly one path. Depending on the JSON data the submit your server will execute some action. This is the behaviour of the legacy api.

Parallel to this legacy API you'd like to implement a new API running in the same Morepath application and of course you want there to be as little shared code as possible.

Am I starting to get this right? Let me know before I go any further :)

[gmc444-b]

It's more the case that there are multiple servers, but yes, clients hit exactly one path, then on the server side, the JSON data determines the action and response. And what I want to do is implement new services in Python that conform to the legacy API.

[href]

Okay I'm starting to get it. I've kind of overlooked your code example:

@App.json_path(GetCustomer, 'GetCustomer')
def get_customer(get_customer_obj):
    return Customers.get(get_customer_obj.id)

You want this code to be called when someone calls /customers with {'GetCustomer': 123456}, right? Then you could basically write new API functions by just using the json_path directive.

[gmc444-b]

That's right. It seems like this should be possible without a lot of effort, but for me working out the Morepath implementation is slow going.

[href]

Alright, what you are looking for then is probably a custom action and some generic handler code. Because you could always just write a simple JSON view for /customers and handle everything inside on a per-case basis.

Obviously you want something more generic. This is where custom actions really come in handy. They basically allow you to create custom decorators which you can override in subclasses (just like you can override @App.path in a subclass for example.

I whipped up an example to show you what I'm talking about:
https://gomix.com/#!/project/telling-door

You can create a remix of this project and try it out yourself. It's a simple example and it could be made more intricate, but I think this shows well enough what I mean.

Also note that with this approach you basically reinvent the wheel a bit. That is you need to do more custom error handling that you would otherwise have to do with Morepath. Maybe there's a cleverer way than this though. I'd say you would figure it out over time though ;)