support GWT-RPC with superdevmode

[dankurka]

Originally reported on Google Code with ID 7522

GWT-RPC currently doesn't work because the compiler running in the CodeServer generates
new .gwt.rpc files and these aren't found on the separately-running server.

Workarounds:
- after compiling in Super Dev Mode, manually copy the .gwt.rpc files to the war directory
of your other server.
- alternately, modify your server to load the .gwt.rpc files from the code server (since
it's available over http).

Discussion:
https://groups.google.com/forum/#!topic/google-web-toolkit/ZugfWpwnXJs/discussion

Current as of 2.5 RC 1.

Reported by skybrian@google.com on 2012-07-16 17:09:01

[dankurka]

Would it be possible to add an optional method in RpcServlet, .getCodeserverHost(),
which allows the specification of the code server url to proxy to?  If specified, look
there first, and if it 404s / IOExceptions, default to standard .findClientOracleData()?

It could default to return a well-known system property, "gwt.codeserver", and users
can override manually in their rpc servlets, if they happen to use multiple gwt modules
on multiple codeservers.

The only problem then is just knowing when to clear out the clientOracleCache.  header-checks
for etags would still be quite expensive per request; another option would be to hook
up a callback for the codeserver to tell the rpc servlet to clear it's cache.  After
every recompile, it could ping a header to every servlet that has been asking for .rpc
files, which can just clear their caches (and silently eat the request).

The latter option would also mean no more reloading the server every time you change
any classes that affect rpc serialization. =}

Reported by James@wetheinter.net on 2012-09-19 12:38:48

[dankurka]

There are a couple of different use cases that we should support:

1. The client and server are both under the developer's control and running on the
same workstation. The developer can make changes to both client and server code and
reconfigure the server.

2. The server is running in the cloud (say on App Engine) and the developer is working
on a laptop with code server on localhost, and just wants to debug client code with
an existing app server (maybe even production). So we can't assume that the app server
can load a URL from the code server, and ideally we shouldn't have to make any config
changes to the app server that aren't appropriate for production.

Also, keep in mind that we still want the original GWT app to work (without Super Dev
Mode on).

Having the app server load a URL from the code server will work for the first case
but not the second. Maybe solving the first one is good enough for many people to move
forward, but I'd like to support both when the serialization classes haven't changed
too much. I wonder if this means using some less fragile format for the GWT-RPC protocol?
Maybe we need to dig into how GWT-RPC works before picking the best solution?

Reported by skybrian@google.com on 2012-09-19 19:53:07

[dankurka]

I agree that support bi-directional request proxies are, in the end, likely the best
solution.

I already have a working override for the WebServer which mindlessly proxies any requests
the code server misses to an arbitrary proxy url (post and get, using the repackaged
jetty httpclient).  It is fully functional (albeit a little slow, since every ajax
request is taxed), although it doesn't yet handle recompiled .rpc files.  I will be
releasing my codebase on github within a week or two, and will include the super dev
proxy as a working reference copy for anyone that wants to try experimenting with support
before the team creates an official implementation.

One idea on debugging client against a live remote server; if we had a way to add signed
request headers that can tell the RpcServlet where to find the correct .rpc files (and
implement a white-list only approach for security), we could, in theory, allow the
remote server to request the .rpc files on the fly...  Although, given that they have
to deserialize into java objects in the server, it will never be so simple as just
updating the .rpc files; if the objects change in breaking ways, the server needs,
at the very least, updated class files; and if you're updating the classes, you might
as well update the .rpc files too!

The more I think about it, the more I don't like telling the live server to use any
.rpc files other than what was uploaded with it; if you need to change rpc, you need
to update the live app.  The proxy I have now can do this, and push all headers + cookies,
though I haven't tested it on anything but simple implementations.

I think, for the time being, simply proxying requests from code server to application
server will give a good "no config in prod" option for developers, and maybe a simple
CodeServerFilter to allow the app server to request static files from the codeserver,
so devs don't pay an Ajax Tax on every request.

Whatever the team feels is the best solution,
I am willing to write and distribute a prototype.

Reported by James@wetheinter.net on 2012-09-19 21:47:40

[dankurka]

I'm not understanding something. I already fixed basic GWT-RPC calls to work and I have
a test for that (without editing code). That's when I introduced  GWT.getModuleBaseURL()
versus GWT.getModuleBaseForStaticFiles(). So it seems to me that a proxy shouldn't
be necessary, since the client code should just use the right method to get the base
URL so it hits the right server. (They point to the same place when not in Super Dev
Mode.)

Reported by skybrian@google.com on 2012-09-19 22:00:57

[dankurka]

Ok, my bad.

I'm not really "in the loop".  I just pull down trunk once in a while and write whatever
overrides I need to work on my test projects.

I will look into removing the need for proxies entirely in my own code, and try to
stay updated on everything tagged SuperDevMode in the issue tracker.

Correct me if I'm wrong, but using GWT.getModuleBaseForStaticFiles will work so that
hitting the application server will cause the clients to load the gwt app from the
code server.  To work in production / qa, you could probably still use this with your
ip and some port forwarding...

So, I guess, running the app through the code server and proxying requests to the app
server really isn't necessary; although, it does have the added perk that it can point
to any app server without modifying production code.

My apologies for needlessly suggesting an over-complication on this issue. =}

Reported by James@wetheinter.net on 2012-09-19 22:09:16

[dankurka]

Well, the proxy might still be useful for the time being, particularly if you haven't
turned on the Super Dev Mode hook in the production server. Eventually it should be
on by default (just as with classic dev mode), but we aren't there yet.

For this bug, we need to figure out what's working or broken for GWT-RPC. I've seen
basic RPC calls work, but it doesn't work for some people.

Reported by skybrian@google.com on 2012-09-19 22:45:04

[dankurka]

Do we have any test cases / bug reports of failed calls?

If I can replicate the issues, I can flesh out the proxy I have and report back on
feasibility / workarounds; being that it copies all headers, params and post body,
I have a hard time imagining how it could fail for anything (I currently use it to
mock authenticate to various servers; any of them that don't verify Host v. referrer
work just fine). 

Reported by James@wetheinter.net on 2012-09-19 23:03:01

[dankurka]

As far as I know, the serialization policy not matching after a change is the only issue.
Fixing issue 7788 probably helped some, since it made policy generation more stable.

Reported by skybrian@google.com on 2013-01-24 02:00:04

[dankurka]

What about implementing a gwt module that defines the servlet which routes to recompile
requests?  So if you want to add the codeserver to a dev module, you could just inherit
the code server, which imports the module, and mounts the recompiler into whatever
normal enviro should be handling your rpc?

Then, all we'd have to do is make the recompile event invalidate cached .rpc files...
Perhaps by setting a system property to the latest version number?

Reported by James@wetheinter.net on 2013-01-24 14:39:02

[dankurka]

Wait; I guess if it's based on the permutation hash, we wouldn't need to bother, would
we?  I'll play with it on the weekend and see if I can come up with an integrated solution.

Reported by James@wetheinter.net on 2013-01-24 14:40:33

[dankurka]

First step towards fixing this committed at r11555

Reported by t.broyer on 2013-03-14 08:30:45

• Status changed: Started

[dankurka]

I'm declaring this done (for now). Relevant changes: r11571 r11576

Here's the fix: when you start the web app containing your RemoteServiceServlets, you
can set the gwt.codeserver.port Java system property to the code server's port (running
on localhost) and it will automatically download serialization policies from there
if it can't find them otherwise.

Also, RemoteServiceServlet has a couple of methods to override if the default implementation
isn't good enough: getCodeServerPolicyUrl() and loadPolicyFromCodeServer()

Reported by skybrian@google.com on 2013-04-02 20:14:20

• Status changed: FixedNotReleased
• Labels added: Milestone-2_6