-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace RMI #1505
Comments
@DanVanAtta thoughts? |
I think you have the right idea here. This would be a very large undertaking though. You'd probably want to draw up a pretty full design before doing much coding. I'd also wonder is there a more standardized format/protocol? I want to avoid custom code as much as possible. |
@ron-murhammer Well using sockets, java is already helping us a lot, building a fully encrypted tcp connection to the server... |
@RoiEXLab I guess sockets and a custom format payload seems kind of outdated. I do mostly web development these days so I'm used to thinking HTTP with JSON/XML/YAML. I realize this is different since we have a desktop client but wondering whether there are socket alternatives and if we should try to move towards a more standard format of JSON/XML/YAML. The 'sendPacket' method just makes me cringe as it would seem as this point there would be some framework to abstract a lot of that. |
@ron-murhammer Ahh that's what you mean... Of course we could send JSON data, but the question I'm asking myself is, if this is taking to much server load, since every "packet" needs to be turned into json and backwards again. Also json is bigger than binary data. |
@RoiEXLab I wonder if something like protobuf would be a good solution: https://developers.google.com/protocol-buffers/ I'd be interested in determining what other online turn based strategy games use. |
The protocol buffer could be a good way to store savegames in the future! It could be used to send data over the network, but would still use sockets and a bytestream containing too much data. |
Protobuf can have some coupling/generation issues. I've worked with it before and was not very impressed. The general idea though is right, we need something to marshal/unmarshal objects over network. I'm in agreement with looking to JSON over HTTP. Jackson has a very nice JSON<->Object mapper, and GSON by google is a good format with a lot of library support. For example, looks like it can be streamed pretty easily: https://sites.google.com/site/gson/streaming, would just need to connect that to a network socket for a proof of concept. One thing to consider as well is if we can do things incrementally. |
@DanVanAtta gson looks very promising, although I'm still convinced my attempt is the most resource saving, network wise. |
Network payload size is not the most important consideration yet IMO. Chances are anything we switch to will be smaller than the object graphs that RMI is sending. We need to keep in mind the transition/migration path in the current code base. A lot of the RMI is done via the Change event objects. I see a migration plan needing to first:
If we then can come up with something that handles nicely the worst case and a couple of other example ones, then with some luck we'll have a pretty clear migration path. |
Curious suggestion. Why does this result in better outcomes for users? |
@simon33-2 we'd be able to update the game engine and lobby independently and without the incompatibility surprises. It's exactly to avoid things like the sudden bot crashes that have been happening of late. |
Exactly! Not 100% independently but better than currently |
Whats the current state on this? |
While reading this again: |
@RoiEXLab Offering some thoughts as requested... 😃 Let me clarify what I meant when I said I was planning on prototyping something related to this. I see several distinct problems that need to be addressed (and I think I'm echoing what you, @DanVanAtta, and @ron-murhammer have already stated here and in other issues):
Decoupling the domain model from the persistence modelThis is what I want to work on first because I'm experiencing how painful it is to be able to do even simple refactorings in the domain model without worrying about breaking backwards compatibility. I've actually done this on two projects in the past (using something very much akin to the Serialization Proxy pattern that has already been discussed), so I think I know the path to take here. To stay focused on the decoupling, I plan to keep Java serialization as the default persistence model and not be concerned about item (2). Overall, it will require more work to do items (1) and (2) in two distinct steps. They could be done at once, but I'll have enough trouble keeping item (1) straight in my head on such a large codebase. 😃 Defining a new persistence modelIf I'm not mistaken, the domain model today is persisted using Java serialization in all contexts (i.e. save games, network, etc.) due to the coupling of serialization to the domain model. Item (1) would allow for the flexibility to use either a single persistence model or multiple persistence models across all contexts. For example, a verbose textual format (that could be subsequently compressed) for save games to allow easy hand editing, and a tight binary format for the network. I tend to agree with the opinions expressed above that JSON is probably a good start if a textual format is desired. @DanVanAtta already pointed out the benefits that existing JSON libraries provide. However, I don't have a good feel for how much data is actually transmitted during network games. So I'm not sure how important it is to save every possible byte in the persistence model representation. I'm hoping this isn't a huge concern for TripleA, as it's much more pleasant to have a persistence format that is easily consumable by humans when it comes time to debug without having to write tools to parse the data. 😃 Defining a new network interfaceThis is necessarily coupled to item (2) because, presumably, the payload of the protocol you choose is generated via a distinct persistence model (when that payload consists of domain objects). As I mentioned in #1613, I would like to see a web client for TripleA. That biases my opinion that the new API should be easily accessible by web clients, but definitely by non-JVM clients, in general. My vote would be for something HTTP-based simply because every platform has access to an HTTP client. However, it doesn't necessarily have to be RESTful. Using REST, RPC, or some hybrid of the two would probably work. Again, I don't have a good feel for the performance implications of TripleA, so I can't address earlier concerns about the overhead of certain protocols. It might take prototyping a few endpoints and then measuring various metrics to give a 👍 or 👎 on any particular proposal. @DanVanAtta proposed above surveying the existing network interface and publishing that information in one place (here? wiki?). I think that's a great idea and would help to reason about the pros and cons of choosing specific protocols. |
@ssoloff
|
@RoiEXLab 👍 Yes, web sockets would be perfectly acceptable from my perspective as they are a (now) common web client technology. I didn't even think to ask how much TripleA network traffic is pushed by the server, but your statement above makes it clear that it's common enough that old school HTTP polling would not be acceptable. 😃 I don't have much experience with this on the Java side, but many Node-based service frameworks encourage you to write your services independent of the protocol, and then provide adapters so that their endpoints can be exposed using REST, web sockets, message bus, etc. That might be something to keep in mind as the networking code is redesigned. It may be that one-size-does-not-fit-all for any single protocol, and some services are better exposed with another protocol (e.g. web sockets for chat but REST for downloading maps). Keeping the services that enable distributed play ignorant of the actual protocol just seems like a Good Thing. 😃 |
Just found out that Java has a websocket client as part of it's standard library. |
Since we probably want to keep up with web standards, and therefore have a secure connection which encrypts passwords etc. we'd need a (sub-)domain pointing at the lobby server in order to generate a valid certificate using let's encrypt. But I agree with @ssoloff First step would be to separate server and non server code... |
There's also a library which adds in WebSocket support with SSL/TLS... https://github.com/TooTallNate/Java-WebSocket |
Sorry to not get full context on this convo, so this is a bit of a drive-by comment. What I think we need is simply a good way to send objects back and forth. EG: we can tell the game to send an object, and on the other side we read an object. The current RMI does that very literally and directly. We need a layer of software in between that introduces a sane inter-change format so we can control that. So basically we need a library that can take a tripleA object, and convert it to an object that we can send over network, and then on the other side we need something that can read and convert up to an object. In json, this would look like a standard The migration path is pretty kind if we take that approach even. We can still for better or worse re-use the existing game infrastructure, just we would send the and read the objects in a new manner. |
👍 We're on the same page here. I think you'll see this is the case in the prototype I intend to submit. Now I just need time... 😃 |
I'm not sure if turning java objects automatically into json is the right way, since this probably introduces the same problem we have with serialization. We can't change the variable names without losing backwards compatibility. |
Also when sending objects automatically, we might end up sending too much data. |
@RoiEXLab In the solution I'm thinking of, it won't happen "automatically." Think of it more like having an adapter that converts between |
Good, I just thought @DanVanAtta was aiming for an automated solution. |
Did you notice any unexpected "underscore" properties in the JSON? Or were they all a straightforward mapping of the object you were sending back in the response? I can't say I've noticed this lately, but I remember in the past, some mappers would assume they were the only ones marshalling/unmarshalling, so they added extra metadata to the JSON to help them do it. 😞 |
The response seems to be very 1:1 with what you would expect. The When interpreting requests we can turn on automatic snake to camel case conversion. My experience with drop wizard has been positive previously. For example, a service endpoint was pretty easy to configure, and notice we return an immutable value object that is automatically converted to JSON for us:
Here are the properties defined in the value class:
Curl output, pretty no-thrills JSON,
|
👍 Cool, looks like any modern JAX-RS/Jackson stack. Just didn't want to see a property in the JSON-encoded form like |
Looks good to me! Keep it up! |
Thanks! |
I was thinking about dropping the whole email notification thing to be honest, I never really played PBEM, but I find the email notifications quite annoying. |
I think the intent was to make the dice server usable for a tourney where some players are more incentivized to cheat. Would an ordinary player be able to follow those steps to verify dice? |
With dice signatures embedded in the savegame, triplea could automatically verify dice rolls |
That is way nicer than email 👍 |
PBEM is rough but dan is correct about why they wanted pbem. To elim cheats. I fully vote my 2 cents for a non pbem @RoiEXLab |
hmm how do i upvote here |
There is also recovery from crashes for which email is useful. Please consider this aspect! Upvote is possible with the smiley face in the top right. |
Ty simon |
i dont think anyone is saying remove pbem. |
it just gets done elsewhere i would hope a separate marti or dice server is a good idea imho |
@DanVanAtta Thinking again about your dropwizard project: While dropwizard might be a good way to achieve your examples, it might not work out for actual game hosting due to the restrictions of HTTP 1.1 |
Thanks for the interest and follow up @RoiEXLab . I was meaning to post similar. For lobby hosting, it perhaps is a very good choice. Game hosting on the other hand would require some re-work. To fake not having pushes, clients would have to requset updates periodically. The game is not quite set up for that, would be an effort. Though, looking at this, it does seem we can change the output stream serializer from an object output stream to a custom output stream writer. Then we'd only need that to write JSON, and create a framework to encode object messages and serialize them back to objects on the other side. |
Closing for now, we now pretty well we want to replace RMI. Specific suggestions are likely best taken up as new issues. |
Just some concluding words, summing up my own opinion:
I'm not sure on wether it's a good idea to use gson or some similar object-to-json serialization framework to encode the data object, because this again limits us to our object class structure, while if we would encode the objects manually, we could easily refactor code without having ro worry about breaking anything + we can make sure no unnecessary data is being sent. |
I think we're on the same page. The exact JSON structure used is an interesting thing to consider, a number of different ways to do that. Though, I do think we have at least made a few steps forward. We may want to track this initiative as a project so we can relate the different ideas/PRs as they come up |
I just looked into the net code, and this is going to be a similar huge project as the JavaFX project...
|
TL;DR: I'm not going to have more than 1 giant project active at the same time: I'm not going to work on this until JavaFX is reaching it's final state, not going to happen soon (although I will continue working on it soon) |
No worries @RoiEXLab , this has been on the radar for some time, we all really would like something that we can update with compile time checks, and let the devs and game players worry far less about versioning. |
@DanVanAtta @ssoloff @ron-murhammer Found this: http://docs.oracle.com/javase/9/rmi/toc.htm |
Note that some of those recommendations may not be implementable as described at the link. We don't actually use RMI™ but a custom network protocol written by Sean that was heavily influenced by RMI™. From /**
* Very similar to RMI
* |
@RoiEXLab Using a type from the |
If you say so... ^^ |
As you may or may not know, tripleA uses the Java RMI system, which stands for Remote Method Invocation for it's internet communication...
I have a more or less concrete idea on how to implement a Network-Communication-System using Java Sockets, but I'm here to discuss the details with you since this is going to be a huge project...
I have a Packet System in mind - see my current implementation for more details.
This Packet systems uses IDs for every action. Currently I have:
Is there anything missing?
How are we going to restart the lobby without killing the java process?
The text was updated successfully, but these errors were encountered: