Client identity storage service #2965

gianluca-nitti · 2017-05-29T22:03:34Z

Contains

The client to synchronize with a client identity storage service, as discussed here in the forum.

How to test

An implementation of the backend server is available here. You can install and configure it locally as described here, or use the public instance I'm hosting here.

To test, register an account on a server by navigating to it's address with a browser (or directly clicking the link above if you use the hosted instance). Then, start the game client (from this branch) and login by going to Settings > Player Settings. Now you can join a game server and as soon as you do, the new identity will be uploaded to the service; you can then start another client with empty/different configuration (e.g. use the 2nd or 3rd client run configurations in IntelliJ, or specify another data directory from the command line) and login into it too; your client identities will be synchronized and if you join the same game server, you will find your character in the same position you left it from the other client.

Outstanding before merging

Marking as WIP since I think there is still some minor improvement that can be done, e.g. localization of some UI messages. Also, the conflict resolving feature (which allows you which identity you want to keep for a certain server in case the locally stored one conflicts with the one stored on the service) is not fully working but this is due to a server issue which doesn't allow to overwrite the identities (will fix it soon).

Update: I just added the localization feature (the strings are in the menu translations files; the values are filled for English only but can be easily be added for the other languages via editing the proper files or - I think - with Weblate if this branch is merged). Also fixed the server to support replacing of already uploaded identities.

Tagging @msteiger since we talked about this recently on appear.in.

For more information about the identity storage service, please see: http://forum.terasology.org/threads/client-identity-cloud-storage-service.1846/#post-14953 This commit by itself is not adding ready-to-use features, but I prefer to break here since a not-so-small refactoring needs to be done before proceeding.

This is used in the login form.

GooeyHub · 2017-05-29T22:13:06Z

Hooray Jenkins reported success with all tests good!

GooeyHub · 2017-05-30T14:31:22Z

Hooray Jenkins reported success with all tests good!

GooeyHub · 2017-05-31T15:09:48Z

Hooray Jenkins reported success with all tests good!

GooeyHub · 2017-05-31T15:48:08Z

Hooray Jenkins reported success with all tests good!

msteiger

Very good PR! Just a few minor things:

Class names should be camel case: GET -> Get and POST -> Post
No wildcard imports, please.

msteiger · 2017-06-02T04:53:16Z

engine/src/main/java/org/terasology/identity/ClientIdentity.java

+
+    @Override
+    public int hashCode() {
+        return playerPrivateCertificate.hashCode() ^ playerPrivateCertificate.hashCode();


I think that this will always be zero. Maybe just just 31 * playerPublicCertificate instead?

Wanted to XOR the public with the private but accidentally typed private two times. I just found there is an Objects.hash for this purpose of building hashcodes for objects composed by multiple fields, would that be OK? Will change again if you prefer the manual implementation.

Also changed the hashCode() implementation in PrivateIdentityCertificate to use this method. I'm not sure if I should change it in PlayerPublicCertificate too since an implementation was already there (I didn't added it in this PR) which simply returns signature.hashCode(); combining with the other fields (id, modulus and exponent) will probably reduce the collision rate but could also slow down the computation of the hash, what do you think about it?

msteiger · 2017-06-02T04:53:36Z

engine/src/main/java/org/terasology/identity/storageServiceClient/Action.java

+/**
+ * Represents an interaction that can be made with the storage service server.
+ */
+abstract class Action {


Either make it an interface or use Consumer<StorageServiceWorker> instead.

Interesting point, making it an interface. In both cases I need to make the method public, however shouldn't be a big deal since both the interface and all the implementor classes are package-private and references to those classes never "leak" outside the package, right?

msteiger · 2017-06-02T04:53:55Z

...e/src/main/java/org/terasology/identity/storageServiceClient/BigIntegerBase64Serializer.java

+
+    @Override
+    public BigInteger deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
+        return new BigInteger(DECODER.decode(json.getAsString().replace("\n", "")));


Why could there be newline chars. in the data?

I know it looks ugly, the problem is that the PostgreSQL base64 encoder used on the server inserts some newlines every a certain numbers of chars. I'll see if I can strip them on the server instead.

msteiger · 2017-06-02T04:54:51Z

engine/src/main/java/org/terasology/identity/storageServiceClient/HttpMethod.java

+/**
+ * Http methods; only the relevant ones for now but can easily be extended with the other standard ones.
+ */
+enum HttpMethod {


Maybe complete the list and make it public? There are not so many HTTP Methods

Ok, adding all the methods according to https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods

msteiger · 2017-06-02T04:55:20Z

engine/src/main/java/org/terasology/identity/storageServiceClient/ServiceAPIRequest.java

+ */
+final class ServiceAPIRequest {
+
+    private static final Gson GSON = new GsonBuilder().registerTypeHierarchyAdapter(BigInteger.class, BigIntegerBase64Serializer.getInstance()).create();


Can you reuse one of the existing instances of GSON? Not sure if it's possible or makes sense ..

I don't think so, I searched for all the usages of GsonBuilder in the project and looks like that all the classes which need JSON serialization make their own instance with the appropriate type adapters.

msteiger · 2017-06-02T04:57:14Z

engine/src/main/java/org/terasology/identity/storageServiceClient/StorageServiceWorker.java

+        }
+        status = StorageServiceWorkerStatus.WORKING;
+        logger.info("Performing action {}", action.getClass().getSimpleName());
+        new Thread(() -> {


Please provide a Runnable to the Thread instead of subclassing it. I hope we have a shared thread pool executor soon, so you can just throw it there.

Please forgive me but I don't understand what you mean here, I'm providing an anonymous implementation of Runnable to the Thread's constructor, not subclassing Thread.

Ah sorry, I misread.

msteiger · 2017-06-02T04:57:40Z

...e/src/main/java/org/terasology/identity/storageServiceClient/StorageServiceWorkerStatus.java

+        return buttonEnabled;
+    }
+
+    public String getLocalizedStatusMessage(TranslationSystem translationSystem, String loginName) {


Those two methods look strange to me. I would move it either to a static helper class or - even better - to the (single?) caller class.

Moving to static helper class since getLocalizedStatusMessage is called in two places (PlayerSettingsScreen and MainMenuScreen)

msteiger · 2017-06-02T04:58:00Z

engine/src/main/java/org/terasology/logic/console/ConsoleSystem.java

 import org.terasology.network.ClientComponent;
 import org.terasology.registry.In;
 import org.terasology.rendering.nui.NUIManager;

 @RegisterSystem
 public class ConsoleSystem extends BaseComponentSystem {
+
+    private static final ResourceUrn MINICHAT_UI = new ResourceUrn("engine:minichatOverlay");


Why is that here now?

I used the MiniChatOverlay to show notifications from the storage service in the menus. If I understand correctly in the menu state there is no ChatSystem so I made some modifications to ConsoleSystem to show certain notifications in MiniChatOverlay. In my opinion it's a good way to notify user when, for example, the synchronization is completed - since it happens asynchronously, I thought that showing message boxes may not be a good idea since they would pop up at random moments possibly annoying the user.
Let me now if something needs to be changed.

That's an interesting idea - I like it. The word "chat" could be confusing though. Maybe we can find a better name... NotificationOverlay maybe? Does not need to happen in this PR though.

I agree, the name "chat" would not be consistent anymore. I did the refactoring anyway in the latest commit since it was just a 5 minute task.

msteiger · 2017-06-02T04:58:26Z

engine/src/main/java/org/terasology/rendering/nui/widgets/UIText.java

@@ -82,6 +82,9 @@
    @LayoutConfig
    protected boolean readOnly;

+    @LayoutConfig
+    private boolean passwordMode;


Many UI frameworks have a separate class (derived from the default text control). Have you thought about creating a UIPasswordText instead? It could be too much duplication, though ..

Yes considered that, but avoided for

too much duplication

exactly this reason. Can move to this approach if you prefer, though.

Ok, fair enough. Thanks for clarifying

msteiger · 2017-06-02T04:58:42Z

engine/src/main/java/org/terasology/rendering/nui/widgets/UIText.java

@@ -177,6 +180,15 @@ public UIText(String id) {
        cursorTexture = Assets.getTexture("engine:white").get();
    }

+    private String buildPasswordString() {


You could use this snippet if you like

char[] arr = new char[33]; Arrays.fill(arr, '*'); String str = String.valueOf(arr);

I was trying to figure out a better approach than using the StringBuilder but nothing came to my mind. Will now replace with your snippet, thanks.

gianluca-nitti · 2017-06-02T10:28:21Z

Thank you for the review! Left some comments and made a commit with various fixes.

GooeyHub · 2017-06-02T10:35:36Z

Hooray Jenkins reported success with all tests good!

Reason: the changes proposed in MovingBlocks#2965 use this widget to show messages from the identity storage service too.

GooeyHub · 2017-06-05T20:12:15Z

Uh oh, something went wrong with the build. Need to check on that

Reason: the changes proposed in MovingBlocks#2965 use this widget to show messages from the identity storage service too.

GooeyHub · 2017-06-05T20:44:44Z

Hooray Jenkins reported success with all tests good!

Previously, the instance of this class was being registered in the main menu state's context, leading to NullPointerException when trying to access the player settings screen from the in-game pause-menu.

GooeyHub · 2017-06-06T15:16:48Z

Hooray Jenkins reported success with all tests good!

gianluca-nitti · 2017-06-06T15:16:55Z

I added one more commit to fix a problem: trying to open the player's settings screen from the in-game pause menu was throwing a NullReferenceException due to the StorageServiceWorker instance (which the player settings screen polls) was being registered in the main menu's context, and thus not accessible from the in-game context.

I apologize if there is something bad in the fact that I added code that nobody asked for after the PR had been approved, possibly making the review/merge process longer, but I think this is still better than letting buggy code to propagate to other branches (develop/master). Unfortunately I didn't noticed this behavior when I tested before opening the PR because I didn't think about opening the settings from the pause menu.

Anyway, now it's fixed: the StorageServiceWorker instance is registered in the rootContext and thus is more decoupled from engine and user interface's states, and the player settings screen can be accessed both from the main menu and from the in-game pause menu.

Cervator · 2017-06-09T04:52:01Z

Tried testing this out, but the server you're hosting seemingly took my registration (again I guess, tried earlier but the site looks much better now) then never sent me a confirmation key. Is it still working? :-)

Also wondering where we should host it. Since our Weblate instance runs out of Docker already maybe we could just add a couple more containers or something? Poking @qwc and @msteiger for ideas

Game itself seemed to run fine and I was able to test it successfully in the past!

gianluca-nitti · 2017-06-09T06:46:08Z

@Cervator : it's OK that it allowed you to "register again" after the previous time you registered because I rebuilt the database about a week ago when I added email verification and reCAPTCHA (the only accounts were mine and your tests, so I guessed it wasn't a big deal; anyway, I have a dump). I had a look at the database now and saw you made an account with email address which is waiting for confirmation.

I tried to register another test account with my email and it worked, I got the activation mail - in the spam folder, however. Did you check yours? The reason it goes into spam, I guess, is that it doesn't come from a well-known/trusted mail service, but from a local SMTP server installed on the VPS, so Gmail thinks it's suspicious.
P.S. even if you find the "old" verification email in spam, if you want to retry you probably need to redo the registration (same username and email is fine) because an unverified account expires after 1-2 hours (not sure about the exact value now).

msteiger · 2017-06-18T16:27:56Z

Can we merge this one, @Cervator?

Cervator · 2017-06-19T01:27:17Z

@msteiger certainly - checked it out again and rested, got the confirmation email via junk mail, confirmed, and was able to validate the identity carried over :-) You don't need to wait for me if you have approved and feel you've tested decently, in that case merge ahead and just assign the PR (and any related issues) to the current milestone

Great work @gianluca-nitti !

One small suggestion: Make the UI element a little more consistent between different screens - the little thing in the top right is only visible on the main menu? Maybe it should be there on the settings screen etc too?

We should probably also find some places to update the docs, and a good on-going way to host the service. Will bump the forum thread :-)

Thanks again!

qwc · 2017-06-19T07:32:20Z

@Cervator Of course, spinning up docker containers is the easiest. That's the fancy thing with docker.
But we should think twice regarding resources usage, the weblate droplet may be to small for things like elastic search, etc.

gianluca-nitti added 11 commits May 19, 2017 23:23

Implemented internals for identity storage service.

34ebf1b

GUI for interaction with identity storage service.

3bf6b38

Improve client for identity storage service

a313c11

Implement password-style option for UIText

c92d0a5

This is used in the login form.

Identity storage client now logs messages to the in-game console.

4f8b411

Refactor identity storage service client.

320b258

Move client identity login/logout button to player settings screen

1f53b38

Show console notifications in the chat overlay

bb4837c

Bug fix: now the mini chat overlay isn't shown empty when entering game

df2f166

Implement conflict management for client identity storage service.

332404e

I18N (localization) support for messages from identity storage service.

670a395

gianluca-nitti added this to In progress in GSOC 2017 - Server API May 30, 2017

Refactor client identity conflict solving.

9e7d1da

Prompt user to delete local client identities when logging out.

38f2b43

gianluca-nitti changed the title ~~[WIP] Client identity storage service~~ Client identity storage service May 31, 2017

msteiger suggested changes Jun 2, 2017

View reviewed changes

Implement changes requested by @msteiger on PR MovingBlocks#2965

dd8a727

gianluca-nitti moved this from In progress to Pull requests in review in GSOC 2017 - Server API Jun 2, 2017

msteiger approved these changes Jun 5, 2017

View reviewed changes

gianluca-nitti added a commit to gianluca-nitti/Terasology that referenced this pull request Jun 5, 2017

Rename minichatOverlay to notificationOverlay

3470168

Reason: the changes proposed in MovingBlocks#2965 use this widget to show messages from the identity storage service too.

Rename minichatOverlay to notificationOverlay

4ed1039

Reason: the changes proposed in MovingBlocks#2965 use this widget to show messages from the identity storage service too.

gianluca-nitti force-pushed the identity-storage-client branch from 3470168 to 4ed1039 Compare June 5, 2017 20:35

Move StorageServiceWorker to rootContext

0927338

Previously, the instance of this class was being registered in the main menu state's context, leading to NullPointerException when trying to access the player settings screen from the in-game pause-menu.

gianluca-nitti mentioned this pull request Jun 13, 2017

Authentication management MovingBlocks/FacadeServer#2

Merged

Cervator merged commit 0927338 into MovingBlocks:develop Jun 19, 2017

Cervator added a commit that referenced this pull request Jun 19, 2017

Merge PR #2965 by @gianluca-nitti - client identity server support

1f9eeb8

Cervator added this to the Alpha 8 milestone Jun 19, 2017

gianluca-nitti moved this from Pull requests in review to Merged/completed in GSOC 2017 - Server API Jun 19, 2017

Client identity storage service #2965

Client identity storage service #2965

Conversation

gianluca-nitti commented May 29, 2017 • edited

Contains

How to test

Outstanding before merging

GooeyHub commented May 29, 2017

GooeyHub commented May 30, 2017

GooeyHub commented May 31, 2017

GooeyHub commented May 31, 2017

msteiger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gianluca-nitti commented Jun 2, 2017

GooeyHub commented Jun 2, 2017

GooeyHub commented Jun 5, 2017

GooeyHub commented Jun 5, 2017

GooeyHub commented Jun 6, 2017

gianluca-nitti commented Jun 6, 2017

Cervator commented Jun 9, 2017

gianluca-nitti commented Jun 9, 2017

msteiger commented Jun 18, 2017

Cervator commented Jun 19, 2017

qwc commented Jun 19, 2017

gianluca-nitti commented May 29, 2017 •

edited