Failable chat messages

[placer14]

Chat messages are currently persisted after successfully sending the message to the remote peer. In the cases where the remote peer is offline or the node is in a high-latency environment a message may take several minutes to be successfully sent. In these cases, the chat messages appear to be persisted immediately after posting the message, but if a user were switch away and revisit the view, the recently posted message may not appear if the message has not been successfully sent at the time.

Solution:

• we return from posting new chat messages as soon as possible (without waiting for the send to complete) ensuring that the message ID is returned in response
• ensure the websocket returning early with the completed state of the chat request does not race the requesting call (prefer to ensure a race condition can be handled (see Add ID Parameter to Async Socket Calls #925) or design to prevent the race condition if we can't)
• we will reorder the operations in our chat API (within {specify api endpoints to include groupchat as well as regular chat}) to persist the message to the local DB immediately regardless of the successful delivery of the message.
• we will migrate our schema to add a failed_reason non-nullable text column to the chat DB table which describes the reason for the message delivery failure.
• unsuccessful message delivery should populate DB record(s) w failed_reason with the cause of the failure
• unsuccessful message delivery should generate a socket notification (body to be defined)
• successful message delivery should generate a socket notification (body to be defined) (optional)
• /ob/chatmessages response should include a new key to present this failed state to the UI

    {
        "message": "Tes #8",
        "messageId": "QmWepAACQP3vcoXRrhz1HNftiE5G3eRKZZtgikwnXhabUE",
        "outgoing": true,
        "peerId": "",
        "read": false,
        "failed": // string or bool?
        "subject": "QmRWDYgYTxqAJk4URoxJD2NvzkrJ3vtSWEoZY1WmS5esAs",
        "timestamp": "2018-08-16T11:38:58-04:00"
    },

UI impact:

• Must represent failed state (and reason?) to the user
• Must accept failure/success message delivery socket notifications
• Must return immediately to minimize open-connections within the UI

Why use a nullable string instead of just using a zero value string?

[placer14]

To have the schema represent the optionality of that data, but it's not a strong requirement. We can use an empty string. I'm also not strong on any of the names. I might make the response key failedReason if we decide on a string type there.

[jjeffryes]

I think we'd want a socket message when the send fails as well. Otherwise clients would need to re-call the chatmessages api every so often after sending a message until they knew whether it succeeded or failed.

[jjeffryes]

If we did have a socket response for a sent message in the case of failure, would it be any more difficult to have one for both success and failure? That way clients would be able to mark the message as "sending" until the socket comes back, and then make it clear if it was sent or not in close to real time.

[placer14]

I've added both socket notifications to the requirements. The success case shouldn't be much more difficult to add, but left it as an optional feature to consider during planning.

[rmisio]

ensure the websocket returning early with the completed state of the chat request does not race the requesting call (prefer to ensure a race condition can be handled (see #925) or design to prevent the race condition if we can't)

Just some background on this. What we were seeing is that even though in the server code, the response to HTTP call was before any socket (TCP) response, the TCP response was at times beating the HTTP response. If that's true, I'm not sure what the server could do (short of implementing some type of HTTP ack system - let's not do that!). What we did then is for these "async" calls is update them so the client can provide a requestId when calling the http api and that id would be sent back in any sockets that the server sends back as a result of that call. Since the client is providing the id, the client doesn't need to wait on the http response for it, which makes any tcp responses beating the http response a non-issue.

[mikakun]

first congratulation to open bazaar dev; you are creating something amazing possibly world changing with the potential of empowering the people vs multinational. keep up the good work.

this being said, in my opinion the messaging system is a huge problem, not efficient enough to conduct transactions smoothly. messages comes and go, also happening : you write something then 2 hours or 2 days later you get a message that was sent before that. sometimes it is fine when both are online at the same time but that is very rare. it is mostly i send stg then wait between 5hours & 2 days to receive a message shown as sent 5hours or 1 day before.

again it's only my opinion but communication is crucial between vendor/customer & an efficient chat / alternative message system should be high priority as transaction are slow down a lot + it creates trust issues

I don't know the system architecture and have no idea how to do that p2p without a centralised server, sorry. but if you have a server maybe worth setting up with encryption freenode or xmpp as alternative at least until open bazaar chat is more efficient ?