Mark messages as received-by-gateway so they are not included in the next response

[estellecomment]

From medic/cht-core#3073 :
Modify medic-gateway to call medic-api and change the status of received messages to received-by-gateway after they have been successfully written to the database. This will stop them being sent again next query and reduce the bandwidth requirements.

This will avoid step 4 in the current protocol

To do it, have gateway set the needs forwarding flag to true when storing a new message in DB.

[alxndrsn]

@estellecomment It look like you've done most of the work for this. Is it in progress, or do you want me to finish it off?

[estellecomment]

I haven't given up on it yet :)
I unassigned myself because I'm doing something else for now, and I don't know how much of a hurry you're in. If it's urgent you can finish it, otherwise I will today/tomorrow.

[estellecomment]

@alxndrsn Can you review?

[alxndrsn]

PR test failures are things like this:

medic.gateway.alert.SettingsDialogActivityTest > generic_shouldDisplayCorrectFields[test(AVD) - 5.0.2] FAILED 

	java.lang.RuntimeException: Waited for the root of the view hierarchy to have window focus and not be requesting layout for over 10 seconds. If you specified a non default root matcher, it may be picking a root that never takes focus. Otherwise, something is seriously wrong. Selected Root:

The espresso tests intermittently fail on Travis with errors like this. I'll re-run and see if they pass the second time.

[garethbowen]

@alxndrsn What's the status of this? Did it fail AT?

[estellecomment]

nah, I merged it but @alxndrsn hadn't reviewed it so reverted it.
It got split into two PRs :

• code : 3408 sms protocol remove duplicate sending #107 is breaking, I have to figure out why
• unrelated tests : Extra tests. #106 is supposed to be all tests but apparently it's not, so gotta sort that out.

I have higher priority stuff that keeps happening, but it's still on my list.

[garethbowen]

@alxndrsn Would you mind finishing this off while @estellecomment is away?

[alxndrsn]

I didn't have time to refamiliarise myself with the original ticket, and the information about motivation here seems sparse. I would like to fully consider whether this is useful, in line with discussions about the protocol and recent work on the server-side by @SCdF. But if we do want to achieve this, PR at #113

[garethbowen]

Delaying as this is a performance improvement only.

[latin-panda]

Moving to AT, PR: #160
This branch is based on the branch that solves issue: #139
Please review and get #139 merged first before AT this.

[latin-panda]

Summary

The issue is when API send to Gateway a message (sms), this message was being saved as "not needing forwarding", so its status wasn't updated as "received by gateway" and API might try to send it again to Gateway.

The fix is:
The SMS status should be set as Needs Forwarding when updating sms statuses or when saving new SMS to Gateway DB. Then Gateway will notify the status change to API from the beginning, reducing the chance of API to resend the same SMS to Gateway.

It's important to note that API <-> Gateway communicates in intervals of time, every 2 or 5 minutes.

[newtewt]

@latin-panda I think this is ready to go.

I've pulled the sqlite database and see the message is marked with needs_forwarding of 1. Once the attempt to send and it eventually fails. The needs_forwarding is marked as 0. Looking at the status_history on the message in couchdb. I see the proper flow through. As shown below. Also, have ensured on my phone with service that the messages are being sent properly.

      "state_history": [
        {
          "state": "pending",
          "timestamp": "2020-09-28T14:28:06.231Z"
        },
        {
          "state": "forwarded-to-gateway",
          "timestamp": "2020-09-28T14:28:58.170Z"
        },
        {
          "state": "received-by-gateway",
          "timestamp": "2020-09-28T14:29:58.089Z"
        },
        {
          "state": "forwarded-by-gateway",
          "timestamp": "2020-09-28T14:29:58.089Z"
        },
        {
          "state": "failed",
          "state_details": {
            "reason": "generic; errorCode=-1"
          },
          "timestamp": "2020-09-28T14:29:58.089Z"
        }

[garethbowen]

@latin-panda Can this issue be closed now?

[latin-panda]

Hi @garethbowen, yes, it's in "done" status and code was merged :)