Rsp 500 based on submitted time

[neilh10]

I get a repeated response of 500 to a POST.
This suggest a server error occurred, and its nice that I get reasonably fast response.
Is it possible that the POST has a time threshold that if the time on the input is before this it is rejected/ causes a problem ?

The condition is that the Mayfly has time before it syncs with NIST and then gets updated later. All readings taken are entered in to a queue for reliable transmission.

Example below when response is 500 for
"2019-12-31T16:02:00-08:00"
"2020-01-01T15:02:00-08:00"

and 201 when time has been synced for "2022-09-15T15:50:20-08:00"

POST /api/data-stream/ HTTP/1.1
Host: monitormywatershed.org
TOKEN: 26d1accb-ba26-404a-8913-482ff0bc4c6d
Content-Length: 415
Content-Type: application/json

{"sampling_feature":"4ab8e4f6-908c-42fc-a6f9-a4c7791c5350","timestamp":"2019-12-31T16:02:00-08:00","f23fa230-9ea0-4ead-a9f8-07e52365b07b":1,"c53ddd73-72b4-428a-b3b5-23cad9ccf65d":4.124,"1d42edba-a60c-4826-b697-6c777181ee7f":-9999.00,"d7c8cc88-708e-4e25-9570-b7cd461279fd":-9999.0000,"fedcae0c-0a0a-4a89-8682-b4135c261934":42.59,"bf133560-ae74-4bfa-b42c-27e7af33cc55":28.94,"f50657dd-7f17-418c-ad31-5fe0e8972cad":-1}

-- Response Code -- 500 waited 2300 mS Timeout 7000

Two similar examples, from a reliability QUEUE -
POST /api/data-stream/ HTTP/1.1
Host: monitormywatershed.org
TOKEN: 26d1accb-ba26-404a-8913-482ff0bc4c6d
Content-Length: 409
Content-Type: application/json

{"sampling_feature":"4ab8e4f6-908c-42fc-a6f9-a4c7791c5350","timestamp":"2022-09-15T15:50:20-08:00","f23fa230-9ea0-4ead-a9f8-07e52365b07b":1,"c53ddd73-72b4-428a-b3b5-23cad9ccf65d":4.139,"1d42edba-a60c-4826-b697-6c777181ee7f":20.08,"d7c8cc88-708e-4e25-9570-b7cd461279fd":0.2924,"fedcae0c-0a0a-4a89-8682-b4135c261934":31.45,"bf133560-ae74-4bfa-b42c-27e7af33cc55":32.02,"f50657dd-7f17-418c-ad31-5fe0e8972cad":-69}

-- Response Code -- 201 waited 4301 mS Timeout 7000
Sent 4 readings in 101.22 sec. Queued readings= 1
POST /api/data-stream/ HTTP/1.1
Host: monitormywatershed.org
TOKEN: 26d1accb-ba26-404a-8913-482ff0bc4c6d
Content-Length: 411
Content-Type: application/json

{"sampling_feature":"4ab8e4f6-908c-42fc-a6f9-a4c7791c5350","timestamp":"2020-01-01T15:02:00-08:00","f23fa230-9ea0-4ead-a9f8-07e52365b07b":674,"c53ddd73-72b4-428a-b3b5-23cad9ccf65d":3.987,"1d42edba-a60c-4826-b697-6c777181ee7f":19.92,"d7c8cc88-708e-4e25-9570-b7cd461279fd":0.2922,"fedcae0c-0a0a-4a89-8682-b4135c261934":32.42,"bf133560-ae74-4bfa-b42c-27e7af33cc55":31.73,"f50657dd-7f17-418c-ad31-5fe0e8972cad":-69}

-- Response Code -- 500 waited 2808 mS Timeout 7000

Clearly the Mayfly time needs to accurate, but I would expect the server to accept a reasonably recent time stamp.
If there is any filtering by time stamp I would expect it to be identified, and ModularSensors and the server need to be in sync as to what that time threshold is.

[neilh10]

I'm trying some solutions to managing a -- Response Code -- 500 from MMW.
My upload algorithm looks for a '201', and if it doesn't receive it, it queues it on the Mayfly for retry.
On processing the queued entries, on retry whenever it doesn't receive a '201' it stops processing. Typically this is a '504'
However for some records/POSTs I always get a '500' and the response is fast ~ 625mS.
So a server failure 500 blocks all further uploads

The first issue is what is causing the 500?

One of the challenges is I don't have a view for how the MMW/server accepts and stores new records.
It could be that it is stored as a key based on the "timestamp", and possibly the '500' is from a key already existing.
I've tried re submitting for the following with only the sequence number changed from 438 to 4380 - and its still accepted
{"sampling_feature":"236c674b-69b9-43af-b0d6-33d67b870ecc","timestamp":"2022-10-22T07:06:00-08:00","8c57835f-a32f-4d62-82dc-0ba09f04cf52":4380,
-- Response Code -- 201 waited 627 mS Timeout 15000

Or it could be a that it is a time filter -
when I try and it does seem to reject an older date.
[2022-10-22 10:50:04.841] {"sampling_feature":"236c674b-69b9-43af-b0d6-33d67b870ecc","timestamp":"2019-12-30T12:00:00-08:00","8c57835f-a32f-4d62-82dc-0ba09f04cf52":4381,
[2022-10-22 10:50:07.198] -- Response Code -- 500 waited 555 mS Timeout 15000

Trying 2021-01-02T00:00:00-08:00 still gives 500 continuously
[2022-10-22 13:17:19.098] {"sampling_feature":"236c674b-69b9-43af-b0d6-33d67b870ecc","timestamp":"2021-01-02T00:00:00-08:00","8c57835f-a32f-4d62-82dc-0ba09f04cf52":4381,
[2022-10-22 13:17:21.425] -- Response Code -- 500 waited 530 mS Timeout 15000

Trying 2022-01-02T00:00:00-08:00 success 201
[2022-10-22 13:21:00.546] {"sampling_feature":"236c674b-69b9-43af-b0d6-33d67b870ecc","timestamp":"2022-01-02T00:00:00-08:00","8c57835f-a32f-4d62-82dc-0ba09f04cf52":4382,
[2022-10-22 13:21:02.497] -- Response Code -- 201 waited 698 mS Timeout 15000

Possible this is a fix for something, rejecting unreasonably early dates - is it a fixed date or a rolling date.?

[neilh10]

I guess this is one of those too hard to figure out with all the "Frankstein scripts" that hold the system together.
Its could be related to this #665
So I'm closing it due to lack of any insighs