Bug? Data API v3: GET entries endpoint has inconsistent response

[nakedsushi]

version r.4208 (7.0.1)

Doing the same GET request to /mt/mt-data-api.cgi/v3/sites/1/entries?fields=author,basename will return different responses even if no entries were added or deleted.

Steps to reproduce:

1. Make an GET request to /mt/mt-data-api.cgi/v3/sites/1/entries?fields=author,basename, take note of the response.
2. Make the same request again. Look at the response, the same number of items are returned, but the parameters in the items are usually missing or different.

I don't think it's a memcached error because I have disabled memcached and this still keeps happening.

[ghost]

@nakedsushi
Thank you for your comment. I could not reproduce it.
Does your MT use 3rd party's plugin or addon? If so, please disable them and check again.

[nakedsushi]

@miniuchi that's the tricky part. It's not a consistent behavior. Normally, it does not show up, but whenever I use the Data API to create a lot of entries at once (serially), after a while, creating the entries silently fail (entry is created but with blank body and title) and listing the entries shows this bug. After an undetermined amount of time, everything goes back to normal and the bug disappears.

I'm running the Amazon MT7 AMI with default settings (connected to MariaDB), memcached disabled. I wonder if there's some sort of caching going on in the perl module or a fallback when the server is overloaded.

No other plugins or addons installed.

[nakedsushi]

More clues. I wonder if it's the way starman is set up? When the bug starts happening, if I restart starman with systemctl restart movabletype.service then it goes away -- until the next time I insert a bunch of entries via the API.

[update]: Can confirm this is effected by how starman starts up. When I change the max-requests=1000 the bug shows up less frequently and if it does show up, more requests are processed before the bug shows up. Changing it from 100 to 10 will almost immediately cause the bug to show up when I start my import using the Data API. I'm not at all knowledgeable about how starman restarts workers, but I wonder if when the worker restarts after processing the max number of requests, it's failing to load everything into movable type. So a request to the Data API sometimes will hit one worker that is valid, and sometimes hit the other (recently restarted) worker with an buggy version of MT.

[ghost]

@nakedsushi
Thank you. I will investigate it again on the AMI.

[ghost]

@nakedsushi
I couldn't reproduce it on AMI... Could you give me more detail procedure to reproduce it if possible?

[nakedsushi]

Steps to reproduce:

1. Set up /data/conf/movabletype.conf like this:

WORKER=2
MAXREQUESTS=10

1. Restart the movabletype service with systemctl restart movabletype.service

2. Create a script to post 1000 new entries to the DataAPI entry endpoint here: https://www.movabletype.jp/developers/data-api/v3-reference.html#entries

3. Run the script so that it does the POST requests serially.

4. BUG: some of the entries are correctly created, others have blank titles and bodies.

5. BUG: after the bug above happens, try listing the entries with the list entries endpoint:
   ../mt/mt-data-api.cgi/v3/sites/SITE_ID/entries sometimes all the fields for the entries are returned and sometimes they are blank.

[ghost]

@nakedsushi
Thank you.

I tried to reproduce it with the following script and MT7 nginx AMI (t2.medium), but couldn't.
https://gist.github.com/miniuchi/3551252947985fd6dc4c5baecc306df5

I wrote this script with reference to #369 (comment).
Am I misunderstanding your procedure?

[nakedsushi]

@miniuchi You're right, I can't reproduce it now either on my AMI with your script. I'll try it again when I get back into work with my full script and see what the difference is. Thanks for working on this!

[nakedsushi]

Update: I haven't been able to consistently reproduce it with my own script. Sometimes it happens, sometimes it doesn't and setting the number of MAXREQUEST to something low like 10 sometimes works and doesn't. I'll play around with it some more and see if I can get more consistency with it.

[ghost]

@nakedsushi
Thank you for your investigation!