BUG: iPhone app cannot retrieve playlist tracks from API

[adrienjoly]

Reported by Arasg:

quand je veux aller dans mes playlists, j'ai un message qui me dit que je n'ai pas encore ajouté de titres alors que c'est faux...38

Continuation of openwhyd/openwhyd-ios#1

=> API calls made by the iOS app are to be tested on the back-end

[adrienjoly]

resolution steps:

• write a failing API test that creates a playlist with a track, then checks that the playlist contains that track, using same API call as the iPhone app
• make the test pass by fixing the problem
• push fix to production and ask Arasg if it solves the problem

[adrienjoly]

For reference, the urls of the iOS app's API calls are in that file: https://github.com/openwhyd/openwhyd-ios/blob/d700d5b45242301591fb5f098d1be4153b54f2b2/Whyd/Config.h#L125

[adrienjoly]

Getting the following error in production, when trying to open a playlist from the iOS app:

$ tail -F appd.err

===
Thu, 17 Aug 2017 13:54:02 GMT Error
    at Console.error (/home/adrien/openwhyd/whydJS/app.js:23:45)
    at /home/adrien/openwhyd/whydJS/app/models/post.js:78:13
    at handleCallback (/home/adrien/openwhyd/whydJS/node_modules/mongodb/lib/utils.js:120:56)
    at /home/adrien/openwhyd/whydJS/node_modules/mongodb/lib/cursor.js:857:22
    at handleCallback (/home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/cursor.js:171:5)
    at /home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/cursor.js:585:22
    at queryCallback (/home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/cursor.js:212:16)
    at /home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/connection/pool.js:469:18
    at _combinedTickCallback (internal/process/next_tick.js:73:7)
    at process._tickCallback (internal/process/next_tick.js:104:9)
model.fetchPosts ERROR on query { uId: '4d94501d1f78ac091dbc9b4d', 'pl.id': 61 } with params { sort: [ [ 'order', 'asc' ], [ '_id', 'desc' ] ],
  limit: '1000,201' } : { MongoError: Failed to parse: { find: "post", filter: { uId: "4d94501d1f78ac091dbc9b4d", pl.id: 61 }, sort: { order: 1, _id: -1 }, limit: "1000,201" }. 'limit' field must be numeric.
    at Function.MongoError.create (/home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/error.js:31:11)
    at queryCallback (/home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/cursor.js:212:36)
    at /home/adrien/openwhyd/whydJS/node_modules/mongodb/node_modules/mongodb-core/lib/connection/pool.js:469:18
    at _combinedTickCallback (internal/process/next_tick.js:73:7)
    at process._tickCallback (internal/process/next_tick.js:104:9)
  name: 'MongoError',
  message: 'Failed to parse: { find: "post", filter: { uId: "4d94501d1f78ac091dbc9b4d", pl.id: 61 }, sort: { order: 1, _id: -1 }, limit: "1000,201" }. \'limit\' field must be numeric.',
  waitedMS: 0,
  ok: 0,
  errmsg: 'Failed to parse: { find: "post", filter: { uId: "4d94501d1f78ac091dbc9b4d", pl.id: 61 }, sort: { order: 1, _id: -1 }, limit: "1000,201" }. \'limit\' field must be numeric.',
  code: 9 }

... while processing that request:

$ tail -F appd.out

=== Thu, 17 Aug 2017 13:54:02 GMT GET /u/4d94501d1f78ac091dbc9b4d/playlist/61 (userLibrary.controller) {"format":"json","limit":["1000","20"],"id":"4d94501d1f78ac091dbc9b4d","playlistId":"61"}
fetching user  { _id: 4d94501d1f78ac091dbc9b4d } ...
fetchPlaylists: 43.156ms
fetchStats: 22.654ms
fetchLikes: 1.150ms
fetchNbTracks: 8.472ms
=> fetched 0 posts
fetchAndRender: 3.656ms

Anomaly: as seen in appd.out, the GET parameters of the request are parsed into an object that contains an array of two values for the limit parameter: ["1000","20"] (it should be just one integer value. 1000, according to https://github.com/openwhyd/openwhyd-ios/blob/d700d5b45242301591fb5f098d1be4153b54f2b2/Whyd/Config.h#L149)

Consequence: in the corresponding db query, the value of limit is passed as "1000,201", whereas mongodb expects an integer. Probably because the limit array (["1000","20"], as parsed from GET parameters) were casted into a string (=> "1000,20"), thus adding 1 to it resulted in a concatenation (=> "1000,201").

Diagnostic:

• nothings seems wrong on the iOS code base: https://github.com/openwhyd/openwhyd-ios/search?utf8=%E2%9C%93&q=API_PLAYLIST&type=

Hypothesis:

• probably something wrong happening with the parsing of GET parameters, at the routing level
  • could be caused by a buggy implementation of one of the helpers from snip.js, similar to this one: 6bafcf4 (which caused iOS bugs BUG: playlists are not listed openwhyd-ios#1 and BUG: iPhone app crashes on iOS 10.1.1 openwhyd-ios#2)
  • => try to reproduce that request (with the limit parameter) in an automated API test
• also, the limit parameter should be parsed into an integer before being sent with the mongodb request

[adrienjoly]

WIP: https://github.com/openwhyd/openwhyd/tree/bug-89

[adrienjoly]

Progress: was able to reproduce the bug, by providing two values of the same limit parameter in the playlist track request: ?...&limit=1000&limit=20

=> next step: check if the iOS does this.

[adrienjoly]

After activating the logging of GET parameters of incoming requests on openwhyd.org, I used the iOS app to open a playlist, and observed the following trace:

=== Fri, 18 Aug 2017 11:48:07 GMT GET /u/4d94501d1f78ac091dbc9b4d/playlist/61?format=json&limit=1000&limit=20 (userLibrary.controller) {"format":"json","limit":["1000","20"],"id":"4d94501d1f78ac091dbc9b4d","playlistId":"61"}
fetching user  { _id: 4d94501d1f78ac091dbc9b4d } ...
fetchPlaylists: 14.517ms
fetchStats: 11.812ms
fetchLikes: 1.001ms
fetchNbTracks: 9.095ms
=> fetched 0 posts
fetchAndRender: 5.602ms

=> the iOS app is indeed sending two values for the limit parameter: /u/4d94501d1f78ac091dbc9b4d/playlist/61?format=json&limit=1000&limit=20, which is probably causing the bug.

[adrienjoly]

Proposed solution:

• The limit parameter should not be provided more than once to the API request. => I proposed a fix in the iOS codebase: fix for bug #3: prevent playlist api request from sending the limit parameter twice openwhyd-ios#4 (pending review)