Two /tags/{uid} APIs?

[ldeffenb]

Summary

Apparently there are two different /tags/{uid} APIs implemented, one on the debug port and one on the API port. Consider:

API port:
curl http://127.0.0.1:8080/tags/172117899 2>nul | jq
{
"uid": 172117899,
"startedAt": "2021-02-10T16:44:50.3778219-05:00",
"total": 182,
"processed": 182,
"synced": 182
}

Debug port:
curl http://127.0.0.1:6060/tags/172117899 2>nul | jq
{
"total": 182,
"split": 182,
"seen": 182,
"stored": 182,
"sent": 0,
"synced": 0,
"uid": 172117899,
"address": "",
"startedAt": "2021-02-10T16:44:50.3778219-05:00"
}

From the startedAt timestamps, you can tell these are two different views of the same tag. But the full form is documented in the main API at https://docs.ethswarm.org/api/#tag/Tag/paths/~1tags~1{uid}/get

And tag isn't even mentioned in the Debug API at: https://docs.ethswarm.org/debug-api/

But yet, in https://github.com/ethersphere/bee/blob/master/openapi/SwarmCommon.yaml you can find both NewTagResponse (the short version) and NewTagDebugResponse (the long version).

Steps to reproduce

Create a tag, upload a file with it, and use the /tags/{uid} on both the gateway and debug port and you'll see the difference.

Expected behavior

I expected only one /tags/{uid} response, and expected the complete documented information one to come out of the gateway port.

Actual behavior

See above.

[ldeffenb]

So I refresh my browser and find that this is fixed. Some's quick (or someone like me needs to refresh before opening issues!).

[zbiljic]

@ldeffenb Not sure what exactly was reported as problem here.

There are in fact two different /tags/{uid} endpoints, one through public API, and one on the debug API. As you noted, these in fact DO point to the same underlying tag structure, however the one through debug API does show some more informations.
The changes that were made relate to public API, where tag model was changed to have less information because there were numerous issues being created that were just result of users not completely understanding how all fields work. Our documentation did contain more detailed explanation for each of the fields in the "Track Upload Status" section, however it is was either that people did not find it or it was still bit complex. I confess that even having worked on the project extensively, after few months time, I had trouble remembering how values must be read, and had to remind myself by reading descriptions.

Reason for the change was to "seemingly" simplify API, and allow easier implementation of progress tracking for upload. Using previous structure one would have to do math with some fields to have correct values. While this is not much work for user, it was deemed as unnecessary.

The "old" API endpoint was basically 'moved' to debug API, because that is still benefit for having all fields for tag, especially while working on changes to this project, where fine grained details are required. The debug API does contain some "private" endpoints, but is also (as the name implies) for debugging the state of the node.

I would generally say to use tags public API for everything, and in the case where some problem is spotted (mismatched counts), if possible report output from public API and debug API for the same tag UID.

[ldeffenb]

@zbiljic I reported the issue because I was surprised to only get the 3 counters from the public API when I knew I used to see more data. Then I went to my nearly-always-open API and Debug API references and saw the details in the public API (which I wasn't getting) and nothing mentioned in the Debug API. So I opened the issue.

Fast-forward 2 days when I refreshed my browser tabs containing these references only to discover that my view was out of date and the difference was indeed documented. So I posted the comment and closed the issue.

I apologize for, but appreciate, your extra effort in explaining the history of the change. I totally support the reasoning, and my data-loading script is using the public API until it notices that any particular upload is taking too long for the three numbers to match. In fact, there seems to be some boundary condition around 133/134 chunks in a single file (513-515KB) that causes Processed and synced to stop at 133 even though total says 134. I've documented this and provided sample files in #1244

[zbiljic]

@ldeffenb 👍