jira_user only listing first 50 entries

[LalitLab]

Describe the bug
When running SELECT * from jira.jira_user WHERE active = true, I get 2 users. There should be like 115.

As additional context:
When I query the table, it only contains about 50 entries. A quick eyeball of the entries indicates that the only 2 that have "active" set to true are the same two that showed up when I ran the above query

My assumption:
Jira uses a paged list
Each of those jira_user tables is a different page in the list
When I run the SQL query, it only queries the first page in the list
Now my question: what can be done about it?

Steampipe version (steampipe -v)
Example: v0.13.6

Plugin version (steampipe plugin list)
Example: v0.0.2

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

[cbruno10]

@LalitTurbot If this table has a pagination issue, do others have the same issue as well?

[LalitLab]

@cbruno10 will verify other tables too for paging while working on fix for it

[cbruno10]

@LalitTurbot Thanks for looking into this issue earlier today. Based on our initial findings, there appear to be 2 blockers:

• The users API doesn't return filtering fields in its responses, which makes implementing pagination client side difficult
• When listing users with the maxResults (default 50) request parameter, the maximum value for that parameter is 1000

For the first issue, it seems like pagination isn't available for users in the Jira API. For instance, when attempting to get users from the API using curl:

curl --request GET \
  --url 'https://myjira-jira.atlassian.net/rest/api/2/users?startAt=0&maxResults=2' \  
  --user 'lalit@turbot.com:...' \
  --header 'Accept: application/json'

The API returns the response:

[
  {
    "self": "https://myjira-jira.atlassian.net/rest/api/2/user?accountId=...",
    "displayName": "Lalit Bhardwaj",
    ...
  },
  {
    "self": "https://myjira-jira.atlassian.net/rest/api/2/user?accountId=...",
    "displayName": "Automation for Jira",
    ...
  }
]

This response is different from other APIs' responses in that it doesn't contain expected fields like startAt, total, or next, e.g.,

curl --request GET \
  --url 'https://myjira-jira.atlassian.net/rest/api/2/dashboard?startAt=0&maxResults=2' \
  --user 'lalit@turbot.com:...' \
  --header 'Accept: application/json'

{
  "startAt": 0,
  "maxResults": 2,
  "total": 3,
  "next": "https://myjira-jira.atlassian.net/rest/api/2/dashboard?maxResults=2&startAt=2",
  "dashboards": [
    {
      "id": "10000",
      "isFavourite": false,
      "name": "Default dashboard",
      ...
    },
    {
      "id": "10001",
      "isFavourite": true,
      "name": "Lalit",
      ...
    }
  ]
}

These are the fields that allow us to implement paging client side, but unfortunately missing when listing users. This was mentioned in JRASERVER-65089.

For the second issue, we can use the maxResults request parameter, but that appears to be capped at 1000 users, so only 1000 users max would be returned. We're still currently doing some testing from our side, but if this is the case, then it could require another workaround from our side to ensure that we can retrieve all of the users.

[tnelson-doghouse]

For the record, I'm on Jira Cloud (and that JRASERVER-65089 bug is for Jira Server). Probably not helpful, but thought I'd mention it.