pagination in get_logs()

[exitcode0]

response.has_next() seems to return True on a get_logs() call when the response is less than one page
the next header seems to return the same result as the self header

This causes issues for the below code

async def get_logs(log_offset, filter):
    logs, response, _ = await client.get_logs({
        "filter": filter,
        "since": log_offset,
        "limit": 1000
    })
    page_counter = itertools.count(1, 1)
    logging.info(f'Got page {page_counter.__next__()} of logs.')
    logs_array = []
    logs_array.extend([log_event for log_event in logs])

    while response.has_next():
        next_page, _ = await response.next()
        logging.info(f'Got page {page_counter.__next__()} of logs.')
        print(next_page)
        logs_array.extend([log_event for log_event in next_page])

    return logs_array

this seems like unintended behaviour, but if im doing something wrong here, please let me know :)

[arvindkrishnakumar-okta]

@exitcode0 Thanks for posting!

@serhiibuniak-okta Can you help with this?

[serhiibuniak-okta]

@exitcode0 Thanks for posting this issue, looks like it is a bug. I'll investigate it deeper as soon as possible. Btw, if no logs in response, then it is in eternal loop?

[exitcode0]

@sergiishamrai-okta
Correct, this code will continue to loop inside the while loop grabbing the next page response which will be an empty array

I'd imagine it is possible to work around this by comparing the link and self header

[serhiibuniak-okta]

@exitcode0 There are 2 issues at least... As a temporary workaround you can perform direct http request to get logs (or try to avoid pagination). I'll start work on this issue soon.

[serhiibuniak-okta]

@exitcode0 Finally, I've found that the issue was in "polling" request type. What you need is "bounded" request type, i.e. you need to specify "until" parameter - https://developer.okta.com/docs/reference/api/system-log/#request-types
Let me know if it works for you

[serhiibuniak-okta]

I'm closing this issue as stalled. Feel free to reopen this issue or create a new one if your problem exists still.