Paging does not work when I set includeProcessVariables to true

[mdmatthias]

This still gives correct results:
/flowable-rest/service/history/historic-process-instances?size=200&start=600&excludeSubprocesses=true&includeProcessVariables=true

"total": 5845,
"start": 600,
"sort": "processInstanceId",
"order": "asc",
"size": 200

But when I request the next page there are only 10 results:
/flowable-rest/service/history/historic-process-instances?size=200&start=800&excludeSubprocesses=true&includeProcessVariables=true
"total": 5845,
"start": 800,
"sort": "processInstanceId",
"order": "asc",
"size": 10

Anything higher than 810 gives 0 results.
/flowable-rest/service/history/historic-process-instances?size=200&start=1300&excludeSubprocesses=true&includeProcessVariables=true
"total": 5845,
"start": 1300,
"sort": "processInstanceId",
"order": "asc",
"size": 0

But when I set includeProcessVariables to false it works correctly
/flowable-rest/service/history/historic-process-instances?size=200&start=1300&excludeSubprocesses=true&includeProcessVariables=false
"total": 5845,
"start": 1300,
"sort": "processInstanceId",
"order": "asc",
"size": 200

[martin-grofcik]

The problem with includeProcessVariables is that we are joining executions and variables in the query. That's why pagination does not work in that case.

[mdmatthias]

But why does it only break after 800 results? if we request for example results 400-600 it works correctly.

[martin-grofcik]

Good question, but hard to answer.
Have processes 0-600 any variables? I would expect so. The only way how to find it out is to debug query.( copy the query from the log with debug level and try to execute it.)

[vikram-ray]

When making a REST API call to query historic tasks using the parameter "includeProcessVariables": "true" , a maximum of 20,000 variables are returned. The results of the REST API when using "includeProcessVariables": "true" are cut off as soon as the 20,000th variable is in the REST API response.

I am facing this issue as I have more than 5000 processes. For performance reason, they have set the limit to 20000 variables ( https://issues.alfresco.com/jira/browse/ACTIVITI-1065). Each of my processes has more than 500 processVariable and for that reason, I am not able to get paginated data for process no more than 40.

Can anyone help me figure out how to increase this limit?
or
Can I get paginated response to get 2 or 3 variables(firstName, email) with includeProcessVariable=true?

[filiphr]

This has been fixed with our rework of how we do paging in PR #2892.