FEAT: Pagination support for /jobs endpoint

[prasadlohakpure]

Description
This pull request adds cursor-based pagination to the jobs listing endpoint in the Heimdall package. The main changes include extracting and enforcing page size limits, updating the SQL query to support cursor and limit parameters, and modifying the result set to indicate if more results are available and provide a cursor for the next page.

Pagination and query improvements:

• Added support for a cursor filter and removed the hardcoded SQL LIMIT in select_jobs.sql, enabling dynamic pagination through the API (internal/pkg/heimdall/job_dal.go, internal/pkg/heimdall/queries/job/select_jobs.sql). [1] [2]
• Extracted and enforced page size limits (defaultPageSize and maxPageSize), and updated the query to fetch one extra row to determine if more results exist (internal/pkg/heimdall/job_dal.go). [1] [2]
  *Added support for filtering using comma seperated tag values
  e.g.:

http://127.0.0.1:9090/api/v1/jobs?name=run-my-query&tags=_name:run-my-query3,_id:fa9af168-b328-4ef3-a1b1-1a9ffbaf60fd

Result structure changes:

• Modified the resultset struct to include HasMore and NextCursor fields, and updated the handler to populate these fields based on the query results (internal/pkg/heimdall/result.go, internal/pkg/heimdall/job_dal.go). [1] [2]

Dependency updates:

• Imported strconv and strings to support parsing and manipulating filter parameters and queries (internal/pkg/heimdall/job_dal.go).

Testing:

curl --location --request GET 'http://127.0.0.1:9090/api/v1/jobs?limit=5' 

Result : Gives 5 objects in respone

curl --location --request GET 'http://127.0.0.1:9090/api/v1/jobs'

Result : Default page size(101) or present number of jobs, whichever is lower along with
"has_more": true/false (depending on values available)
"next_cursor": 112

curl --location --request GET 'http://127.0.0.1:9090/api/v1/jobs?limit=5&cursor=5

Result : Gives 5 objects ordered by system_job_id, and ahead of cursor specified

curl --location --request GET 'http://127.0.0.1:9090/api/v1/jobs?limit=5&cursor=5&tags=_name:run-my-query3,_id:fa9af168-b328-4ef3-a1b1-1a9ffbaf60fd

Result : Gives 5 objects ordered by system_job_id, and ahead of cursor specified with exact match for both tags

[Copilot]

Pull request overview

Adds cursor-based pagination to the Heimdall /jobs listing endpoint to enable incremental fetching of job history while enforcing page-size constraints and returning pagination metadata to clients.

Changes:

• Add cursor filtering and move LIMIT handling into the DAL for dynamic page sizing.
• Fetch pageSize+1 rows to determine HasMore and compute NextCursor.
• Extend the shared resultset response wrapper with pagination fields for JSON responses.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
internal/pkg/heimdall/job_dal.go	Adds cursor filter support, limit extraction/capping, dynamic SQL limit, and populates pagination metadata in responses.
internal/pkg/heimdall/queries/job/select_jobs.sql	Removes hardcoded LIMIT to allow DAL-driven pagination.
internal/pkg/heimdall/result.go	Extends API response wrapper to include pagination metadata fields in JSON.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.