Public contracts endpoint not returning all contracts

[CarbonAlabel]

Bug

The new public contracts endpoint returns no more than 100 contracts for a region. Pagination is not available, as the 100 contracts are below the 1000 item page size.

Request

https://esi.evetech.net/v1/contracts/public/10000002/

Response

Only 100 contracts are returned.

Expected

The Forge has a lot more than 100 public contracts.

[ghost]

Probably coming from a limitation inside the contract system somewhere

[ghost]

Found the issue, I'll be checking in the fix today or tomorrow after testing. Release probably Tuesday. Nothing will change on the endpoints, they'll just start returning more stuff, so feel free to start developing against the current endpoint on the assumption that it'll use the standard X-Pages paginating mechanism.

[ccp-zoetrope]

Fix is checked in and going out Tuesday.

[CarbonAlabel]

The fix has gone out, raising the limit to 10 000 contracts returned.

Turns out, The Forge has more than 10 000 public contracts.

[ghost]

The bug itself is fixed. 10,000 was a specifically chosen sanity limit we picked while fixing the issue. We might raise it further in the future depending on what the memory graphs look like, but I want at least a few weeks data before I consider that further.

[lyttol]

This bug is NOT fixed. Public contracts endpoint is NOT returning all contracts.

This is creating an unfair situation after the amazing work done in mutaplasmid.space to fix the gaping hole in the abyssal mod market, since only some contracts are showing up. CCP please fix this ASAP and allow all contracts to be retrieved. If you have to find a way to return them "page of 10,000 at a time" do it.

[ddavaham]

Please read the comment left by the CCP Bartender. This no longer a BUG, so much as it is a limitation placed on the endpoint. Bartender mentioned that after a period of time, if the memory gods will it, they will increase the limit, until that time however, applications utilizing this endpoint will need to adapt to this limit.

[jowrjowr]

please direct further comments to /dev/null where they will be ignored in the order in which they appear

[ghost]

The memory graph looks good, I've submitted a change to raise the limit to 20k. It'll probably get deployed Tuesday next week.

[CarbonAlabel]

The change was deployed today, but the number of contracts in The Forge once again exceeds the new limit.

This has been noted. Same as before, memory usage will be monitored for a while, and the limit should get bumped again in a week or two.

[ghost]

Correct, I'll look closer and ask again how many there are. When I first asked p sure it was 17k but I'll recheck.

In the mean time, as carbon says, same drill. I'm gonna observe the new limit, and I'll bump it in two weeks if it's not causing issues.

[ghost]

The memory graph has shown an increase and trend I'm not happy with over the past few weeks. It may not be related to increasing the limit, but I need to investigate it before I can increase the limit further. More as it happens.

[ghost]

We've smoothed out the contract memory usage considerably, and the limit has been bumped to 35,000. As the forge has ~30,399 contracts in it at the time of writing this will hopefully be the last bump needed.

I've integrated the change, hopefully it'll go out tomorrow otherwise some time next week I guess. No changes to the endpoint, it'll just start returning more once the change is deployed.