Adding new endpoints

[stav]

What is the best way to add new endpoints to the client that the app/hubstorage apis already support?

For example: https://app.scrapinghub.com/api/v2/projects/206606/scripts

[stav]

Should I add new modules to the client package or extend the methods in the legacy module?

Maybe I should submit a PR for refactoring the legacy module into the client package?

[vshlapakov]

Thanks for raising this. Talking specifically about adding scripts endpoint, I don't think that it's worth implementing for now, because there's an intention to completely get rid of script concept, and stop handle it in a special way. So adding a new endpoint to drop it in a few months or so doesn't look like a good idea.

Talking about extending the client in a general case, the topic itself is a bit tricky. First of all, hubstorage client and legacy module are frozen, we don't want to evolve it or change, because there're still a lot of clients relying on it and even a minor change can lead to disaster.

A new client located in scrapinghub.client assembles logic from the both legacy clients and unifies it, so if we want to extend the library e.g. support new endpoints - for now it should be done on the scrapinghub.client level only.

Given that the new client is written on top of frozen legacy clients, it's not an easy task to extend it in general case, but something to keep in mind is that we're going to rework Scrapinghub API in the near future (we have it in our roadmap), to make it more sound and easier to use, and when it's done we'll start discussion about reworking the client completely.

I'm going to close the issue for now, but let me know if you have any thoughts/concerns/proposals and we'll continue the discussion (here or in a private chat).