Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Disallow anonymous API access #552
Bit of a placeholder really - feel free to comment...
Can be merged in as/when we feel it's appropriate, but I suggest we give 4-8 weeks notice by email to anyone who has an API app registered, and maybe do a post on social media etc. as well for anyone who's using it anonymously.
left a comment
Seems a bit drastic tbh - nothing wrong with people being able to get the JSON view of a show anonymously, for example. Authentication is always more faff and more to break, esp. for third parties! Would make more sense to require API keys for anything person-related but maybe not for the basics of show data.
Do we have stats on how much traffic, with and without API keys, the API gets?
I agree we do need to lock down the API much more than it is at the moment, and maybe we'll end up with something close to this by some point. I don't really know how many people are using the API (which is admittedly a little scary), but I think we need to go a little slower as it'll be Camdram that looks bad if we break too many other websites at once. And while we don't want to make the rules too complex either, I think there are disadvantages to having a one-size-fits-all approach.
In lieu of a fully thought out response, here's a bit of a brain dump for now:
In principle I agree with you - and I do assume good faith on the part of users. However I do worry that Camdram is a potential treasure trove of information that could be quite easily monetised or otherwise exploited in a way that, crucially, users are not expecting nor consenting to. I know this issues seems quite abstract but it would be so easy for someone to graph the entirety of Camdram data for the last, say, 3 years and then predict friendship groups, real-world habits/patters and people's locations from this data. This is actually an issue that someone has directly raised with me in person before now. I think this possibility is remote, but I think it would be more responsible of us than not to implement API access controls. I'm not necessarily talking full registration & approval like a social media giant, but at least free & open registration so we can keep track of what & who is using our site and its data. Then if we find out someone is abusing the API in this way we can revoke access & block their user account.
Additionally, I think it would make API rate limiting, logging and abuse prevention much easier to implement. If we can trace excessive usage of an endpoint to a single application via it's credentials then we in turn know the user responsible; we can contact them and advise them how to change their app to improve their + our performance. Currently the only data we get from anonymous usage is an IP address which is a lot more work to trace (and potentially impossible).
I actually dispute this! There's some really great third-party OAuth2 clients out there that can be wrapped around the camdram.net API more easily than trying to roll-ones-own in my opinion. Obviously I don't want to prescribe the way a developer should work but if someone has written a module that handles App ID's, Secrets & Access Tokens completely transparently then why reinvent the wheel.
Working on it right now (although Pete's giving me quite a bit of help)!
Well if they're legitimate then they should really be using proper credentials as advised on the API page!
I do think this discussion is a bit academic. I would guess the total number of serious services/platforms/apps that use the API numbers ~5 and judging by
How about this potential reduced scope for this PR:
I think all the above can still be achieved fairly simply using security.yml alone (https://symfony.com/doc/current/security/access_control.html#matching-options)
Other, potential separate steps for the near future (perhaps the subject of separate tickets):
Thanks for getting the ball rolling @CHTJonas ... I'll admit the complexities of this have probably deterred me from making a move to date.
Sounds good to me. I should add that I'm about to release version 2 of https://github.com/CHTJonas/camdram-ruby and whilst obviously I maintain it, it's very much not meant to be Camdram doctrine! Was written as a side project to go with new room booking. Client-side JS & CORS is an interesting point - don't really know a lot about it myself.
To be honest, in order to make POST/PUT/DELETE requests an app would have to be using OAuth already. And I'm tempted to say that they can continue doing this if they want since they're doing nothing wrong. As you say though, I suspect this affects nobody/nothing.
Can track endpoint usage & query strings but auth bearer & user agent headers won't show up so difficult to trace usage to a specific application, save
This was definitely a long-term proposal on my part and was meant to be coupled with documentation on how OAuth works before being merged.
I guess the point I was trying to make is that due to the sprawling use of FOSRestController I think Camdram needs to prune the available API endpoints a little, and camdram-ruby is an interesting pseudo-third-party perspective on what a useful subset might look like - obviously evolve it as you see fit!
Hmm maybe we can leave PUT/POST/DELETE in place for now, but leave them out of the documentation or something, or at least put some "here be dragons" warnings. I'm not very confident that we're returning sensible responses in the case of validation errors etc so worry about managing expectations etc.
It might be interesting to summarise which json/xml URLs are being hit with which methods, even if we can't (yet) tell who's making the requests.