Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
In response to my RFC from Monday, @hoyes made the following comment:
I think a sensible approach for the Diary iCal endpoints may be to restrict them so that shows are only returned in the range of today's date ± 4 weeks? This way people can still use it to 'plan' which shows they might want to see whilst also preventing others from scraping mass data programatically (e.g. via https://www.camdram.net/diary.ics?end=2021-01-01). It might also be an idea to remove the whole-diary iCal endpoint in favour of the individual society/venue iCal endpoints? This would further limit excessive scraping.
Personally, I'm tempted to say that the iCal/RSS feeds on the vacancies section are okay to leave as they are. There is personal information in there (contact details, for example) but I can't really think of a good solution here. I suppose the ideal aim is to make the site as unattractive as possible for scrapers whilst still making ICS/RSS useful for people in their newsreaders and digital calendars. This may be a losing game...
± 4 weeks seems like an odd time period. I'd prefer to allow people to look indefinitely into the future since people could reasonably expect a whole-term view, and once you've allowed that there aren't many shows published >>8 weeks in advance so there is little point in restricting access to the future at all. On past shows though we could easily cut off at 20 or even 14 days for unauthenticated users without anyone really noticing.