Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watching webpage that needs a session cookie #748

Closed
kjabl opened this issue Jan 27, 2023 · 2 comments
Closed

Watching webpage that needs a session cookie #748

kjabl opened this issue Jan 27, 2023 · 2 comments

Comments

@kjabl
Copy link

kjabl commented Jan 27, 2023

Is it possible to watch a website that needs an actual session cookie which is valid only once?
I want to watch the result of the form on this site: https://rathaus.dortmund.de/antragsstatusausweis/statusrequest.jsf
The problem is when you open this webpage a session cookie is set which is valid only once for submitting the form. So Urlwatch would need to first access this site, save the cookie and then send the post data for the form.

Is there a way to achieve this?

@thp
Copy link
Owner

thp commented Feb 1, 2023

If you can write a script using mechanize or some other tool, you can integrate that script using the shell job type. Since this sounds very specific to that page, I don't think it's something that we can easily implement in a generic way.

As the URL sounds like it's something of public interest, maybe there's a way to just ask them to provide a nice API that you can call instead?

@kjabl
Copy link
Author

kjabl commented Feb 1, 2023

Thx for your answer. Mechanize looks really interesting.
In the meantime I investigated further. I found a tool which can do this. It's called changedetection. BUT it's a lot more heavyweight then urlwatch. I thin urlwatch is great because its simple and lightweight.

Changedetection makes use of a browser for opening pages and analysing that. The benefit is that you can even execute javascript. But therefore it's a lot more complicated to set up. For me it was two docker containers, one for the browser engine and one for the tool and they have to talk to each other and so on...

This special job can be done quite simple, the final call can contain the form data as post body. All you need is the session cookie from the first call which is valid only for some minutes, so you can't set static cookie data.
But I think you're right. This is very special and I'm not sure if it can be abstracted to be generalised for configuration.

@kjabl kjabl closed this as completed Feb 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants