Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTPError 308: Permanent Redirect #426

Closed
xalienxx opened this issue Dec 21, 2022 · 10 comments
Closed

HTTPError 308: Permanent Redirect #426

xalienxx opened this issue Dec 21, 2022 · 10 comments
Assignees
Labels
bug http parsers of IMDb web pages

Comments

@xalienxx
Copy link

Have IMDB changed the URL structure?

raise IMDbDataAccessError(
imdb._exceptions.IMDbDataAccessError: {'errcode': None, 'errmsg': 'None', 'url': 'https://www.imdb.com/find?q=the+batman&s=tt', 'proxy': '', 'exception type': 'IOError', 'original exception': <HTTPError 308: 'Permanent Redirect'>}

Redirect to:
https://www.imdb.com/find/?q=the%20batman&s=tt

@mickdupreez
Copy link

Have IMDB changed the URL structure?

raise IMDbDataAccessError( imdb._exceptions.IMDbDataAccessError: {'errcode': None, 'errmsg': 'None', 'url': 'https://www.imdb.com/find?q=the+batman&s=tt', 'proxy': '', 'exception type': 'IOError', 'original exception': <HTTPError 308: 'Permanent Redirect'>}

Redirect to: https://www.imdb.com/find/?q=the%20batman&s=tt

I am getting the same error.

@xalienxx
Copy link
Author

Have IMDB changed the URL structure?
raise IMDbDataAccessError( imdb._exceptions.IMDbDataAccessError: {'errcode': None, 'errmsg': 'None', 'url': 'https://www.imdb.com/find?q=the+batman&s=tt', 'proxy': '', 'exception type': 'IOError', 'original exception': <HTTPError 308: 'Permanent Redirect'>}
Redirect to: https://www.imdb.com/find/?q=the%20batman&s=tt

I am getting the same error.

The url structure can be replaced and fixed temporarily.But I think IMDB is going to make some big changes in the future. It is possible to limit the information we can get without signing in. (human verification)

The best way to get information is to store it in a database for later use. But we will not be able to get ratings and real time updates.

@FinlandApollo
Copy link

Are there any timetable for the updates? Or are there a simple fix to this?

@nielth
Copy link
Contributor

nielth commented Dec 21, 2022

#427 This should do the trick. Let's hope someone can review it as soon as possible.

@FinlandApollo
Copy link

#427 This should do the trick. Let's hope someone can review it as soon as possible.

Thank you, this change fixed the issue for me (I've used the search phrase and movieID search and so far, those work!)

Great job!!

@xalienxx
Copy link
Author

#427 This should do the trick. Let's hope someone can review it as soon as possible.

Yes, this will fix the issue. And it is a simple but critical issue. As I said before, it looks like IMDB is going to make a lot of changes in the future. However, don't forget that IMDB is owned by Amazon.

@xalienxx
Copy link
Author

Are there any timetable for the updates? Or are there a simple fix to this?

I think it will probably take a day or two to give a major pypi update. Or you can simply clone directly from the repository.

@alberanid alberanid self-assigned this Dec 21, 2022
@alberanid alberanid added bug http parsers of IMDb web pages labels Dec 21, 2022
@alberanid
Copy link
Collaborator

Thanks for the report and the PR; I'll review it as soon as possible.

alberanid added a commit that referenced this issue Dec 22, 2022
* nielth-master:
  support 308 Permanent Redirect HTTP code; #426
  s32cinemagoer.py - Add cleanup option (#423)
  Fixed HTTPError 308: Permanent Redirect
@alberanid
Copy link
Collaborator

thanks for the report and the PR!

It's a little more complex than this, and I think I've added a more generic fix with commit 2e6737e

The point here is that recently (?) IMDb started using Cloudfront as a CDN (surprisingly, being IMDb part of Amazon...) which for some redirections returns a 308 Permanent Redirect code which apparently is not handled by some (most?) Python versions.

The new code should fix all off the 308 redirections.

I will probably release a new version on Pypi in the next few days.

Thanks!

@xalienxx
Copy link
Author

thanks for the report and the PR!

It's a little more complex than this, and I think I've added a more generic fix with commit 2e6737e

The point here is that recently (?) IMDb started using Cloudfront as a CDN (surprisingly, being IMDb part of Amazon...) which for some redirections returns a 308 Permanent Redirect code which apparently is not handled by some (most?) Python versions.

The new code should fix all off the 308 redirections.

I will probably release a new version on Pypi in the next few days.

Thanks!

Yes, that is the reason. Let's continue as long as possible.

I am currently trying to create an API for Rotten Tomatoes and IMDB with login credentials. But the problem I have is that if they become like Facebook and Instagram in the future with their interactive interface, it will be a very difficult task. I will let you know if it works.

Thank you !!! Great work !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug http parsers of IMDb web pages
Projects
None yet
Development

No branches or pull requests

5 participants