Skip to content

Commit

Permalink
Fix not corectly passed parameters in callInThread call
Browse files Browse the repository at this point in the history 
Added User-Agent header since some servers are answering 403 if they are missing
  • Loading branch information
@henrylicious
henrylicious committed Jul 12, 2023
1 parent 6eb43b6 commit 5f9335c
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/EPGImport/EPGImport.py
View file
Expand Up @@ -195,9 +195,9 @@ def urlDownload(self, sourcefile, afterDownload, downloadFail):
if ext and len(ext) < 6:
filename += ext
host = sourcefile.split('/')[2]
Headers = {'host': host}
Headers = {'host': host, 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'}
print("[EPGImport][urlDownload] Downloading: " + sourcefile + " to local path: " + filename)
callInThread(threadGetPage, url=sourcefile, file=filename, headers=Headers, success=afterDownload, fail=downloadFail)
callInThread(threadGetPage, url=sourcefile, file=filename, urlheaders=Headers, success=afterDownload, fail=downloadFail)

def afterDownload(self, filename, deleteFile=False):
# print("[EPGImport][afterDownload] filename", filename)
Expand Down

5 comments on commit 5f9335c

@urie57
Copy link

@urie57 urie57 commented on 5f9335c Jul 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have had to revert to Commits on Jan 18, 2023 because EPGImporter was not pulling in my providers via jedimakerxtream, but it does work with xstreamity
running latest version of openatv7.3

getting this error in debug.log

14:02:06.5745 [EPGImport][threadGetPage] error: Exceeded 30 redirects.
14:02:06.5746 [EPGImport][downloadFail] download failed: Exceeded 30 redirects

@TwolDE2
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henrylicious thanks for fixing this stupid label error.
@urie57 -you might have a word with Kiddac as its not directly an import issue. With the test files I use there are upto 3 redirects and they are handled corrcetly, so not sure why/how there are so many redirects from one url.

@urie57
Copy link

@urie57 urie57 commented on 5f9335c Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reply I will talk to KiddaC.

@kiddac
Copy link

@kiddac kiddac commented on 5f9335c Jul 18, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the possible cause of this might be jedimakerxtream wraps urls in cdata wrapper. As those urls have querystrings.

example

<url><![CDATA[http://blah.uk:8080/xmltv.php?username=blah&password=blah&next_days=7]]></url>
in xstreamity I don't use cdata tags but instead convert ampersands in the querystring to &amp;

jedimakerxtream plugin is old now and wrote it a long time ago. Its not really in a position to be updated. There is a totally new version being written in the background.

but from the conversation above, I presume .encode() might still be needed on the url for lots of iptv plugins that used the cdata method in the sources xml.

response = get(url, verify=False, headers=urlheaders, timeout=15)

@kiddac
Copy link

@kiddac kiddac commented on 5f9335c Jul 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have all been doing some testing tonight. Pretty sure what I wrote above is not the reason anymore.
But some observations so far.
You are creating filenames from urls, without correctly removing illegal characters that are not allowed in filenames.
If a host has a port number. The filename will include colons.
There is definitely a redirect issue for any urls that redirect. Redirects are failing.

Please sign in to comment.