No group parsing

[arjunior26]

Hi,
First, thank you for your fantastic script.

I just test it but I don't understand why my output file doesn't content any channel in this configuration :

My m3u file obtained from URL contains this line:
#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD

I defined group argument like this in my command line:
-g="'|EU| FRANCE TNTSAT'" -gm='keep'

Log:
2020-04-29T18:59:13.147659 saving retrieved m3u file: ./original.m3u8 2020-04-29T18:59:13.147659 parsing m3u into a list of objects 2020-04-29T18:59:13.757026 m3u contains 24464 items 2020-04-29T18:59:13.757026 keeping channel groups in this list ['|EU| FRANCE TNTSAT'] 2020-04-29T18:59:13.850776 filtered m3u contains 0 items 2020-04-29T18:59:13.866385 saving to log: ./process.log 2020-04-29T18:59:13.866385 script runtime: 0 minutes 7 seconds 2020-04-29T18:59:13.866385 process completed

Any hints appreciated.

PS : I use last python3 script

[bebo-dot-dev]

Hi there, can you please include your full command line with all of your arguments starring out anything sensitive like urls and passwords? I'll run some tests. Thanks.

[arjunior26]

This is my command:
python m3u-epg-editor-py3.py -m=" http://xxxxxxxxx.xxx/get.php?username=xxxxxxxxx&password=xxxxxxxxx&type=m3u_plus&output=ts" -e="http://xxxxxxxxx.xxx/xmltv.php?username=xxxxxxxxx&password=xxxxxxxxx" -g="'|EU| FRANCE TNTSAT'" -gm='keep' -d="./" -f="output_file"

No m3u file is created after this command is executed but original.channels.txt file is generated and contains some channels information (some from specified group and some other ones)

Thanks

[bebo-dot-dev]

I constructed a new minimal m3u file for the test that consisted solely of the following contents and I saved this file as minimal.m3u8:

#EXTM3U
#EXTINF:-1 tvg-id="BBC1.uk" tvg-name="BBC 1 HD ( 1080P )  [c]" tvg-logo="https://iptveditorlogo.netsons.org/BBC_1.png" group-title="UK Entertainment ",BBC 1 HD ( 1080P )  [c]
http://sstv.live:25461/XXXX/XXXX/79
#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD

I then executed the python script against this minimal m3u file as follows:

$ python ./m3u-epg-editor-py3.py -m="file:///home/joe/Dev/git_repositories/m3u-epg-editor/minimal.m3u8" -g="'|EU| FRANCE TNTSAT'" -gm="keep" -ne -nt -hi -l -d="./" -f="output_file"

This is the output from that test run:

2020-04-29T21:53:35.389566 m3u-epg-editor-py3.py process started with Python v3.8.1 (default, Dec 21 2019, 14:01:05) 
[GCC 5.4.0 20160609]
2020-04-29T21:53:35.389610 performing HTTP GET request to file:///home/joe/Dev/git_repositories/m3u-epg-editor/minimal.m3u8
2020-04-29T21:53:35.391137 saving retrieved m3u file: ./original.m3u8
2020-04-29T21:53:35.391323 parsing m3u into a list of objects
2020-04-29T21:53:35.392691 m3u contains 1 items
2020-04-29T21:53:35.392798 keeping channel groups in this list ['|EU| FRANCE TNTSAT']
2020-04-29T21:53:35.393218 filtered m3u contains 1 items
2020-04-29T21:53:35.393252 desired channel sort order: ['|EU| FRANCE TNTSAT']
2020-04-29T21:53:35.393299 saving new m3u file: ./output_file.m3u8
2020-04-29T21:53:35.393553 saving to log: ./process.log
2020-04-29T21:53:35.393650 script runtime: 0 minutes 0 seconds
2020-04-29T21:53:35.393673 process completed

I then checked that the resulting newly written ./output_file.m3u8 file contained the 1 expected matching row.

As can be seen from my test run output above: filtered m3u contains 1 items whereas for some reason your output was: filtered m3u contains 0 items so we're obviously doing something different or working with different input data.

It basically worked as expected for me so I think you'll need to look more closely at your input m3u data to be sure that it really contains what you think it contains (i.e. spaces where they are expected, not tabs / other unexpected whitespace characters etc)

[bebo-dot-dev]

I apologize please ignore my last comment, there is something odd going on here - investigating further.

[bebo-dot-dev]

ok I think I need to see a little bit more of your input m3u file data because this alone is incomplete and it does not constitute a complete m3u entry:

#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD

[arjunior26]

OK, supposing we have this minimal.m3u8 file:

#EXTM3U
#EXTINF:-1 tvg-id="BBC1.uk" tvg-name="BBC 1 HD ( 1080P )  [c]" tvg-logo="https://iptveditorlogo.netsons.org/BBC_1.png" group-title="UK Entertainment ",BBC 1 HD ( 1080P )  [c]
http://sstv.live:25461/XXXX/XXXX/79
#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD
http://sstv.live:25461/XXXX/XXXX/79

and then we execute this command line:

python ./m3u-epg-editor-py3.py -m="file:///path_to_the_file/minimal.m3u8" -g="'|EU| FRANCE TNTSAT'" -gm="keep" -ne -nt -hi -l -d="./" -f="output_file"

Are you agree that the result m3u file should contains only this ?

#EXTM3U
#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD
http://sstv.live:25461/XXXX/XXXX/79

Because here, all the lines of original file are appended to the generated file :(

[arjunior26]

Got it!
There is an error in is_item_matched() function:

        if not matched:
            # try a regex match against all groups
 			matched = any(re.search(regex_str, item_name, re.IGNORECASE) for regex_str in item_list)

regex_str is not defined in the current script

[bebo-dot-dev]

There is an error in is_item_matched() function
regex_str is not defined in the current script

You're close but there is no bug here. The is_item_matched function is in play when you saw and reported this:

all the lines of original file are appended to the generated file

The problem you reported there lies with the pipe (|) characters in your group name '|EU| FRANCE TNTSAT'
These pipe characters cause the -g="'|EU| FRANCE TNTSAT'" argument to be treated as a regular expression which is incorrectly and unexpectedly matching on everything because this is not the correctly formed regex expression that you need for what is being attempted and expected.

Using the same minimal.m3u8 sample input file from above (which is now complete and valid m3u file contents ;))

#EXTM3U
#EXTINF:-1 tvg-id="BBC1.uk" tvg-name="BBC 1 HD ( 1080P )  [c]" tvg-logo="https://iptveditorlogo.netsons.org/BBC_1.png" group-title="UK Entertainment ",BBC 1 HD ( 1080P )  [c]
http://sstv.live:25461/XXXX/XXXX/79
#EXTINF:-1 tvg-id="Tf1.mu" tvg-name="TF1 SD" tvg-logo="http://www.topstb.info/logo/FRANCE/TF1HD.png" group-title="|EU| FRANCE TNTSAT",TF1 SD
http://sstv.live:25461/XXXX/XXXX/79

try this:

$ python ./m3u-epg-editor-py3.py -m="file:///home/joe/Dev/git_repositories/m3u-epg-editor/minimal.m3u8" -g="'\|EU\| FRANCE TNTSAT'" -gm="keep" -ne -nt -hi -l -d="./" -f="output_file"

This will now work as expected because the pipe characters are escaped with backslash characters in the passed -g argument causing the is_item_matched function to match as intended.

[arjunior26]

Hi,

You're close but there is no bug here. The is_item_matched function is in play when you saw and reported this:

Oh sorry you are right, regex_str takes value from item_list :)

The problem you reported there lies with the pipe (|) characters in your group name '|EU| FRANCE TNTSAT'
These pipe characters cause the -g="'|EU| FRANCE TNTSAT'" argument to be treated as a regular expression which is incorrectly and unexpectedly matching on everything because this is not the correctly formed regex expression that you need for what is being attempted and expected.

Using the same minimal.m3u8 sample input file from above (which is now complete and valid m3u file contents ;))
try this:

$ python ./m3u-epg-editor-py3.py -m="file:///home/joe/Dev/git_repositories/m3u-epg-editor/minimal.m3u8" -g="'\|EU\| FRANCE TNTSAT'" -gm="keep" -ne -nt -hi -l -d="./" -f="output_file"

You're right ! It's working perfecly now.
Thank you for your time 👍

[bebo-dot-dev]

Excellent, no problem, I'm happy to hear that it's working for you now.