Live attribute not working again!

[sp7dpt]

Version of the custom_component

0.3.3

Describe the bug

Live attribute return always false but stream is live

[chairman2s]

I just noticed the same thing and was going to post on the forum but then I thought I should check here first...

[sp7dpt]

I found this relationship at https://www.youtube.com/channel/[channel_ID]/live html code returns a response "isLive":true
Maybe this will help repair the component.

[pinkywafer]

The live attribute is still working for me.
It should be noted that the sensor reports on the most recent video... It is possible that a channel is live, but there is a video which has been updated since the stream started.
It may be possible to add an additional 'channel_live' attribute as an extra.
Thoughts please @ludeeus

[ludeeus]

Without sensor attributes and configuration there is no way to tell

[sp7dpt]

In this moment on the Kreator channel is stream live.

My sensor return:
url: https://www.youtube.com/watch?v=8Ut1S1fka_w
published: 2021-01-12T15:52:25+00:00
live: false
friendly_name: Kreator
icon: mdi:youtube
entity_picture: https://i1.ytimg.com/vi/8Ut1S1fka_w/hqdefault.jpg

Sensor are defined:

#Kreator

• platform: youtube
  channel_id: UC1SHj5QU7AHPVHpLe0UGJ7Q

What else should I do?

[ludeeus]

I can't even get it to start, something changed with Python 3.9 with html it seems, if I fix that it will break for everyone else.

[pinkywafer]

Shows as live on mine

[sp7dpt]

it doesn't work for me. Why?
What is your Home Assistant configuration?
My Home Assistant version is 2021.1.1 and supervisor version 2020.12.7
What can I check or set more?

[pinkywafer]

I'm running full home assistant (home assistant os, supervisor + core) in a proxmox vm.
YouTube s sensor integration v0.3.3 installed through hacs.

This is strange though as it shouldn't make a difference that I can see, unless you are running core on a different version of python.

Unless... I'm just wondering if YouTube generates the page differently depending on country for example. This is however only a speculation
For reference, I'm in uk

[sp7dpt]

The script looks for the string in https://www.youtube.com/feeds/videos.xml?channel_id=
This is my xml

[ludeeus]

I wonder if it's a language thing, where YT serves the page that is scraped with a different language based on IP

[sp7dpt]

Maybe this is it. That's why I included my xml.

I am currently using YouTube API to check live streams, but there is a query limit of up to 10,000 per day. One search costs 100 units.

[ludeeus]

enable debug for the integration, see example in .devcontainer/configuration.yaml

[sp7dpt]

I enabled debug and this is result:

2021-01-13 12:44:02 DEBUG (MainThread) [custom_components.youtube.sensor] Kreator - Running update
2021-01-13 12:44:02 DEBUG (MainThread) [custom_components.youtube.sensor] Kreator - Skipping live check

[pinkywafer]

OK, the "Started streaming" is NOT scraped from the channel's videos.xml page.
The url of the latest video is scraped from that page, and then the latest video page source is scraped for "Started streaming"
If you are able to save the source of a live video page and upload it, I should be able to compare.
Can you please also let us know what country you are in.
I expect that as @ludeeus suggested, this may be being auto generated based on country IP, so I'd expect to find "Started streaming" in your local default language.

[sp7dpt]

Yes, the latest video is scraped.
This is my xml when streaming was live. I did not find any keywords for a live stream. I'm from Poland

[pinkywafer]

That XML is the channel videos.xml which is only scraped for the link to the latest video.
In that example, the video URL (and page which is scraped for live) is https://www.youtube.com/watch?v=8Ut1S1fka_w

[sp7dpt]

I don't understand now how to get you what you need.
This channel is live all the time, that would be a good example https://www.youtube.com/watch?v=L9Sum_pL7m4
What file format should I download for you, HTML or else?

[pinkywafer]

yes, the html page of any live video.

Note that while I can use the channel you just linked to work out what is going on, it is not the most recent video, so the sensor would still report the latest video as not live

[sp7dpt]

ok, I saved this page in the browser as HTML. This is the file. You have inside two examples,
Is it what you want?

[pinkywafer]

thanks, that is more than I need. I really only needed the html, but this is the html complete, but no matter, I can get the info from this.

It looks like the scraped text is indeed automatically translated into the local language. I'll look if there are alternative options

[pinkywafer]

Ok, I think I've found something I can use. Can you please save a page from the browser of a live stream which has finished so that I can verify.
Thanks

[sp7dpt]

ok, I saved one of the channels wherein strem is offline. This is the file

[pinkywafer]

That's great, thanks

I think I've found a fix, I'll get back to you!

[pinkywafer]

Are you able to test the sensor.py located in the live-attribute-global-fix branch please

[sp7dpt]

I replaced the sensor.py file and now it works fine. I will test a little and give feedback.

[pinkywafer]

great, thanks.
@ludeeus I'll do PR and merge in. Please don't release yet as I have a few extras up my sleeve!!

[ludeeus]

I think you should have permissions to release, if you do, go ahead when you are ready, if not ping me and I'll adjust 👍

[sp7dpt]

I checked few channels and sometimes detection not worked. This is a file HTML when detection not worked. Maybe it will help.
Strange because the word "isLiveBroadcast" occurs in HTML

[pinkywafer]

The first video in the zip shows a start date of back in November. If there has been a more recent video published, the sensor would not show live for this. I can't see any other reason why this does not show as live.
The second html actually has an end listed in the html, indicating that the video was streamed live, but is no longer live.
I've just added additional attributes that you can see in my pull requests #20 and #21 which should go some way to addressing this. If you install the current master, you should be able to test

[sp7dpt]

I tested and it looks like the attributes are working.