Support for YouTube live chat integration

[DovahDoVolom]

The implementation of new live chat/live streaming provider YouTube live.

YouTube live API: https://developers.google.com/youtube/v3/live/docs/liveChatMessages

[BlazeVR]

There's a lot of content creators with big audiences streaming exclusively on Youtube, would be a nice addition next to our Twitch integration.

[Frooxius]

This can be added at any time, it's just matter of prioritization and seeing how many people are interested in this over other features.

[TehTurk]

Would it be okay to ask for a good hard number at all to get a good feel of what might be interested? I can definitely try to get folks to chime in more on this issue because I actually had a few folks approach me about it today as a "big issue" folks were kinda concerned about and I can understand and recognize the sentiments. Some of the feedback I've been getting from folks is

• Youtube for a lot of different cultures/regions(Like the Korean/Japanese community) is more popular and more outreaching to those cultures. (We've actually recently gotten a dedicated Korean Community in Neos due to Youtube (Thanks GreatMoon and Neos KR!), so I would definitely push for this being prioritized and us reaching out to more content creators who'd be interested in showing off the platform.
• A lot of VTubers and VTubing will typically be done on Youtube (This can be a boon vs Twitch) so you can also appeal to folks who'd like to be using Neos for interactivity, or overall public interaction as a tool massively giving folks in youtube something that Youtube is also often lacking with bots or chat interactions.
• Youtube's Video/Vod Viewing is a bit more stable then twitch which will also have a more passive over time effect that most folks realize

[Frooxius]

It's hard to say the exact number, we'd just need to see people being interested in this. E.g. vote on the issue so it goes above others to increase the likelyhood of it being prioritized. E.g. how many people do we have that would be interested in having this integrated?

Right now this only comes up every few months, if at all. There's no denying that it'd be a nice thing to have, it just has to be weighted against other needs and wants and overall development.

[Mookoo-m]

I'm just leaving a message here for people who wonder why I'm pushing for Youtube Live chat integration.
I have been asking for this for a while now. Even asking Frooxius himself about it. There no denying Youtube Live chat integration will be great to have in NeosVR. But I know a few wondering "Who even use Youtube for streaming?"

Well from what I see in this community, the Jp (and KR now) community have been streaming NeosVR on youtube the most. I have asked if they are interested in it and got a good response for it.

Now as a Vtuber like myself, I find NeosVR to be a great place to create my content in. I have streamed on Twitch before and had fun messing with Twitch integration especially my viewers at that time. As of now I'll be moving over to Youtube for streaming but I don't want to lose what I enjoy and the plans I made for future events.

Many of my Vtubers friends who tried NeosVR find this to be easier than going to making their own thing like Unity or Unreal engine since a lot of stuff is pretty much there and we all can do collabs.

I really love Neos and want to use it for its great potential while streaming on youtube.

[Hambonethewise]

Leaving this message here to hopefully further convince the priority of this. There may be a small number of creators using YouTube streaming as a platform to stream, however any streams done there at all will only encourage more streamers to use Neos who aren't on Twitch.

Personally I run a channel with my wife of 155k ( VBunny Go: https://www.youtube.com/channel/UCuY-O8kE4bTSO3FUKLCJ-CQ ) and would love to see this feature since we had moved from streaming on Twitch to YouTube. Not to mention this has been the only thing stopping me from streaming on my personal account as well, as I have had a lot of issues with Twitch myself.

Love Neos as well and I agree with Mookoo, the potential that can come from this one feature alone isn't to be underestimated. In our case it's the difference between making Neos the most streamed game as it gives us so many options for interactivity with viewers. Our audience would absolutely LOVE this feature.

[MinkyFox]

I'm a small content creator (~3.7k by now) and I would love to stream in Neos VR and being able to interact with viewers like you are able to do with twitch. I don't want to make my Viewer switch the platform to view a livestream, that's why I look forward to when we finally get a YouTube Live Comment Panel :)

[saltysaltk]

I hope that it will happen because neos vr has a lot to officer which is great for small content creators. Many of the streamers start at YouTube for many reasons so hopefully I could see the YouTube chat thing in game too which could make a lot of difference!

[Frooxius]

We started looking into this one on how it actually could be implemented, but so far it seems this is a fair bit more tricky than Twitch. YouTube requires all API requests to be authenticated and they have a limit on how many can be made each day.

If we do requests using a shared key for everyone, we'll probably run out very quickly, unless YouTube grants us expansion.

A potential alternative might be that you link up your YouTube account to Neos account, which will make Neos request a personal key for you. It would also mean you could only ever view your own chat, unlike the Twitch where you enter anyone's.

Would that solution work well enough?

Or alternatively, does anyone have more insights into YouTube's API that might help here?

[DovahDoVolom]

Would binding an account provide more functionality for things like chat moderation and what looks like YouTube calls "Cuepoints"? Also what does this mean for sharing a chat window bound to a users YouTube, Would anyone with access to that window be able to do things like delete messages or ban users and if not can those actions be allowed for a moderator inside of Neos if their account is tied to YouTube and the are moderator for that users chat?

[Hambonethewise]

I know a bit about the YouTube API

I definitely think you should at least start the process (as I did when I needed access to YouTube API) and get access even with the limited amount of API requests in a day (which is still somewhere around 10,000). Then request the expansion, keep in mind though that you only REALLY need it for things like to determine whether or not a person is a member to a channel, maybe when they super chat something etc.

Having another way to make Neos read chat or even possibly integrate through something like stream elements API with their alert system, I am sure can easily read chat like a chatbot does and do commands based on what they type (!example).

I’d be ok personally with only myself seeing chat, at the very least as a start and having that be able to potentially give us access to edit effects for in game, hopefully with some basic options like a cooldown in the same way a chatbot would operate.

I know we can also drag YouTube chat links into the game if necessary too if everyone wanted to see it, much like browser sources in OBS. I can even walk through what that would look like in a sense if interested.

Hope that helps, anything is better than nothing for now even if it’s limited on capability. Then maybe we can find ways to improve it later on but at least having Neos be able to read chat and have things respond to commands would be a huge deal!

[Frooxius]

I've looked into it some more, but unfortunately every time I do I keep hitting a wall.

Unfortunately I can't even get the official .NET samples they provide to work even when I register a fresh application, they all just fail with "API Quota Exceeded" on any request, despite not having any quota usage in the dashboard. I've tried waiting a month or so and registering new projects & client keys, but I'm not sure how to get even something basic to work.

From what I found each check for the live chat messages costs 1-5 units and you need to keep re-checking often to get any new messages, so that 10000 would run out very quick with multiple users (though with OAuth app everyone should have their own quota I think).

If anyone would like to get some proof of concept sample working in C# (e.g. a console application) that could help greatly for this or if you know if there's anything special to setup the API keys to get proper access.

Otherwise I think the Steam Elements API might be worth a check instead. @Hambonethewise can you elaborate on that one a bit? Do you get the YouTube chat messages through that somehow? I'm not too familiar with it, so I'm not too sure where to get started.

[Danyy59]

I think japan community made youtube chatbox.

[Frooxius]

@Danyy59 Oh cool! Do you know how is it implemented?

[Mookoo-m]

@Frooxius This is what Rabbuttz told me "Using "multi comment viewer" to get youtube comments and hosting local websocket server which is an extension by hinano.aira to send the comments to Neos"
https://ryu-s.github.io/app/multicommentviewer

[Frooxius]

@Mookoo-m Thank you for the info! Looks like it's based on a 3rd party Japanese tool. I've found the source of it, but it's licensed under GPL so we couldn't use it directly. I haven't found how exactly it actually communicates with the YouTube API yet, but I haven't dug around it too much. It almost looks like it might be scraping the data from the website?

[kazu0617]

I read through it. it's scraping.
https://github.com/CommentViewerCollection/MultiCommentViewer/blob/db31953b31952d2f586a86b35cbae09be3c6ec1e/YouTubeLiveSitePlugin/Test2/VidResolver.cs#L116-L129

If you want to try it out, hinano.aira's public folder contains a plugin for it and a comment viewer that can be displayed in Neos.
neosrec:///U-hinanoaira/R-d7078c53-19d8-40e1-ba5d-c6ecc66f70fe

1. Donwload MultiCommentViewer from website https://ryu-s.github.io/app/multicommentviewer
2. Download MCV_NeosVRPlugin from hinano.aira's public (direct: https://cloudxstorage.blob.core.windows.net/assets/81d3f6d916d57f4a1437a0fcca62b7eca48c847f0b37b18d2a13e4ddacc781d9 ).
3. extract MCV_NeosVRPlugin and install to "plugins". Make sure it is installed in such a way that the dlls will appear directly after opening the folder (not in a hierarchical structure).
4. Launch MultiCommentViewer and click "接続を追加" and insert URL from below "URL,放送ID等". Make sure it is upcomming livestream url or livestreaming url.
5. Spawn CommentViewer form hinano.aira's public, click "option" and click Connect (if request hostrequest, accept it).
6. You can now check comments.
7. If you want generate Comment, spawn CommentGenerator too and click Connect.
8. You can now check comments and hold it.

(Here's how it looked when I picked up Gawr Gura Ch.'s comment and tested it)