-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is vtt subtitle format supported? #1168
Comments
@Marzal I'll try to take a look at this issue this week. It might just be due to a simple bug / oversight... |
Thanks, now that I have the transcripts working in the web client using Should I open a separate issue for that in its repo? |
@Marzal could you create another issue in podverse-rn repo? And include a link to this issue in the description? If it's working in one but not the other that is odd...but also might point to a simple fix. |
@Marzal unrelated to this ticket, but I see 2 instances of KDE Express in Podcast Index (and Podverse, because we sync with PI). Is that intended? If not, please email info@podcastindex.org and let them know which one should be removed. |
@Marzal is this issue still reproducible in web or mobile after the mobile v4.13.6 update? If no, please close this ticket. Thanks |
Dave is aware of the duplicate, we are in works to fix it. Right now is a consequence of the framework generator that KDE Express uses. For context: skymethod/op3#16 (comment) About the VTT, I will try to find another podcast that only uses VTT, or does Podverse refresh the cached feed every X hours/days? Right now all episodes have both VTT and SRT so it's not a valid feed to test. And deleting the srt in one episode I'm not sure if it will works, as PI doesn't re-parse the old episodes (right know PI don't even show transcript in the most old ones). I did ask Dave to refresh the feeds a few days and it worked, and I will have to asking again when I'm done with the changes I'm doing to the feed generator. I'm also studding how podping works in case this will force a total refresh or the feed in PI and will make not necessary the manual intervention of Dave. And about transcripts: |
@Marzal we don't have any special caching for transcripts as far as I know...if the url path to a transcript updates in an RSS feed, we just need to make sure the feed gets re-parsed so our client apps use the new transcript path. Ok if you find a VTT we can test with please let me know. And good catch on the membership asterisk! That is indeed out-of-date. I just deployed a fix to the website. |
@mitchdowney as there is no cache I've just changed the feed in order to have episode 16 (the one that I originally linked ) with only vtt subs and the description to be able to check that is re-parsed. How is the policy to re-parse the feeds? when a new episode / lastBuildDate is detected, the whole feed is reload? |
@Marzal this is ugly, ugly code (our parser is probably the ugliest of all 🤦♂️) but here is the logic that controls whether if, when parsing a feed, a "change" is detected, and if a change is detected, then the parser will continue to parse the rest of the feed and update in our database. Our problem with updates is either 1) Podcast Index is not notifying us when a feed changes BUT does not have a new episode, or 2) this logic I am linking to has a flaw that results in it only working when a new episode is detected. https://github.com/podverse/podverse-api/blob/develop/src/services/parser.ts#L320-L331 As for transcripts...I just ran our api locally so I could view error logs when I try to request a transcript directly. Our API actually has a "priority" system that will use whichever transcript is available that is the "most reliable" or common according to what we've seen. In this case, even though episode 19 of KDE Express has a
When our API made a request to that API, it received the following error response:
And when I tried to open that URL in a browser, it displayed: Is archive.org somehow involved in your transcript hosting? If yes, I would not expect archive.org to be a reliable host for this kind of content, as they intend for their files to be accessed relatively infrequently, and my guess is they might auto-block IPs as "spam" after just a few tries. The "transcript proxy" logic from our API (we need to use this proxy endpoint to avoid CORS errors in web browsers) can be found here: https://github.com/podverse/podverse-api/blob/develop/src/routes/episode.ts#L127 It's very difficult for me to say at this time if this issue has anything to do with |
I tried to follow where is getting
Both audio and subs are hosted in archive.org (the feed and web are in Gitlab Pages), we have done lots of tests/downlaods with a bigger Podcast (Podcast Linux for instance) and never have any problem/502 (it slow that for sure, but enough to have streaming audio) maybe is because of the CDN that archive uses based in geolocation. As the subs are very small I can try to leave them in Gitlab Pages. I did see the priorities in https://github.com/podverse/podverse-api/blob/e3e6e4074f384e28ab60ef1fda7a61653ad7e10c/src/routes/episode.ts#L98 as far as I understand it , And last ,for the VTT support, I did change the feed to only offer VTT in episode 16, if you can re-parse the feed manually we can see if that episode uses VTT of the transcript option disappear from the web or the F-Droid app. Thanks for all the explanations about the app |
Another thing I've found is that for episodes 09 and 10 that do advertise JSON, VTT and SRT transcripts, neither the web or the app shows the transcripts. Not sure if the JSON is not what the Podverse expects https://podverse.fm/episode/hazOEWk-m2 https://archive.org/download/kde-express-akademy-es EDIT: offtopic, will open another issue in the future |
Ok...well I'm sorry, I'm pretty stumped on how to proceed at this point. I'm also not exactly sure what we're supposed to solve for this ticket? Is the issue possibly broader than vtt? Do vtt files that other people generate and host also have this issue? I may have to defer to someone else to work on this as I don't honestly understand the regex and parsing logic for the transcript files. In case someone can explore this more soon, here are I think the most relevant files: https://github.com/podverse/podverse-shared/blob/master/src/transcript.ts |
Sorry if I went a bit off topic, to summarize the original question: Now, in order to test if podverse-web actually works well with VTT I see 3 options:
|
OK, I see that now than the feed has being re-parsed, Podverse (both web and F-Droid beta) do detect that there is an transcript but don't show any text: https://podverse.fm/episode/yuY4qZ4Cc Tested the other episodes and they work if the have the SRT format. |
@Marzal ok cool, this test case helps a lot. When I test this transcript using podverse-api and podverse-web locally, I see that our api successfully returns the
It appears this error may be surfacing from the transcriptator library which @stevencrader maintains. Steven I just tested this with the latest 1.1.2 version of transcriptator. Any chance you could verify sometime if this is a bug with transcriptator? It seems like it may not be successfully parsing this vtt transcript? Here is the full response our api sends to our web app:
|
Thanks for the ping. I'll take a look. |
I published a new version of transcriptator (1.1.3) that parses this VTT file. The reason it is saying it isn't valid VTT is because the SRT parser is used for parsing VTT files because the formats are very similar and I re-use the parsing code. This VTT file didn't have and index line so it wasn't being parsed right. The only VTT example I had before this had an index line so I made the assumption that they all do. |
Great, thank you both. I assume that Podverse Android/F-Droid would need a new beta to test it, right? I did some changes to a few episodes so I can test differents variants, in case you need test cases:
|
@Marzal I actually just submitted a new version 4.13.8 of the mobile app to the stores...and I'm not 100% sure if the new versions will have pulled in the newer version of transcriptator, or I built them right before the new version of transcriptator was published. So there's a chance this is fixed in 4.13.8, but if it's not, then it will definitely be fixed in 4.13.9. Also, since this is a feature for Podverse web, and web seems to be fixed, I'm going to go ahead and close this issue. I think the different test case scenarios you're referring to might make more sense posted to the transcriptator repo https://github.com/stevencrader/transcriptator |
Describe the bug
Transcripts won't show in a feed with vtt subtitles
¿Is vtt supported? It's mentioned in
podverse-web/src/lib/utility/transcript.ts
Line 8 in 2df1615
To Reproduce
Steps to reproduce the behavior:
getParsedTranscript error: TypeError: Data is not valid SRT format
Expected behavior
Subtitles are shown or a warning about format not supported
Screenshots
![imagen](https://private-user-images.githubusercontent.com/2069735/260172247-746b32fd-36a2-4d54-9f4c-771e0c58be82.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU2NjIwMTcsIm5iZiI6MTcyNTY2MTcxNywicGF0aCI6Ii8yMDY5NzM1LzI2MDE3MjI0Ny03NDZiMzJmZC0zNmEyLTRkNTQtOWY0Yy03NzFlMGM1OGJlODIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDZUMjIyODM3WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MGFkYmY2ODM0ZDI3YTc0NmQyNTc1ZmYzOWRmNDE1YmFiZDgxZjY5ZGNlN2E2ZWFmNmNmNjZkNzhmYzZlZDBlZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.k20xqKq9-lLbCH_XZuorqCGFunHPG7LaKK3ODOggchg)
Example episode and error
Desktop (please complete the following information):
Additional context
Android app doesn't show the subs also.
The text was updated successfully, but these errors were encountered: