Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error parsing N subscriptions (N is ~10% of the total list) #49

Closed
VovaFreeman opened this issue Apr 6, 2022 · 6 comments · Fixed by #50
Closed

Error parsing N subscriptions (N is ~10% of the total list) #49

VovaFreeman opened this issue Apr 6, 2022 · 6 comments · Fixed by #50

Comments

@VovaFreeman
Copy link

VovaFreeman commented Apr 6, 2022

Tubefeeder constantly shows the Error message at the top of the window. About 10% of my ~300 subscriptions are not parsed. What information or logs can I provide to investigate this?

OS: Manjaro/Posh;
Build: On pinephone natively from latest source;
Start: From terminal Tubefeeder-1.6.2/target/release/tubefeeder;
STDERR: ERROR tf_platform_youtube::subscription Error generating youtube videos from subscription IDofAChannelHere;

The list was imported to csv file with bash. Double checked row entries and channels - all are working well in a browser or as separate entries.

@Schmiddiii
Copy link
Collaborator

Schmiddiii commented Apr 6, 2022

I will need a little more information:

  • Is the N varying or always the same number?
  • Can you provide a more detailed log using RUST_LOG=trace cargo run in your terminal (or similar). I will not need everything, the main interest is probably the lines above the error you mentioned.
  • Is there a pattern in above mentioned log, e.g. more failures at the end than at the start)?
  • At least one youtube ID where this failed would be nice if always the same channels fail.
  • Do you see the correct subscriptions (including names) in the subscription list? Do some subscriptions have their ID instead of their names (it will take a few moments before the names will be filled out, so please wait a while there)

@VovaFreeman
Copy link
Author

  • N is a constant for a day with constant playlist. (If a single subscription added/changed/deleted the whole picture changes: Number of unresolved channels and those channels);

  • RUST_LOG

  • Every 6th or 7th channel are not resolved (shown as IDs) but this is not consistent through a whole list. (sometimes its 2 together and 10 resolved after it). Most of the time there's a message at the top: "Error connecting to the network". But my pinephone is constantly connected to a 4G cellular network.

  • For example 10 unresolved IDs (for today with my main playlist):
    UC2_mG3h-AxxK2oz08j2bz2g
    UC4HgnAtfzjyUhYV83jBe3Kg
    UC5ZsENSXOVzI0zHt4i2bZFw
    UC7Awuu4OccGePAeQlwtI1Yg
    UCCj7k6twL9MD9-fVU__Rk5A
    UCDqHUzLZmlESwpoJ6-1bxaQ
    UCEhBM2x5MG9-e_JSOzU068w
    UCFpGp4hHe03nvF9c8_gF_jA
    UCFsZ2CadKpAt_yInoTcVRnQ
    UCFxiUzeQ6ZPtnzZpUyNj9jw;

  • Yes. All other channels are resolved (shown as Names) in the subscriptions tab and their new videos are shown in the feed if there are any.

@Schmiddiii
Copy link
Collaborator

Schmiddiii commented Apr 7, 2022

I already see a few problematic channels on your subscription list (from the log):

  • You query a channel with id Channel { here and one with id Id here and UCzml9bXoEM0itbcE96CB03w} where } is probably not allowed. Probably a error in the bash script of converstion.
  • The channel The Silenced with ID UCOeX05Mko7ffxOl3Gg85NeA seems to have been deleted. It should still be possible to parse this, so this is a bug.
  • From the list you have posted, the last ID has a ;. This is wrong, but correct in the log, probably just a copy-paste mistake on your side.
  • UCl4C5SCTEt8_mo0VasoV0BA violated YouTubes TOS so it cannot be parsed correctly. I do not think I can do anything sensible in this case.

Other channels seem fine, I still have to investigate it a little bit closer. I already found some null values where there should be none.

Schmiddiii added a commit to Tubefeeder/piped-rust-sdk that referenced this issue Apr 7, 2022
Some fields have been marked as optional. See
Tubefeeder/Pipeline#49 for some details and
examples.
@Schmiddiii
Copy link
Collaborator

I have fixed the parsing errors of (as far as I checked) all but the above mentioned problematic ones and made a PR to a upstream library used. As soon as this will be merged and a new version is released, I am going to publish a new version of Tubefeeder that should fix this issue. Thank you for your detailed logs.

@VovaFreeman
Copy link
Author

Thank you for pointing on errors in my csv and for a fix. Hope this will merge soon. Have a great day/

@Schmiddiii
Copy link
Collaborator

A new version should be published to Flathub in a few hours. Please try if this fixed all issues and reopen otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants