"Video not found. youtube.playlistItem" when adding to a playlist

[RoelVdP]

Describe the bug

I get the following on a subset of my uploads:

Adding video to playlist: ....playlistid_shown....
[RequestError] Server response: {
  "error": {
    "code": 404,
    "message": "Video not found.",
    "errors": [
      {
        "message": "Video not found.",
        "domain": "youtube.playlistItem",
        "reason": "videoNotFound"
      }
    ]
  }
}

This may be related to having many playlists or many video's in the channel, i.e. some sort of timeout.

To Reproduce
Upload regularly to a channel with many large playlists and many video's

Expected behavior
No such error, as is shown on the other subset of my uploads. Really does like a timeout issue.

Desktop (please complete the following information):

• OS: Linux Ubuntu
• OS Version 20.04
• Python Version 2.7.17

Additional context
Please check if the timeout can be increased? Perhaps few seconds more? Happy to test any proposed fixes? Thank you

[schneemaier]

Having the same issue in the last couple of days. It is 'random'. In 5 uploads it happens 2 times.

[RoelVdP]

Agreed, that is exactly what I see. It has been around for maybe 1-3 weeks now. I have not checked yet if the video's actually make it into the playlist. It really seems like an easy-to-fix timeout or similar.

[schneemaier]

So the error code is 404 (Not found). This seems to be a change on Youtube end. So far the video ID was available directly after upload, this seems to be a little slower now.

The questions is how much slower. Adding a 1-5 sec wait between the upload and the playlist addition may solve the issue, but what if the wait time has to be >6 seconds?

[RoelVdP]

@schneemaier Interesting idea (to at least try the 1-5 sec addition). Do you know where to patch this in the code please?

[schneemaier]

Sorry, did not look into the code, so i have no idea where this delay can be added

[mauryfs]

Same issue here. I only have two playlists on the account and less than 30 videos in each playlist. The requests I'm making are also uploading a custom thumbnail.

[RoelVdP]

Same here.

Could we have a fix please?

[vigilantspb]

using old 2016 version, same problem
updated - got tons of errors) rolled back, waiting for solution

[RoelVdP]

@vigilantspb what sort of errors, these ones or?

[vigilantspb]

@RoelVdP nope, same as here #298

and smth like

Traceback (most recent call last): File "/root/youtube-upload-master/bin/youtube-upload", line 9, in <module> from youtube_upload import main File "/root/youtube-upload-master/bin/../youtube_upload/main.py", line 286 raise RequestError(u"Server response: {0}".format(response)) ^ SyntaxError: invalid syntax

[vigilantspb]

channel has 10.000+ videos in 20 playlists, with this situation uploading fails from time to time. Sometimes with this error video appears on the channel, sometimes not

[deepaktalwardt]

Same issue! Waiting for a solution.

[RoelVdP]

It may be a very easy to insert a "sleep" patch in the code somewhere?

It's starting to happen more frequently too, almost every video now.

Adding video to playlist: ....ID....
[RequestError] Server response: {
  "error": {
    "code": 404,
    "message": "Video not found.",
    "errors": [
      {
        "message": "Video not found.",
        "domain": "youtube.playlistItem",
        "reason": "videoNotFound"
      }
    ]
  }
}

[vigilantspb]

@RoelVdP should be easy but nobody knows the answer :)

[schneemaier]

Created a fork: https://github.com/schneemaier/youtube-upload

Added 5 seconds wait between the video upload and any other operations on the video in main.py

`def run_main(parser, options, args, output=sys.stdout):
"""Run the main scripts from the parsed options/args."""
parse_options_error(parser, options)
youtube = get_youtube_handler(options)

if youtube:
    for index, video_path in enumerate(args):
        video_id = upload_youtube_video(youtube, options, video_path, len(args), index)
        video_url = WATCH_VIDEO_URL.format(id=video_id)
        debug("Video URL: {0}".format(video_url))
        # Adding sleep(5) to test fix for https://github.com/tokland/youtube-upload/issues/305
        debug("Sleep 5 sec, to test https://github.com/tokland/youtube-upload/issues/305")
        sleep(5)
        if options.open_link:
            open_link(video_url)  # Opens the Youtube Video's link in a webbrowser

        if options.thumb:
            youtube.thumbnails().set(videoId=video_id, media_body=options.thumb).execute()
        if options.playlist:
            playlists.add_video_to_playlist(youtube, video_id,
                                            title=lib.to_utf8(options.playlist), privacy=options.privacy)
        output.write(video_id + "\n")
else:
    raise AuthenticationError("Cannot get youtube resource")

`

I don't have time for testing now, but feel free to download it from my fork.

This is not a nice solution and it is only for testing!

[vigilantspb]

added sleep to my main.py (didn't download new file, just added "from time import sleep" & "from io import open" in the beginning and sleep(7) after "debug("Video URL: {0}".format(video_url))", got same error as before

[vigilantspb]

It is strange, I have scripts run at night, generating and uploading timelapses from several ip cameras, videos 1-2 minutes, same options, same channel, each goes to a playlist. Every night video for one playlist fails to upload, for another it's ok.. ) No difference, failing playlist is shorter..

[schneemaier]

added sleep to my main.py (didn't download new file, just added "from time import sleep" & "from io import open" in the beginning and sleep(7) after "debug("Video URL: {0}".format(video_url))", got same error as before

Just to make it sure: did you run sudo python setup.py install in the main youtube-upload directory to install the new modified version?

[vigilantspb]

@schneemaier without setup.py cause it was not changed

[schneemaier]

@schneemaier without setup.py cause it was not changed

@vigilantspb do you call main.py directly or you sue the official youtube-upload command? If you modify any of the files, you have to re-compile and re-install with the sudo python setup.py install the package, otherwise the youtube-upload command will still use the original code

[vigilantspb]

@ schneemaier at first I made some mistake in the code, upload failed with

Traceback (most recent call last):
  File "/root/youtube-upload-master/bin/youtube-upload", line 10, in <module>
    main.run()
  File "/root/youtube-upload-master/bin/../youtube_upload/main.py", line 252, in run
    sys.exit(lib.catch_exceptions(EXIT_CODES, main, sys.argv[1:]))
  File "/root/youtube-upload-master/bin/../youtube_upload/lib.py", line 39, in catch_exceptions
    fun(*args, **kwargs)
  File "/root/youtube-upload-master/bin/../youtube_upload/main.py", line 247, in main
    run_main(parser, options, args)
  File "/root/youtube-upload-master/bin/../youtube_upload/main.py", line 181, in run_main
    sleep(7)

NameError: global name 'sleep' is not defined

so I decided it should be fine without any additional movements) fixed and it uploaded with same error as before (404 etc)

[sam-moreton]

Having the same issue, tested the fork above with no luck.

The video uploads to youtube but the code produces error:

[RequestError] Server response: {
  "error": {
    "code": 404,
    "message": "Video not found.",
    "errors": [
      {
        "message": "Video not found.",
        "domain": "youtube.playlistItem",
        "reason": "videoNotFound"
      }
    ]
  }
}

[RoelVdP]

@vigilantspb you mentioned above your video actually does not upload? It thought that (for this issue) the video's succeed in uploading, but only the playlist addition fails? Or do the actual video uploads sometimes fail also?

[schneemaier]

I had some time to do a few test. Interestingly both the video upload and the playlist addition works (i can see the video on the correct playlist in youtube), but the result is still failed on the client end.

I tried it with 20 sec wait between the upload and the playlist addition, thus this seem to be an issue on Youtube end, which cannot be fixed form this code :(

[RoelVdP]

Thank you for the update!

[sam-moreton]

I had some time to do a few test. Interestingly both the video upload and the playlist addition works (i can see the video on the correct playlist in youtube), but the result is still failed on the client end.

I tried it with 20 sec wait between the upload and the playlist addition, thus this seem to be an issue on Youtube end, which cannot be fixed form this code :(

This is my experience too

[mgiacomo72]

Hi everyone,

I have the same problem, has anyone found a solution or a workaround?

Thanks in advance

[schneemaier]

I found that after upload the video is available in youtube and it is in the playlist despite the error message. You can check if the error is 404, but the function returned a video id then just ignore the error message. I updated my scripts to check for the video ID after the error, and ignore it if the video id is present

[mgiacomo72]

Thanks for your reply schneemaier, can you share your updated script?

[RoelVdP]

@schneemaier yeah I do the same by scanning the log for that failure and ignoring. Seems your method is better as you check for VideoID. Can you share your script. Thank you for confirming it actually does add to the playlist.

[schneemaier]

This is what i use:

lnk='/usr/local/bin/youtube-upload` --credentials-file="$credFile" --client-secrets="$clieFile" --privacy="unlisted" --title="$cm" --recording-date="$dateRec" --description="$date" --category="Music" --playlist="$cm" $i'
if [ $? -eq 0 ]
then
echo "Everything OK"
else
if [ $lnk -ne "" ]
then
echo "Upload ok, but got error"
else
echo "Upload Error"
fi
fi

[RoelVdP]

@schneemaier thanks. Please use code blocks. Your solution only checks for success/fail, not specific errors.

[RoelVdP]

The error has changed.

Adding video to playlist: ....name....
[RequestError] Server response: {
  "error": {
    "code": 503,
    "message": "The service is currently unavailable.",
    "errors": [
      {
        "domain": "youtube.CoreErrorDomain",
        "reason": "SERVICE_UNAVAILABLE"
      }
    ],
    "status": "UNAVAILABLE"
  }
}

The videos still seem to upload just fine otherwise.

[RoelVdP]

If you pipe the result/output of youtube-upload to an output file like /dev/shm/upload_last.log, you can some something like:

PLAYLIST_ERROR_BUT_STILL_OK_UPLOAD=0
if [ "$(grep -oi "youtube.playlistItem" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Likely playlist addition error, sporadic issue
  if [ "$(grep -oi "Video not found" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Confirmation of the same
    if [ "$(grep -oi "Video URL: https://www.youtube.com/watch?v=" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Ensuring video really uploaded
      echo 'Known sporadic issue with YouTube: video not found in youtube.playlistItem on/after adding of video to playlist; not a real problem...' | tee -a /dev/shm/upload_last.log
      PLAYLIST_ERROR_BUT_STILL_OK_UPLOAD=1
    fi
  fi
elif [ "$(grep -oi "youtube.CoreErrorDomain" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Likely playlist addition error, sporadic or changing issue
  if [ "$(grep -oi "SERVICE_UNAVAILABLE" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Confirmation of the same
    if [ "$(grep -oi "Video URL: https://www.youtube.com/watch?v=" /dev/shm/upload_last.log | head -n1)" != "" ]; then  # Ensuring video really uploaded
      echo 'Known sporadic issue with YouTube: video not found in youtube.playlistItem on/after adding of video to playlist; not a real problem...' | tee -a /dev/shm/upload_last.log
      PLAYLIST_ERROR_BUT_STILL_OK_UPLOAD=1
    fi
  fi
fi
if [ ${PLAYLIST_ERROR_BUT_STILL_OK_UPLOAD} -eq 1 ]; then
  echo "All good"
else
  echo "Uh oh"
fi

[RoelVdP]

The YouTube error seems to have disappeared for the moment.

[RoelVdP]

Yep, seems fixed on YouTube side. Closing ftm. Feel free to re-open if you see it again after this comment.