Utime failing worryingly often

[MaryWeeb]

• I've verified and I assure that I'm running youtube-dl 2017.02.24.1
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)

---

Last night I let my phone download a list of songs in a JSON file, formatted as

{
    "playlist_name": [
        "url_1",
        "url_2",
        ...
    ],
    ...
}

Today when I woke up, it was done, but a lot of songs had no duration. The code I used is as follows:

import json
import youtube_dl as ytdl

data = json.load(open("playlists.json"))

my_options = {
    'format': 'bestaudio/best',
    'noplaylist': True,
    'audioformat': 'flac', 
    'quiet':True, 
    'outtmpl': 'songs/{playlist}/%(title)s.flac'
}

for playlist, songs in data.items():
    opts = my_options.copy()
    opts['outtmpl'] = opts['outtmpl'].format(playlist=playlist)
    for song in playlist:
        with ytdl.YoutubeDL(opts) as ydl:
            try:
                ydl.download([song])  # Download all separately in case of errors
            except ytdl.DownloadError:
                pass

It downloads most songs just fine, with only a few failing due to copyright restrictions and similar.
However, a lot of them seem to say WARNING: Cannot update utime of file.
It seems to me that exactly those files have a duration of zero, and from all 7035 files downloaded this has affected about 3000.

Any idea why this has happened?

[MaryWeeb]

Since this got the cant-reproduce label, here is the full json file, and then environment it is in is as follows:

• Device: OnePlus3T
• OS: OxygenOS Latest
• Termux: Latest
• Python: 3.6.0
• Youtube-DL: Latest

[dstftw]

Issue template clearly states you should provide verbose log.

[MaryWeeb]

give me 18 hours and a way to log everything and I'll add one

[yan12125]

This can print more information:

diff --git a/youtube_dl/postprocessor/common.py b/youtube_dl/postprocessor/common.py
index 599dd1df2..3e603610b 100644
--- a/youtube_dl/postprocessor/common.py
+++ b/youtube_dl/postprocessor/common.py
@@ -58,8 +58,8 @@ class PostProcessor(object):
     def try_utime(self, path, atime, mtime, errnote='Cannot update utime of file'):
         try:
             os.utime(encodeFilename(path), (atime, mtime))
-        except Exception:
-            self._downloader.report_warning(errnote)
+        except Exception as e:
+            self._downloader.report_warning(errnote + ': ' + str(e))
 
     def _configuration_args(self, default=[]):
         return cli_configuration_args(self._downloader.params, 'postprocessor_args', default)

[MaryWeeb]

Here's the most verbose I could get it to be link

[yan12125]

Well, I guess just remove the try-except block makes debugging easier. Could you change youtube_dl/postprocessor/common.py and try again?

diff --git a/youtube_dl/postprocessor/common.py b/youtube_dl/postprocessor/common.py
index 599dd1df2..6a6fc3be9 100644
--- a/youtube_dl/postprocessor/common.py
+++ b/youtube_dl/postprocessor/common.py
@@ -56,10 +56,7 @@ class PostProcessor(object):
         return [], information  # by default, keep file and do nothing
 
     def try_utime(self, path, atime, mtime, errnote='Cannot update utime of file'):
-        try:
-            os.utime(encodeFilename(path), (atime, mtime))
-        except Exception:
-            self._downloader.report_warning(errnote)
+        os.utime(encodeFilename(path), (atime, mtime))
 
     def _configuration_args(self, default=[]):
         return cli_configuration_args(self._downloader.params, 'postprocessor_args', default)

[MaryWeeb]

how would I do that? I don't think my editor can access anything in /data

[yan12125]

Seems you're using termux? The termux shell can access its own files.

[MaryWeeb]

Editing using nano or vim is a pain though, I have tried to use them but it never seems to work properly.

[yan12125]

vim works just fine on my phone. If there's really a problem in vim, Termux can access files in the internal storage. (usually /sdcard)

[MaryWeeb]

How do I get youtube_dl to install on /sdcard?

[yan12125]

See https://github.com/rg3/youtube-dl/blob/master/README.md#developer-instructions

[MaryWeeb]

Did some testing, only seems to happen on both my mobile devices. Will investigate.

[MaryWeeb]

PC does not seem to raise utime warnings, however it does produce files with no length in the metadata.