New CLI Flag for Playlist and Mix Behavior (#83)

* Add CLI flag `--no-flatten` to control Playlist and Mix behavior (#82)

Also, add some type hints in dash.py and update some doc strings

* Update README.md with new CLI flag example

README.md

@@ -107,6 +107,7 @@ Usage: python -m tidal_wave [OPTIONS] TIDAL_URL [OUTPUT_DIRECTORY]
 │ --loglevel                   [DEBUG|INFO|WARNING|ERROR|CRITICAL]      [default: INFO]                                                                                                                                             │
 │ --include-eps-singles                                                 No-op unless passing TIDAL artist. Whether to include artist's EPs and singles with albums                                                                  │
 │ --no-extra-files                                                      Whether to not even attempt to retrieve artist bio, artist image, album review, or playlist m3u8                                                            │
+│ --no-flatten                                                          Whether to treat playlists or mixes as a list of tracks/videos and, as such, retrieve them independently                                                    │
 │ --install-completion                                                  Install completion for the current shell.                                                                                                                   │
 │ --show-completion                                                     Show completion for the current shell, to copy it or customize the installation.                                                                            │
 │ --help                                                                Show this message and exit.                                                                                                                                 │
@@ -145,8 +146,8 @@ Similarly, all media retrieved is placed in subdirectories of the user's default
  ```
 
  - To (attempt to) get a playlist, the following will do that. **N.b.** passing anything to `--audio-format` is a no-op when retrieving videos.
- ```bash
- > .\tidal-wave_py311_pyapp.exe https://tidal.com/browse/playlist/...
+ ```powershell
+ PS C:\Users\User > & tidal-wave_py311_pyapp.exe https://tidal.com/browse/playlist/...
  ```
 
  - To (attempt to) get a mix, the following will do that. **N.b.** passing anything to `--audio-format` is a no-op when retrieving videos.
@@ -163,6 +164,26 @@ Similarly, all media retrieved is placed in subdirectories of the user's default
  ```bash
  (.venv) $ tidal-wave https://listen.tidal.com/artist/... --audio-format hires --include-eps-singles
  ```
+#### Playlists and Mixes
+By default, when passed a playlist or mix URL, `tidal-wave` will retrieve all of the tracks and/or videos specified by that URL, and write them to a subdirectory of either `Playlists` or `Mixes`, which itself is a subdirectory of the specified `output_directory`. E.g. `~/Music/Mixes/My Daily Discovery [016dccd302e9ac6132d8334cfbc022]`. In this directory, once all of the tracks and/or videos have been retrieved, they are renamed based on the order in which they appear in the playlist. E.g.
+```bash
+(.venv) $ tidal-wave https://listen.tidal.com/playlist/1b418bb8-90a7-4f87-901d-707993838346
+
+$ ls ~/Music/Playlists/New Arrivals [1b418bb8-90a7-4f87-901d-707993838346]/
+'001 - Dance Alone [CD].flac'
+'002 - Kissing Strangers [CD].flac'
+'003 - Sunday Service [CD].flac'
+```
+If this is not the desired behavior, pass the `--no-flatten` flag. This flag instructs `tidal-wave` to leave the tracks and/or videos in the directory where they would be written if they had been passed to `tidal-wave` independently. E.g.
+```bash
+(.venv) $ tidal-wave https://listen.tidal.com/playlist/1b418bb8-90a7-4f87-901d-707993838346 --no-flatten
+
+$ ls ~/Music/
+'Sia/Dance Alone [343225498] [2024]/01 - Dance Alone [CD].flac'
+'USHER/COMING HOME [339249017] [2024]/05 - Kissing Strangers [CD].flac'
+'Latto/Sunday Service [344275657] [2024]/01 - Sunday Service [CD].flac'
+```
+
 #### Docker example
 The command line options are the same for the Python invocation, but in order to save configuration and audio data, volumes need to be passed. If they are bind mounts to directories, **they must be created before executing `docker run` to avoid permissions issues**! For example,
 ```bash
@@ -220,5 +241,5 @@ The easiest way to start working on development is to fork this project on GitHu
   5. From a Python REPL (or, my preferred method, an iPython session), import all the relevant modules, or the targeted ones for development:
   ```python
   from tidal_wave import album, artist, dash, hls, login, main, media, mix, models, oauth, playlist, requesting, track, utils, video
-  from tidal_wave.main import *
+  from tidal_wave.main import logging, user_music_path, Path
   ```

tidal_wave/dash.py

@@ -20,8 +20,8 @@ class S:
     r: Optional[str] = field(default=None)
 
     def __post_init__(self):
-        self.d = int(self.d) if self.d is not None else None
-        self.r = int(self.r) if self.r is not None else None
+        self.d: Optional[int] = int(self.d) if self.d is not None else None
+        self.r: Optional[int] = int(self.r) if self.r is not None else None
 
 
 @dataclass(frozen=True)
@@ -51,33 +51,38 @@ class XMLDASHManifest:
     segment_timeline: Optional["SegmentTimeline"] = field(default=None, repr=False)
 
     def __post_init__(self):
-        self.bandwidth = int(self.bandwidth) if self.bandwidth is not None else None
-        self.audio_sampling_rate = (
+        self.bandwidth: Optional[int] = (
+            int(self.bandwidth) if self.bandwidth is not None else None
+        )
+        self.audio_sampling_rate: Optional[int] = (
             int(self.audio_sampling_rate)
             if self.audio_sampling_rate is not None
             else None
         )
-        self.timescale = int(self.timescale) if self.timescale is not None else None
-        self.startNumber = (
+        self.timescale: Optional[int] = (
+            int(self.timescale) if self.timescale is not None else None
+        )
+        self.startNumber: Optional[int] = (
             int(self.start_number) if self.start_number is not None else None
         )
 
     def build_urls(self, session: Session) -> Optional[List[str]]:
-        """Parse the MPEG-DASH manifest in the way that it was *supposed* to
-        be parsed, with a few network calls because we aren't actually THAT
-        clever."""
+        """Parse the MPEG-DASH manifest into a list of URLs. In
+        particular, look for a special value, r, in self.segment_timeline.s.
+        If there is no such value, set r=1. In both cases, start substituting
+        r into the special substring, '$Number$', in self.initialization.
+        Continue incrementing r and substituting until the resulting string
+        returns a 500 error to a HEAD request."""
         if len(self.segment_timeline.s) == 0:
             return
 
         def sub_number(n: int, p: str = r"\$Number\$", s: str = self.media) -> str:
             return re.sub(p, str(n), s)
 
         try:
-            _r: Optional[str] = next(S.r for S in self.segment_timeline.s)
+            r: Optional[int] = next(S.r for S in self.segment_timeline.s)
         except StopIteration:
-            return
-        else:
-            r: Optional[int] = int(_r) if _r is not None else None
+            r = None
 
         # New path for when r is None; e.g. TIDAL track 96154223
         if r is None:
@@ -93,8 +98,6 @@ def sub_number(n: int, p: str = r"\$Number\$", s: str = self.media) -> str:
             urls_list: List[str] = [self.initialization] + [
                 sub_number(i) for i in number_range
             ]
-            # Add each value of self.initialization with incremented `r`
-            # that doesn't result in a 500 response to a HEAD request.
             number: int = r + 1
             while session.head(url=sub_number(number)).status_code != 500:
                 urls_list.append(sub_number(number))

tidal_wave/main.py

@@ -61,6 +61,13 @@ def main(
             help="Whether to not even attempt to retrieve artist bio, artist image, album review, or playlist m3u8",
         ),
     ] = False,
+    no_flatten: Annotated[
+        bool,
+        typer.Option(
+            "--no-flatten",
+            help="Whether to treat playlists or mixes as a list of tracks/videos and, as such, retrieve them independently",
+        ),
+    ] = False,
 ):
     logging.basicConfig(
         format="%(asctime)s,%(msecs)03d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s",
@@ -127,24 +134,40 @@ def main(
             raise typer.Exit(code=0)
         elif isinstance(tidal_resource, TidalPlaylist):
             playlist = Playlist(playlist_id=tidal_resource.tidal_id)
-            playlist.get(
-                session=session,
-                audio_format=audio_format,
-                out_dir=output_directory,
-                no_extra_files=no_extra_files,
-            )
+            if no_flatten:
+                playlist.get_elements(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=output_directory,
+                    no_extra_files=no_extra_files,
+                )
+            else:
+                playlist.get(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=output_directory,
+                    no_extra_files=no_extra_files,
+                )
 
             if loglevel == LogLevel.debug:
                 playlist.dump()
             raise typer.Exit(code=0)
         elif isinstance(tidal_resource, TidalMix):
             mix = Mix(mix_id=tidal_resource.tidal_id)
-            mix.get(
-                session=session,
-                audio_format=audio_format,
-                out_dir=output_directory,
-                no_extra_files=no_extra_files,
-            )
+            if no_flatten:
+                mix.get_elements(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=output_directory,
+                    no_extra_files=no_extra_files,
+                )
+            else:
+                mix.get(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=output_directory,
+                    no_extra_files=no_extra_files,
+                )
 
             if loglevel == LogLevel.debug:
                 mix.dump()

tidal_wave/mix.py

@@ -80,9 +80,11 @@ def get_items(
         self, session: Session, audio_format: AudioFormat, no_extra_files: bool
     ):
         """Using either Track.get() or Video.get(), attempt to request
-        the data for each track or video in self.items"""
+        the data for each track or video in self.items."""
         if len(self.items) == 0:
+            self.files = {}
             return
+
         tracks_videos: list = [None] * len(self.items)
         for i, item in enumerate(self.items):
             if item is None:
@@ -104,7 +106,6 @@ def get_items(
                     session=session,
                     out_dir=self.mix_dir,
                     metadata=item,
-                    no_extra_files=no_extra_files,
                 )
                 tracks_videos[i] = video
             else:
@@ -236,12 +237,12 @@ def get(
           - self.set_metadata()
           - self.set_items()
           - self.set_mix_dir()
-          - self.save_cover_image()
           - self.get_items()
           - self.flatten_playlist_dir()
+        Then, if no_extra_files is False,
+          - self.save_cover_image()
         """
         self.set_metadata(session)
-
         if self.metadata is None:
             self.files = {}
             return
@@ -251,13 +252,67 @@ def get(
 
         if self.get_items(session, audio_format, no_extra_files) is None:
             logger.critical(f"Could not retrieve mix with ID '{self.mix_id}'")
+            self.files = {}
             return
+
         self.flatten_mix_dir()
         logger.info(f"Mix files written to '{self.mix_dir}'")
 
         if not no_extra_files:
             self.save_cover_image(session, out_dir)
 
+    def get_elements(
+        self,
+        session: Session,
+        audio_format: AudioFormat,
+        out_dir: Path,
+        no_extra_files: bool,
+    ):
+        """The main method of this class, executing a number of other methods
+        in a row:
+          - self.set_metadata()
+          - self.set_items()
+        """
+        self.set_metadata(session)
+        if self.metadata is None:
+            self.files = {}
+            return
+
+        self.set_items(session)
+        if len(self.items) == 0:
+            self.files = {}
+            return
+        else:
+            files: List[Optional[str]] = [None] * len(self.items)
+
+        for i, item in enumerate(self.items):
+            if item is None:
+                files[i] = None
+                continue
+            elif isinstance(item, TracksEndpointResponseJSON):
+                track: Track = Track(track_id=item.id)
+                track_file: Optional[str] = track.get(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=out_dir,
+                    metadata=item,
+                    no_extra_files=no_extra_files,
+                )
+                files[i] = track_file
+            elif isinstance(item, VideosEndpointResponseJSON):
+                video: Video = Video(video_id=item.id)
+                video_file: Optional[str] = video.get(
+                    session=session,
+                    out_dir=self.mix_dir,
+                    metadata=item,
+                )
+                files[i] = video_file
+            else:
+                files[i] = None
+                continue
+        else:
+            self.files: List[Optional[str]] = files
+
 
 class TidalMixException(Exception):
     pass

tidal_wave/playlist.py

@@ -93,7 +93,9 @@ def get_items(
         self, session: Session, audio_format: AudioFormat, no_extra_files: bool
     ):
         """Using either Track.get() or Video.get(), attempt to request
-        the data for each track or video in self.items"""
+        the data for each track or video in self.items. If no_extra_files
+        is True, do not attempt to retrieve or save any of: playlist
+        description text, playlist m3u8 text, playlist cover image."""
         if len(self.items) == 0:
             return
         tracks_videos: list = [None] * len(self.items)
@@ -322,9 +324,9 @@ def get(
         self.set_items(session)
         self.set_playlist_dir(out_dir)
 
-        _get_items = self.get_items(session, audio_format)
-        if _get_items is None:
+        if self.get_items(session, audio_format, no_extra_files) is None:
             logger.critical(f"Could not retrieve playlist with ID '{self.playlist_id}'")
+            self.files = {}
             return
 
         self.flatten_playlist_dir()
@@ -349,6 +351,60 @@ def get(
             else:
                 (self.playlist_dir / "playlist.m3u8").write_text(m3u8_text)
 
+    def get_elements(
+        self,
+        session: Session,
+        audio_format: AudioFormat,
+        out_dir: Path,
+        no_extra_files: bool,
+    ):
+        """The main method of this class when no_flatten is True at
+        the program top level. It executes a number of other methods
+        in a row:
+          - self.set_metadata()
+          - self.set_items()
+        """
+        self.set_metadata(session)
+
+        if self.metadata is None:
+            self.files = {}
+            return
+
+        self.set_items(session)
+        if len(self.items) == 0:
+            self.files = {}
+            return
+        else:
+            files: List[Optional[str]] = [None] * len(self.items)
+
+        for i, item in enumerate(self.items):
+            if item is None:
+                files[i] = None
+                continue
+            elif isinstance(item, TracksEndpointResponseJSON):
+                track: Track = Track(track_id=item.id)
+                track_file: Optional[str] = track.get(
+                    session=session,
+                    audio_format=audio_format,
+                    out_dir=out_dir,
+                    metadata=item,
+                    no_extra_files=no_extra_files,
+                )
+                files[i] = track_file
+            elif isinstance(item, VideosEndpointResponseJSON):
+                video: Video = Video(video_id=item.id)
+                video_file: Optional[str] = video.get(
+                    session=session,
+                    out_dir=out_dir,
+                    metadata=item,
+                )
+                files[i] = video_file
+            else:
+                files[i] = None
+                continue
+        else:
+            self.files: List[Optional[str]] = files
+
 
 class TidalPlaylistException(Exception):
     pass

tidal_wave/video.py

@@ -233,14 +233,16 @@ def get(
         outfile: Optional[Path] = self.set_outfile()
         if outfile is None:
             return None
+        else:
+            self.absolute_outfile = str(self.outfile.absolute())
 
         if self.download(session, out_dir) is None:
             return None
 
         return str(self.outfile.absolute())
 
     def dump(self, fp=sys.stdout):
-        json.dump({self.metadata.title: str(self.outfile.absolute())}, fp)
+        json.dump({self.metadata.title: self.absolute_outfile}, fp)
 
     def dumps(self) -> str:
-        return json.dumps({self.metadata.title: str(self.outfile.absolute())})
+        return json.dumps({self.metadata.title: self.absolute_outfile})