Get Rid of 100-Element Limitation (#158)

* Add BUILDME.md to pyinstaller binaries

* More than 100 items (#156)

* Revamp tidal_wave/playlist.py to handle > 100 items

* Revamp mix.py to handle paginated API responses

* Add logic for 'transparent' flag to mix.py

* Revamp album.Album.set_tracks() for albums with more than 100 tracks

* Add in playlist.request_playlist_items() the logic for 'transparent' flag

* Remove from playlist.flatten_playlist_dir() a duplicate "subdirs: Set[Path] = set()"

* Fix reference to `TidalPlaylistException` in mix.py

* Update README.md

Fix the instructions for long-running docker container execution

* Merge Upstream Branch `trunk` into `develop` (#157)

* Put back 'macOS [m]' in login.py prompt

* Fix typo of `tidal-wave` in README.md

It had been `tidal_wave` in the Docker run instructions

* Make README.md Changes

Point out the `--transparent` flag feature, a couple of other things, and update the TIDAL pricing plans documentation

* Format playlist.py with Black

* Format mix.py with Black

* Format video.py with Black

* Format album.py with Black

.github/workflows/pyinstaller-linux.yml

@@ -87,10 +87,12 @@ jobs:
       - name: Create Executable with PyInstaller
         run: |
           pyinstaller \
-            --name tidal-wave_ubuntu_amd64 \
+            --name tidal-wave_ubuntu_22.04_amd64 \
+            --target-arch=x86_64 \
             --paths tidal_wave \
             --exclude-module pyinstaller \
             --add-data "README.md:." \
+            --add-data "BUILDME.md:." \
             --add-binary "ffmpeg-n7.0/ffmpeg:." \
             --clean \
             --noupx \
@@ -100,8 +102,8 @@ jobs:
 
       - name: Test just-compiled binary
         run: |
-          chmod +x ./dist/tidal-wave_ubuntu_amd64
-          ./dist/tidal-wave_ubuntu_amd64 --help
+          chmod +x ./dist/tidal-wave_ubuntu_22.04_amd64
+          ./dist/tidal-wave_ubuntu_22.04_amd64 --help
 
       - name: Add artifact to release
         uses: softprops/action-gh-release@v2

.github/workflows/pyinstaller-macos_arm64.yml

@@ -118,6 +118,7 @@ jobs:
             --paths tidal_wave \
             --exclude-module pyinstaller \
             --add-data "README.md:." \
+            --add-data "BUILDME.md:." \
             --add-binary "ffmpeg-n7.0/ffmpeg:." \
             --clean \
             --noupx \

.github/workflows/pyinstaller-macos_x86.yml

@@ -114,6 +114,7 @@ jobs:
             --paths tidal_wave \
             --exclude-module pyinstaller \
             --add-data "README.md:." \
+            --add-data "BUILDME.md:." \
             --add-binary "ffmpeg-n7.0/ffmpeg:." \
             --clean \
             --noupx \

.github/workflows/pyinstaller-windows.yml

@@ -69,6 +69,7 @@ jobs:
             --name tidal-wave_windows.exe `
             --paths tidal_wave `
             --add-data "README.md:." `
+            --add-data "BUILDME.md:." `
             --add-binary "ffmpeg.exe:." `
             --exclude-module pyinstaller `
             --clean `

README.md

@@ -33,16 +33,21 @@ This software uses libraries from the [FFmpeg](http://ffmpeg.org) project under
 * Artist's entire works retrieval support (video and audio; albums or albums and EPs and singles)
 * Because of the use of the `requests` package, system proxies are respected (HTTP, HTTPs, Socks); or can be specified by typical environment variable
 * Also because of the use of `requests`, very simple [`Cache-Control`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control) request caching occurs via `CacheControl`
+* If desired, all JSON responses from the TIDAL API can be saved for inspection or posterity or debugging
 
 ## Getting Started
-A [HiFi Plus](https://tidal.com/pricing) account is **required** in order to retrieve HiRes FLAC, Dolby Atmos, and Sony 360 Reality Audio tracks. Simply a [HiFi](https://tidal.com/pricing) plan is sufficient to retrieve in 16-bit, 44.1 kHz (i.e., lossless) or lower quality as well as videos. More information on sound quality at [TIDAL's site here](https://tidal.com/sound-quality).
+A current, valid TIDAL subscription is required in order to run `tidal-wave`. Previously, TIDAL segmented the available audio qualities into HiFi and HiFi Plus plans: now, 
+> All current TIDAL plans feature Max sound quality formats such as full lossless, HiRes FLAC, and Dolby Atmos (up to 24-bit, 192 kHz).
+
+More information on sound quality at [TIDAL's site here](https://tidal.com/sound-quality).
 
 ### Requirements
  - As resources will be fetched from the World Wide Web, an Internet connection is required
- - The venerable [FFmpeg](http://ffmpeg.org/download.html) is necessary for audio and video data manipulation. This project's [container image](https://github.com/ebb-earl-co/tidal-wave/blob/trunk/Dockerfile) as well as its [PyInstaller](https://pyinstaller.org/en/stable/)-created [binary for Ubuntu GNU/Linux](https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_ubuntu_amd64), [binary for Apple Silicon macOS](https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_macos_aarch64), [binary for x86\_64 macOS](https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_macos_amd64), and [binary for x86\_64 Windows](https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_windows.exe) build FFmpeg from source, so separate installation is unnecessary.
+ - The venerable [FFmpeg](http://ffmpeg.org/download.html) is necessary for audio and video data manipulation. This project's [container image](https://github.com/ebb-earl-co/tidal-wave/blob/trunk/Dockerfile) as well as its [PyInstaller](https://pyinstaller.org/en/stable/)-created [binaries](https://github.com/ebb-earl-co/tidal-wave/releases/latest) build FFmpeg from source, so separate installation is unnecessary.
    - Static builds of FFmpeg are available from [John Van Sickle](https://www.johnvansickle.com/ffmpeg/) for GNU/Linux, or most package managers feature `ffmpeg`.
    - For macOS, the [FFmpeg download page](http://ffmpeg.org/download.html#build-mac) links to [this download source](https://evermeet.cx/ffmpeg/); or there is always [Homebrew](https://formulae.brew.sh/formula/ffmpeg)
    - For Windows, the [FFmpeg download page](http://ffmpeg.org/download.html#build-windows) lists 2 resources; or [`chocolatey`](https://community.chocolatey.org/packages/ffmpeg) is an option
+   - If a minimal FFmpeg, compiled from source, is desired, take a look at this project's [BUILDME.md](https://github.com/ebb-earl-co/tidal-wave/blob/trunk/BUILDME.md) file for decent instructions
  - This is a Python package, so **to use it in the default manner** you will need [Python 3](https://www.python.org/downloads/), version 3.8 or newer, on your system.
    - *However*, as of December 2023, an [OCI container image](https://github.com/ebb-earl-co/tidal-wave/pkgs/container/tidal-wave); and [PyInstaller](https://pyinstaller.org/en/stable/)-created binaries for x86\_64 GNU/Linux, Apple Silicon macOS, x86\_64 macOS, and x86\_64 Windows are provided for download and use that *do not require Python to be installed*
  - Only a handful of Python libraries are dependencies:
@@ -69,13 +74,6 @@ $ python3 -m pip install tidal-wave
 PS > python.exe -m pip install tidal-wave
 ```
 
-Optionally, to get the full `typer` experience when using this utility, add `[all]` to the end of the `pip install command`:
-```bash
-$ python3 -m pip install tidal-wave[all]
-```
-```powershell
-PS > python.exe -m pip install tidal-wave[all]
-```
 ### `pip` Install from the Repository
 Alternatively, you can clone this repository; `cd` into it; and install from there:
 ```bash
@@ -88,9 +86,9 @@ $ (.venv) pip install .
 ### PyInstaller executable
 These release artifacts are created with [PyInstaller](https://pyinstaller.org). It bundles Python 3.12.2, FFmpeg 7.0, and the `tidal-wave` program into one binary, licensed under the terms of FFmpeg: with the [GNU Lesser General Public License (LGPL) version 2.1](https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html). Installation is as simple as downloading the correct binary for your platform giving it execute permissions, and running it.
 ```bash
-$ wget https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_ubuntu_amd64
-$ chmod +x ./tidal-wave_ubuntu_amd64
-$ ./tidal-wave_ubuntu_amd64 --help
+$ wget https://github.com/ebb-earl-co/tidal-wave/releases/latest/download/tidal-wave_ubuntu_22.04_amd64
+$ chmod +x ./tidal-wave_ubuntu_22.04_amd64
+$ ./tidal-wave_ubuntu_22.04_amd64 --help
 ```
 
 ### Docker
@@ -125,7 +123,7 @@ Usage: python -m tidal_wave [OPTIONS] TIDAL_URL [OUTPUT_DIRECTORY]
 ## Usage
 Invocation of this tool will store credentials in a particular directory in the user's "home" directory: for Unix-like systems, this will be `/home/${USER}/.config/tidal-wave`: for Windows, it varies: in either OS situation, the exact directory is determined by the `user_config_path()` function of the `platformdirs` package.
 
-Similarly, by default, all media retrieved is placed in subdirectories of the user's default music directory: for Unix-like systems, this probably is `/home/${USER}/Music`; for Windows it is probably `C:\Users\<USER>\Music`. This directory is determined by `platformdirs.user_music_path()`. 
+Similarly, by default, all media retrieved is placed in subdirectories of the user's default music directory: for Unix-like systems, this probably is `/home/${USER}/Music`; for Windows it is probably `C:\Users\<USER>\Music`. This directory is determined by [`platformdirs.user_music_path()`](https://github.com/platformdirs/platformdirs?tab=readme-ov-file#platformdirs-to-the-rescue). 
  - If a different path is passed to the second CLI argument, `output_directory`, then all media is written to subdirectories of that directory.
 
 ### Which Audio Formats Are Available to Which Clients
@@ -249,11 +247,12 @@ $ docker run \
     --volume ./config/tidal-wave:/home/debian/.config/tidal-wave \
     --entrypoint "/bin/bash" \
     ghcr.io/ebb-earl-co/tidal-wave:latest
-$ docker exec -it tidal-wave python3 -m tidal_wave https://tidal.com/browse/album/...
-$ docker exec -it tidal-wave python3 -m tidal_wave https://tidal.com/browse/mix/...
-$ docker exec -it tidal-wave python3 -m tidal_wave https://tidal.com/browse/playlist/...
-$ docker exec -it tidal-wave python3 -m tidal_wave https://tidal.com/browse/track/...
+$ docker exec -it tidal-wave tidal-wave https://tidal.com/browse/album/...
+$ docker exec -it tidal-wave tidal-wave https://tidal.com/browse/mix/...
+$ docker exec -it tidal-wave tidal-wave https://tidal.com/browse/playlist/...
+$ docker exec -it tidal-wave tidal-wave https://tidal.com/browse/track/...
 ```
+Note: the first `tidal-wave` is whatever `--name` you give the container, so that can be whatever your heart desires, but the second `tidal-wave` is invoking the Python program *inside* the container and needs to exactly `tidal-wave`.
 ## Development
 The easiest way to start working on development is to fork this project on GitHub, or clone the repository to your local machine and do the pull requesting on GitHub later. In any case, there will need to be some getting from GitHub first, so, roughly, the process is:
   1. Get Python 3.8+ on your system

tidal_wave/album.py

@@ -18,7 +18,7 @@
 from .requesting import (
     request_albums,
     request_albums_credits,
-    request_album_items,
+    request_albums_items,
     request_album_review,
 )
 from .track import Track
@@ -39,10 +39,32 @@ def __post_init__(self):
     def set_tracks(self, session: Session):
         """This method populates self.tracks by requesting from
         TIDAL albums/items endpoint."""
-        album_items: AlbumsItemsResponseJSON = request_album_items(
+        album_items: Optional[AlbumsItemsResponseJSON] = request_albums_items(
             session=session, album_id=self.album_id, transparent=self.transparent
         )
-        _items = album_items.items if album_items is not None else ()
+        _items = album_items.items if album_items is not None else []
+        if self.metadata.number_of_tracks > len(_items):
+            items_to_retrieve: int = self.metadata.number_of_tracks - len(_items)
+            offset: int = 100
+            while items_to_retrieve > 0:
+                airj: Optional[AlbumsItemsResponseJSON] = request_albums_items(
+                    session=session,
+                    album_id=self.album_id,
+                    transparent=self.transparent,
+                    offset=offset,
+                )
+                if (airj is not None) and (airj.items is not None):
+                    _items += airj.items
+                    offset += 100
+                    items_to_retrieve -= 100
+                else:
+                    logger.warning(
+                        f"Could not retrieve more than {len(_items)} "
+                        f"tracks of album '{self.album_id}'. Continuing "
+                        "without the remaining "
+                        f"{self.metadata.number_of_tracks - len(_items)}"
+                    )
+
         self.tracks: Tuple[TracksEndpointResponseJSON] = tuple(
             _item.item for _item in _items
         )