docker on Windows

[ruoti]

In the documentation for using m4b-tool through Docker on Windows, you state "Note: If you are on windows, it might be difficult to make it work".

I was able to get m4b-tool to work on Windows through docker without too much challenge. There are two things that you must do:

1. Make sure Docker is running using the Hyper-V engine, not the WSL2 engine,
   • There is a bug in WSL2 that make certain file system enumeration calls fail. Unfortunately, those calls are used by PHP, causing m4b-tool to struggle to see many of the files on the file system during operation.
2. Replace $(pwd) with ${PWD} in docker commands:

• Using powershell, the alias command then becomes:

function m4b-tool { docker run -it --rm -v ${PWD}:/mnt sandreas/m4b-tool:latest $args }

Other than that, it works as you would expect. It might be worth adding a section on this to the docs for Windows users.

[sandreas]

Oh, this is a nice hint. Thank you very much for pointing this out.
I already had prepared some Tutorials here with WSL2, but as I do not have any Windows PC to experiment with at the moment, I did not manage to spin up a VM yet :-) Windows is definitely not my personal priority, but you prepared this pretty well. I'm trying to verify this in the next days.

[ruoti]

For reference, here at the bugs related to WSL2 and python (not limited to usage in docker).

microsoft/WSL#5074
https://bugs.php.net/bug.php?id=80227

I didn't try building in WSL2, just using docker, but based on these bugs I think the tool would also struggle in WSL2. To give a sense, I was merging around 400 books, It worked for all of them, but 1/4 of them were missing portions in the audiobook. Debugging showed that they were only grabbing a portion of the files. Moving to Hyper-V-based docker solved it and everything went to working as expected.

[akirataguchi115]

Or at the very least we could mention Windows AND wsl2 not working correctly rn. Currently the docker version runs flakey on my W11.

[kanjieater]

@ruoti I created a separate issue to track, but it seems like you might have seen or confirm not seeing some of the issues I'm mentioned in #222. Would you be able to advice or suggest any other things to try.

I have turned off my wsl2 integration (so I think I'm using hypervisor as I can only run it from the windows side now, powershell etc), and I still have the issue.

[sandreas]

As I already mentioned in #222, it may be possible, that using a bind mount instead of a -v fixes at least some of the issues.
Example

docker run -it --rm -u $(id -u):$(id -g) \
--mount type=bind,source=$FOLDER,target=/mnt
  sandreas/m4b-tool:latest merge "./$(echo $NAME)_splitted/" --output-file="./$NAME.m4b" --jobs=$(nproc --all)

But it does not fix #222 ... so something is still fishy.

[ruoti]

@kanjieater It is not the wsl integration you need to change. It is the docker engine. It has to be set to use WSL1.

[dr-BEat]

As I already mentioned in #222, it may be possible, that using a bind mount instead of a -v fixes at least some of the issues. Example

docker run -it --rm -u $(id -u):$(id -g) \
--mount type=bind,source=$FOLDER,target=/mnt
  sandreas/m4b-tool:latest merge "./$(echo $NAME)_splitted/" --output-file="./$NAME.m4b" --jobs=$(nproc --all)

But it does not fix #222 ... so something is still fishy.

I tried the bind mount just now. Sadly it did not help for me on WSL2 and Windows 11.
The workaround I found so far is to generate a custom file list using ls -1 *.mp3 and then pass that to m4b-tool instead of an input folder.

[kanjieater]

As I already mentioned in #222, it may be possible, that using a bind mount instead of a -v fixes at least some of the issues. Example

docker run -it --rm -u $(id -u):$(id -g) \
--mount type=bind,source=$FOLDER,target=/mnt
  sandreas/m4b-tool:latest merge "./$(echo $NAME)_splitted/" --output-file="./$NAME.m4b" --jobs=$(nproc --all)

But it does not fix #222 ... so something is still fishy.

I tried the bind mount just now. Sadly it did not help for me on WSL2 and Windows 11. The workaround I found so far is to generate a custom file list using ls -1 *.mp3 and then pass that to m4b-tool instead of an input folder.

Thanks for sharing this idea. Would you mind sharing what command you run to sub in the list from ls into your command? Do you write it out to a file first or is there someway to substitute that output into a m4b command?

[dr-BEat]

Sure!
So if a command like m4b-tool merge . --output-file "Cool book.m4b" -v would not detect all mp3 files in the current folder I switch to this style m4b-tool merge ./t 'Cool book Chapter 1.mp3' ... 'Cool book Chapter 10.mp3' --output-file "Cool book.m4b" -v.
The ./t in there is an empty folder I would create for m4b-tool to be happy with. Probably there is a better way to do this. :)

The conversion from ls -1 *.mp3 to the command I did in a text file since my terminal skills are not that good.

[kanjieater]

Thanks for the hint. I still haven't figured out how to get a file from ls output into the docker command just yet.
One concern with this method might be how m4b-tool sorts files loaded in this way.

Does it take the order or does it load all files then sort them by natural language sorting? Otherwise you might get merge to mix up things like I ran into recently.

[dr-BEat]

You are right, forgot about that part. I did not risk it and made sure the order in the command is correct and not just the direct ls output.
I just found ls has a natural sort mode with -v. So using ls -1v *.mp3 should give you the right order.

[kanjieater]

You are right, forgot about that part. I did not risk it and made sure the order in the command is correct and not just the direct ls output. I just found ls has a natural sort mode with -v. So using ls -1v *.mp3 should give you the right order.

I've been trying to get this to work, but how are you passing the list from ls to the merge command. Please include code snippet🙏

EDIT:

I ended up using this:

#!/bin/bash

audiobooks_path="/mnt/a/audiobooksm4b/"

# Loop over each audiobook folder
for audiobook_folder in "${audiobooks_path}"/*/
do
  # Print out the name of the audiobook folder
  echo "Merging files in ${audiobook_folder}..."

  # Get the audiobook name
  audiobook_name="$(basename "${audiobook_folder}")"

  # Get the list of mp4 files in the audiobook folder
  mp4_files=$(ls -1v "${audiobook_folder}"*.mp4)

  # Replace audiobooks_path with "./" in each mp4 file path
  mp4_files=$(echo "$mp4_files" | sed "s|${audiobook_folder}|./|g")

  # Wrap each mp4 file path in quotes
  mp4_files=$(echo "$mp4_files" | awk '{printf "\"%s\" ", $0}')

  # Run the m4b-tool Docker container to merge the MP4 files
  docker_cmd="docker run -it --rm -u $(id -u):$(id -g) -v \"${audiobook_folder}\":/mnt sandreas/m4b-tool:latest merge ${mp4_files} --output-file \"./${audiobook_name}.m4b\" --jobs $(nproc --all)"

  echo "$docker_cmd"
  eval "$docker_cmd"
done

[dr-BEat]

There is no code snippet. I just copied the output of ls into VS Code and removed the newlines so that I have all files in a single line and pasted it into the merge command.