Transmission has been downgraded to 2.94 in the latest image, issues and possible fixes.

[haugene]

Picking up the conversation from these comments: #1936 (comment) and #1936 (comment)

Background:
Upgrading the image to version 4.0 we actually downgraded Transmission from 3.00 to 2.94. The reason for this is in general because we want to be at the latest versions, over the years it has caused trouble for us. Right now - to be specific - it is this bug that caused us to make the move. transmission/transmission#1314

I was hoping this would not have big consequences, but it turns out these versions hash torrents differently which trips up the listing of torrents.

Let's talk solution:
I've thought a bit about this. And even if I was a bit unsure - we're not "rolling back" to v3.00 of Transmission. Long term I believe this is the most stable solution, and that's what we're going for now.

So... What can we do. Here are the options I've come up with:

1. Accept 4.0 as it is, clean your torrent list and take the one-time pain this is
2. Stay at version 3.7.1, downside being you will get no more updates
3. You can add a pre-start script that upgrades Transmission to v 3.00
4. We could add the pre-start script to the image and you could trigger it using an environment variable instead of mounting it yourself. (USE_TRANSMISSION_PPA=TRUE for example)

For options 3 and 4 I think this would do the trick (thanks to @Forage for below script that avoids downloading unnecessary items):

#! /bin/bash
wget -O 976b5901365c5ca1.key "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xa37da909ae70535824d82620976b5901365c5ca1"

cat > /etc/apt/sources.list.d/transmission.list <<EOF
# Transmission PPA https://launchpad.net/~transmissionbt/+archive/ubuntu/ppa
deb [signed-by=/976b5901365c5ca1.key] http://ppa.launchpad.net/transmissionbt/ppa/ubuntu focal main 
#deb-src http://ppa.launchpad.net/transmissionbt/ppa/ubuntu focal main 
EOF

apt update -o Dir::Etc::sourcelist="sources.list.d/transmission.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
apt install -y transmission-daemon transmission-cli

Put that in a script locally and mount it to /scripts/transmission-pre-start.sh. That should cause you to both run image v 4.0 and have Transmission 3.00.

Now... Why don't embed this into the image and make it easier for people to upgrade to 3.00 if they want?
The short answer is that I might be persuaded, but I'm not super eager to "officially support" two versions.

Thoughts?

[mhbates]

Thank you for this. If it's not too much to ask, can you please go into some more detail on the hashing issue and what the results of that are? I'd like to have a better understanding before making a decision on this. Are you saying that every single torrent will be affected by this, and thus every torrent needs to be downloaded again, at least in order to reach the same state as prior to the 4.0 upgrade?

I'm worried about situations like:

• Long-time seeding of torrents from private trackers that haven't hit required ratio yet; the act of re-downloading will impact ratio, etc.
• Being the only seeder of a rare torrent, so re-downloading is not possible or will take very long waiting for other seeders

I'm wondering if there's some workaround like adding the torrent anew and somehow pointing it to the data in the completed folder - forgive me if there is an obvious fix for this or if this is obviously not possible, but I'm blanking at the moment.

[tmknight]

Yes, re-adding the torrent and the location to the already downloaded content is the solution for the hash issue. You may have to "verify local data", though typically not required.

[mhbates]

In that case, I think I would lean towards option 4 (embedded script), if that becomes an option.

The re-adding method would probably not be feasible now that I think about it, at least for me with 1500+ torrents. I imagine some other people would also want to stick with this version of Transmission and simply wait until the aforementioned bug is fixed, assuming they're not seriously affected by that bug.

If option 4 does not become an option, then I'd probably stick with option 2 (stay on image 3.7.1) and just hope that no serious bugs or vulnerabilities come to light before 4.x (or whatever future version of this image) is back on Transmission v3+.

[tmknight]

You might consider creating a new container (with different ports) and "migrate" from the old to new with something like this: https://github.com/machsix/transmission-migrate

Then take down the old and reconfigure new with desired prod settings.

[QuiGonLeong]

I have about 100+ large torrents and the reverification is taking a while and chewing up CPU. I can't even imagine for folks with thousands of torrents. I've reverted back to 3.7.1 and will stay there until option 4 is available with environmental variable.

[tmknight]

IMO, stay the course developing for the version of transmission you wish to support. Now that we know why this is occurring and have options to resolve, this shouldn't be too burdensome. For my situation, I have nearly 200 seeds, adding a few lines to my dockerfile to upgrade transmission is trivial. Those who have a more manageable library, should convert them for the supported bt version and be done with it.

[eiddor]

I agree with this approach, but for the uninitiated (me), I just want to confirm this process.

To "redo" the torrents, I should:

• Copy the *.torrent files from transmission-home/torrents to my local machine
• In the GUI: "Remove" each torrent but do not check "Delete local data"
• Manually add each of the torrent files in the GUI and count on the locations being the same (I assume they should be by default)

[pkishino]

If it didn’t change locations when adding it will be in default, but yeah, after adding, you might need to re-verify if it doesn’t trigger itself

[eiddor]

Looks like that worked - Thanks @pkishino! And thank you, @haugene for the excellent image.

[Forage]

Thanks for at least addressing this issue.
That's quite a heavy script to run for each start of a container though. Installing software-properties-common just for the apt-add-repository -y ppa:transmissionbt/ppa one-liner is a bit overkill as it will download and install 16 MB for 75 additional packages, plus 19,2 MB for a complete repository update.

I suggest using the following instead:

#! /bin/bash

wget -O 976b5901365c5ca1.key "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xa37da909ae70535824d82620976b5901365c5ca1"

cat > /etc/apt/sources.list.d/transmission.list <<EOF
# Transmission PPA https://launchpad.net/~transmissionbt/+archive/ubuntu/ppa
deb [signed-by=/976b5901365c5ca1.key] http://ppa.launchpad.net/transmissionbt/ppa/ubuntu focal main 
#deb-src http://ppa.launchpad.net/transmissionbt/ppa/ubuntu focal main 
EOF

apt update -o Dir::Etc::sourcelist="sources.list.d/transmission.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
apt install -y transmission-daemon transmission-cli

[pkishino]

have updated the suggestion above, thanks

[Forage]

And to avoid a 19,2 MB complete update of all repositories it is enough to update only the PPA one by using:

apt update -o Dir::Etc::sourcelist="sources.list.d/transmission.list" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"

I've updated my initial reply accordingly.

[ksurl]

On the vpnunlimited provider, it times out trying to connect to the keyserver on 443

[joe-eklund]

So I attempted solution 1, taking the "one-time" pain and it was successful. Earlier this week I was able to remove all my torrents (and cleaned the folders), then add them back in after upgrading to 4.0.

I was previously having the issue of phantom torrents kept reappearing. It seemed to have fixed it, where torrents that removed and deleted would not come back on a container restart.

Just today I ran some updates and restarted my containers, and now it's adding previously deleted torrents back again. So in the mean time I have rolled back to 3.7.1 for stability reasons. I did try solution 3 before solution 1 and was unsuccessful with that as well.

So any ideas on why solution 1 worked for me last Tuesday and now it's not? I'd rather not have to continue doing solution 1 over and over in the future.

[matthewmcpartland]

Personally I'm also having the issue of torrents being re-added, I get the feeling the torrents in /transmission-home/torrents/ are doubled up for me. Thanks for your comment though, otherwise I wouldn't have recognized the issue was a result of a change to this docker image and not a system issue I was having.

For me, this was the straw the has pushed me to my own solution with rTorrent + Flood. I can run them in separate containers and use much more standard Docker conventions. The webUI performance in Transmission was getting too slow with 5,000 torrents and at this point I need to re-setup 5,000 torrents so I might as well do-so in the more stable and performant system.

[pkishino]

Well, I might not have triple figure torrents running but on my dev setup there were the same issues when I replicated it, after deleting transmission home and reverting the problem disappears here at least. I’ve also tested solution 3 which works just fine, just startup takes a little bit longer.
@joe-eklund why it did work before but then failed again.. did you recreate the transmission folder? Or at least the resume folder? While container was stopped? The dev container restarts very frequently for testing etc and after the cleanup it will only happen if I force upgrade, add torrents, downgrade and then delete them, as expected. If I do container, delete resume then restart no such problem

[matthewmcpartland]

Just wanted to provide an update here for anyone that this may help.

I have specified version 3.7.1 in the systemd file I was using to manage this docker. Once restarted this set Transmission back to 3.0 and everything is back to how it was (all torrents are added back with the correct paths and no need to re-verify the hashes). I did not remove the previous torrent files and resume files while I was experiencing the issues so it seems Transmission is mostly fine when going from 3.0->2.94->3.0

I'm still playing around with a new setup but am ultimately going to use something similar to this: jesec/flood#120 (comment)
And I'll run two rTorrent instances both managed by the same Flood webui, one rTorrent instance will be using a VPN container for it's network (currently using the image thrnz/docker-wireguard-pia).

[joe-eklund]

Apologies for the late response @pkishino. I killed the resume and torrents folder. The container was stopped yes.

I am attempting this again though. This time I stopped the container (like last time), but instead of killing only specific folders in the transmission-home folder, I killed the entire volume that was getting mounted. Now there should theoretically be nothing left over from previous installs, not even logs.

We shall see how it goes...I will let you know if running latest somehow finds previous torrents even though it shouldn't. :)

I restarted the containers and no previous torrents added as of yet, but the issue before didn't show up until a couple days later for some reason, so I will give it some time.

[joe-eklund]

Seems like killing the entire transmission volume has solved it for me. Was able to do an update yesterday and no more phantom torrents. :)

[balupton]

How do I enforce the container stays at v3, I am new to docker.

[balupton]

So to downgrade, you do haugene/transmission-openvpn:3.7.1 instead of haugene/transmission-openvpn wherever such is mentioned in your docker commands

[pkishino]

Or you use workaround script in the first post, this will upgrade transmission before launching it..

[pkishino]

Either use 3.7.1 image of container or read the work around script at the top

…

On Fri, Oct 22, 2021 at 19:42 Benjamin Lupton ***@***.***> wrote: How do I enforce the container stays at v3, I am new to docker. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1937 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7OFYXKZFPHBPJP43AYWCDUIE52DANCNFSM5DNQDGYA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[pduchnovsky]

Put that in a script locally and mount it to /scripts/transmission-pre-start.sh. That should cause you to both run image v 4.0 and have Transmission 3.00.

That won't work, since /scripts/transmission-pre-start.sh is executed right post vpn init but before actual connection, the DNS won't work, therefore I had to use /scripts/openvpn-pre-start.sh instead.

[reticle5]

It did use to work when the first versions of the v4 image came out. Then it stopped working after an update. For some reason my VPN provider, PIA, stopped being able to resolve the Ubuntu servers straight after startup.

pduchnovsky is right. Move the script to /scripts/openvpn-pre-start.sh and it works fine.

[Forage]

/scripts/transmission-pre-start.sh has worked for me up until now without issues.

[mhbates]

Did anyone find a fix for this? I'm encountering it now using both openvpn-post-start and transmission-pre-start.

Edit: That is, a fix that uses the VPN's DNS and not the host/container DNS.

[pkishino]

From when I check my connection log etc.. any time my transmission-pre-start.sh script runs it uses the VPN dns successfully as the connection is already established

[mchccc]

It did not for me; switching to openvpn-pre-start fixed it

also, wow! my torrents are finally resuming correctly instead of rechecking over and over! 🙌

now waiting for when this workaround inevitably comes back biting my ass 😬

[AythamiEsp]

Where can I check what image version I'm running? Still using transmission 2.94.

[pkishino]

During container start it will print the hash in the log, can compare with docker image

[epichub]

I never managed to get any of these scripts to run - openvpn-pre-start etc, so i had to execute the script manually inside of the container - however now I am rolling my own fork of the code (with ufw fix mentioned in #2255 ) so will just add the script to the dockerfile to get 3.00 :) I never had any problem with transmission/transmission#1314 - only unresponsiveness I get is if I move large files from web UI / rpc.. but still not a big issue for me.

But if I want to get back non-fork haugene main, how do I get this script to run? What is the correct way to auto-launch @Forage 's script?

[Forage]

The script works as is. You only need to save it as transmission-pre-start.sh and you add the following to your docker params:
-v /location/to/local/transmission-pre-start.sh:/scripts/transmission-pre-start.sh

[haugene]

To anyone still following this thread. Sorry about the delay, as this could have been bumped a while ago even with the LTS strategy we've been on.

Just changed the base image on the dev branch so the dev tag of the image now has Transmission v 3.00. Will give it over the weekend at least to see that there isn't a lot of issues for anyone (OpenVPN is also updated). But you should be able to use the dev tag now, and very soon the latest tag and it should not break when updating from image 3.7.1.

[DuvelCorp]

My morning story.

Got my container running for like one year without issue.
Today I am telling myself it might me time to upgrade it.
I check the readme, see nothing obvious, and so I do a docker-compose pull, and up
And then I realize that I have lost all torrents (and stats) in transmission.
Since I had >1000 torrents on private trackers, I was close to the heart attack.
After having reviewed all my parameters and config files and scrutinized logs during 15-20 minutes , I was absolutely panicked to not find any explanations.
Then I started to search in the project #issues and read a lot of things for 15 more minutes but nothing like my exact problem.
Then I started googling (and at this point i was desperate) and found this discussion after 15 minutes of reading garbage.

Reverted my container image to 3.7.1, and everything is back fine.

So why do I post this here? Because this issue/update/discussion whatever is seriously lacking visibility. I think there should be a note about this in the main page of the project. Bceuase if an experienced sysadmin like I am had to take 1 hour to find an explanation and a fix, it is probable that a complete noob will take 3 days, or just give up.

Also If something as simple as docker-compose pull breaks everything for an existing and long-time user, there's a solid problem. Got a hundred of apps to administrate over a dozen of servers, and I cant and wont read everything everywhere and beyond, before simply pressing the "upgrade" button. Sounds common sense.

Cheers

[ksurl]

On a side note, the /config volume is an excellent change. I didn't like at all to have the app config in the data folder and not being able to move it elsewhere simply.

my workaround before this was added was to set it via the environment variable for transmission home and mount it anyways. downloads still go in data

[ilike2burnthing]

As @ksurl said, it looks like the readme was overlooked in the update. Opened a new PR - #2451

[haugene]

If reverting to 3.71 was "a temporary fix", what does it makes not temporary anymore nowadays?

I removed the note in the README because of what @ilike2burnthing said. The bug was assumed to be related to the unfortunate downgrade from transmission 3 to 2.94. In hindsight I might have done that differently if I'd known the problems it would cause. But we had changed the base image from ubuntu to alpine some years before and always being on the latest and greatest of versions caused a lot of bugs as well and it was a long term move for stability that had bigger consequences than expected. Anyways, you figured that out already.

About the breaking stuff you see now, I would have noted it if I had known about it, and more than that I had probably not gone through wit it either. Haven't gotten to it yet but it seems I have to dig some more into why this transmission-home change is breaking the setup. I thought that we actually made it backwards compatible by looking for the previous location and using it if it exists.

So at least it's not a clear violation of common sense but a bug that was introduced unknowingly. I actually tried (I think) to run a container on the old version and then do an update. Not sure why I didn't catch it before merging and releasing it. Happy for any insight into why it happens, and I'll have a look when I have time.

[haugene]

Did some testing and found an error, but I can't make it break in the ways you're describing.
Following up in #2409

Would be nice if you can join there and help out identifying the bug 🙏

[haugene]

FYI: I believe I've found the bug. A fix should be available on the :dev tag of the image. For more info see my last comment in #2409

[mhbates]

To anyone else finally upgrading from 3.7.1 to 4.3, I feel like I should notify that I ran into stability issues with the 4.3 image, seemingly unrelated to the known changes like directory structure changes (and note Transmission itself seems to function normally, no missing torrents, etc.). I am still troubleshooting to see the exact cause or solution, hence why I haven't yet opened an issue about this. It could be that the issue is not 4.3 at all, but for now, it is looking like it is.

In short, what happens with my container is that, over time, Transmission will continue to function (download & upload), but the container will lock up on the Docker side and stop responding - for example, I cannot stop the container, I cannot look at Docker logs, etc. A side effect of this is that any scripts that I run against the container also stall out and accumulate in the background, resulting in massive memory usage leak over time. The workaround is to stop all other Docker containers (which continue to be responsive), and then simply restart the host machine, and it is fine until the Transmission container goes unresponsive again at some point and the memory leak accumulates.

Yesterday, I rolled back to 3.7.1 and will see if this issue continues to happen with that.

[haugene]

@mhbates Just out of curiosity and to see if this can also be a side affect of the ubuntu:22.04 issues we've seen. Can you try to upgrade your Docker installation, or even the workaround described here #2410 (comment)? I don't have much reason to believe it's related but might be worth a shot.

And if it doesn't work I think a new issue is the way to go, as mentioned here. The more logs and run setup you can share, the better. Makes it easier to reproduce and debug 👍

[mhbates]

@haugene I checked your linked comment - I am already on Docker 20.10.21. Host OS is Ubuntu 20.04. I have since rolled back to 3.7.1 and it is stable again. I will stay on this version for a little while and try upgrading again later, and if it's still acting up, I'll submit a new issue.

[DuvelCorp]

Thats what I am doing as well, staying on 3.7.1 until clarification or fix. I am very happy with it anyway, it works perfectly.
Thanks @haugene to look it to it

[ilike2burnthing]

@DuvelCorp did you try 4.3.X with the correct volumes?

[haugene]

Memory leak also reported in discussions here and issue created #2469

I thought staying on LTS versions of Ubuntu would do wonders for the stability, but apparently not 🙈

[mchccc]

So what's the status on this, and/or what's the preferred current workaround?

I think I'm already in this thread, anyway I'm using a pre-run script to pin the Transmission version down, but sometimes I'm also running into issues when my network is down or CPU load is high or whatever, resulting in connection errors to launchpad servers and therefore the container starting with the newer version, then a bunch of older torrent starting to download again in the default folder, which is really inconvenient.

Thanks!