APIs adding torrents not responding while others work well

[KurumiSerori]

APIs adding torrents (including magnet or with torrent file) not responding while other apis (like get list) works well. The binary is built with go with latest src.

Here's my auth config.

[Unit]
Description=Cloud torrent download manager
After=network.target

[Service]
Type=simple
User=torrent
WorkingDirectory=~
Environment=AUTH=_user:password_
Environment=PORT=3378
ExecStart=/usr/local/bin/cloud-torrent -c /usr/local/cloudtorrent/cloud-torrent.json --host 0.0.0.0 --disable-log-time
Restart=always
RestartPreventExitStatus=42
RestartSec=3

#adjust file limit if using non-root user.
#LimitNOFILE=50000

[Install]
WantedBy=multi-user.target

Here's my cloud-torrent.json

{
  "alwaysaddtrackers": true,
  "autostart": true,
  "donecmd": "",
  "donecmdthreshold": "3s",
  "downloaddirectory": "/usr/local/cloudtorrent/downloads",
  "enableupload": false,
  "incomingport": 50007,
  "obfspreferred": true,
  "obfsrequirepreferred": false,
  "proxyurl": "",
  "scraperurl": "https://raw.githubusercontent.com/boypt/simple-torrent/master/scraper-config.json",
  "seedratio": 0,
  "trackerlisturl": "https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_best.txt",
  "watchdirectory": "/usr/local/cloudtorrent/torrents"
}

I'm calling api using curl
curl "http://localhost:3378/api/magnet" -u _user:password_ --data "_magnet-link_" -m 60

and here's the response

curl: (28) Operation timed out after 60001 milliseconds with 0 out of -1 bytes received

I also setup Nginx to access the UI, no need to say these APIs are returning 504 Gateway TImeout (I have configured Cloudflare and Cf is suggesting it cannot connect to the instance upon target API), but everything else were good, I can see the torrent list added before (with 3 tasks) and left disk space.

Any help is appreciated.

[boypt]

that's strange.

curl -v --data "magnet:?xt=urn:btih:de140d7ad0865b1c29e3fe250942f8b28477e1cd&dn=ubuntu-20.04.1-legacy-server-amd64.iso&tr=https%3A%2F%2Ftorrent.ubuntu.com%2Fannounce" -u user:pass  http://localhost:3000/api/magnet

and I'm getting OK.

A possible place is that , if you reconfigured there are several seconds while the engine restarts, meanwhile API calls are locked. An other issue.

In the 1.2.19 release I added logs before the calls to adding tasks, try and if the issue still present, paste the log as well.

[KurumiSerori]

Here's systemctl log, only [NewMagnet] appeared.

Apr 19 13:42:52 [hostname] cloud-torrent[14216]: [NewMagnet] called: magnet:?xt=urn:btih:de140d7ad0865b1c29e3fe250942f8b28477e1cd&dn=ubuntu-20.04.1-legacy-server-amd64.iso&tr=https%3A%2F%2Ftorrent.ubuntu.com%2Fannounce

I tried some other interfaces, it seems only GET /api/torrents and /api/torrent/[link] is acting good while other POST and GET APIs are not working.

[boypt]

Looks like some how Lock() is called and the program got stuck some where.

I added log to every Lock calls, it needs to run at your environment to pin down the spot without releasing the lock.

try to complie the debuglock branch

git clone -b debuglock https://github.com/boypt/simple-torrent.git
cd simple-torrent
./scripts/make_release.sh 

and you will get the executable file cloud-torrent_linux_amd64, run it.

[KurumiSerori]

Here's the log since last restart until adding new magnet. [magnet6] contains the link of ubuntu-20.04.1-legacy-server-amd64.iso

Apr 20 15:18:26 [hostname] systemd[1]: Stopping Cloud torrent download manager...
Apr 20 15:18:26 [hostname] systemd[1]: Stopped Cloud torrent download manager.
Apr 20 15:18:26 [hostname] systemd[1]: Started Cloud torrent download manager.
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: ############# SimpleTorrent ver[1.2.19-1-g4b3fdc8] #############
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [config] selected config file: /usr/local/cloudtorrent/cloud-torrent.json
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/Config
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] engine/Config
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Torrent Watcher: watching torrent file in /usr/local/cloudtorrent/torrents
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Enabled HTTP authentication
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Listening at http://0.0.0.0:3378
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: fetchSearchConfig: loading search config from https://raw.githubusercontent.com/boypt/simple-torrent/master/scraper-config.json
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [NewTorrentBySpec] called
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] bg/init
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] bg/init
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] rss/sort
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] rss/sort
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: UpdateTrackers: loading trackers from https://raw.githubusercontent.com/ngosang/trackerslist/master/trackers_best.txt
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Inital Task Restored: /usr/local/cloudtorrent/torrents/_CLDAUTOSAVED_[torrent1].torrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [NewTorrentBySpec] called
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Inital Task Restored: /usr/local/cloudtorrent/torrents/_CLDAUTOSAVED_[torrent2].torrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [NewTorrentBySpec] called
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: StartTorrent [torrent1]
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/getTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/getTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: UpdateTrackers: loaded 20 trackers
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/NewTorrentBySpec
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: StartTorrent [torrent2]
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/getTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/getTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [newTorrent] added 20 public trackers
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [LOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Inital Task Restored: /usr/local/cloudtorrent/torrents/_CLDAUTOSAVED_[torrent3].torrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [NewMagnet] called:
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/NewMagnet
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Inital Task: fail to add _CLDAUTOSAVED_[torrent4].info, ERR:&errors.errorString{s:"unexpected scheme \"\""}
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [NewMagnet] called: magnet:?xt=urn:btih:[magnet5]
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/NewMagnet
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/NewMagnet
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [newTorrent] added 20 public trackers
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [RLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: [UNRLOCKED] engine/upsertTorrent
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: StartTorrent [torrent3]
Apr 20 15:18:26 [hostname] cloud-torrent[30508]: Loaded new search providers
Apr 20 15:18:58 [hostname] cloud-torrent[30508]: [NewMagnet] called: magnet:?xt=urn:btih:[magnet6]
Apr 20 15:19:14 [hostname] cloud-torrent[30508]: 2021-04-20T15:19:14+0000 NONE client.go:371: dht server on [::]:50007 completed bootstrap (dht.TraversalStats{NumAddrsTried:535, NumResponses:153})

An error related to parsing saved task?

[boypt]

OK I reimplemented your situation.

A malformed magnet link cause the function returned without releasing the lock.

I'll release a bug fix.