Allow Creation of Custom Search Engines for Private Trackers

[SchizoDuckie]

Make it as simple as possible to define a scraper that fetches relevant info

[SchizoDuckie]

todos / braindump:

• allow remote torrents to be fetched and uploaded as file upload to the torrent client (if it supports that)
  (because torrents can be behind cookie/login and then sent to a remote)

  var xhr = new XMLHttpRequest();
                xhr.open("POST", http+localStorage["host"]+":"+localStorage["port"]+relpath+"?token="+token+"&action=add-file", true, localStorage["login"], localStorage["password"]);
                xhr.onreadystatechange = ut_handleResponse;
                // mostly stolen from https://github.com/igstan/ajax-file-upload/blob/master/complex/uploader.js
                var boundary = "AJAX-----------------------"+(new Date).getTime();
                xhr.setRequestHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
                var message = "--" + boundary + "\r\n";
                   message += "Content-Disposition: form-data; name=\"torrent_file\"; filename=\"file.torrent\"\r\n";
                   message += "Content-Type: application/x-bittorrent\r\n\r\n";
                   message += torrentdata + "\r\n";
                   message += "--" + boundary + "--\r\n";

                xhr.sendAsBinary(message);

[SchizoDuckie]

I just discovered

https://github.com/zone117x/Jackett/

zone117x/Jackett@abdca8c

Note to self: contact these guys when the time comes, see if we can share a common repo with tracker info

[edit] moved to https://github.com/Jackett/Jackett.
Also note: jacket supports only private trackers.

[SchizoDuckie]

happy to report: This latest commit basically makes the custom search engines work!

Todos:

• support functions (such as those in 1337x, ExtraTorrent, Idope, ThePrateBay, Zooqle and IsoHunt)
• support %o (in endpoints) and orderby, as introduced by option on torrent dialogue to change sort order #668
• Permanently store the custom searchengine data as a database fixture
  • add 1337x
  • add kat
  • add KATcr
  • add Idope
  • add tpb
  • add nyaa
  • add torrentz torrentZ2
  • add RARbg (note: this is an API not a scraper)
  • add strike IsoHunt
  • add showrss
  • add torrentleech
  • add ExtraTorrent SkyTorrents
  • add Zooqle
  • add LineTorrents
  • add EzTV.ag
  • torrentDownloads
• Provide a way to export these engines
• Autoload enabled engines on page load
• Figure out a way around the ugly confirm dialog. Should notify the user he should login, but not lock the UI
• Allow the user to edit all the existing genericsearch implementations, show these by default in the customsearch panel
• make enable/disable buttons work
• Figure out an easy way to export/import/share custom search engines

[garfield69]

Looking to understand fixtures...

• Are fixtures basically pre-loaded DB entities in a table?
• They get loaded into the table during the table creation phase?
• How does making subsequent changes to the fixtures definitions get propagated to the existing DB entities previously created (like the nyaa change you made later to the already created nyaa fixures in the prior commit) ?

[SchizoDuckie]

Are fixtures basically pre-loaded DB entities in a table?

Yes.

They get loaded into the table during the table creation phase?

Correct :)

How does making subsequent changes to the fixtures definitions get propagated to the existing DB entities previously created (like the nyaa change you made later to the already created nyaa fixures in the prior commit) ?

They don't currently :) While developing this, i'm dropping the database table using

CRUD.executeQuery('DROP table SearchEngine') in the console

[garfield69]

replaced by Jackett #899

[ManoharOfficial]

So, a fully private tracker like td.af would be able to fully integrate with the chrome extension V.1.1.5?

[garfield69]

Yes, Jackett has Torrent Day on it's list of supported sites.

[ManoharOfficial]

awesome! can't wait for the stable 1.1.5 release!

[garfield69]

You could try it out with the DuckieTV development nightly https://github.com/SchizoDuckie/DuckieTV/wiki/How-to-Install-a-Development-version-of-the-DuckieTV-Chrome-Browser-Extension-NIGHTLY
[edit] oh, you may find this useful https://github.com/SchizoDuckie/DuckieTV/wiki/Using-Jackett-with-DuckieTV

[ManoharOfficial]

Awesome! Just got everything setup. Can't wait for the stable release!
[edit] You guys are very through with your guides. I'm surprised that you don't accept donations.
Question : Is DuckieTV out of passion or, as an open source collaboration between various enthusiasts?