More tests, add `filename` to Torrent, fast Database#include? method #14

winterthediplomat · 2015-10-17T14:34:05Z

I'm sorry I did not make separated Pull Requests, but I got carried out and pushed too much.

more tests
This PR adds some tests (and travis-ci support!) to yamazaki libraries.
add filename to Torrent class
This modification is needed for some modifications in Miyuki, but it may be useful for several clients who just want to know where are the downloaded torrents.
fast Database#include? method
Track databases will only grow larger and larger, even though the rate is low. It should be noted that clients will perform more searches than insertions, so a faster #include is key to a faster Yamazaki. Yes, 56x faster.
Unfortunately, the modification requires Array#bsearch, which is available only on ruby 2.x. Let me know if you think it has to be compatible with 1.9.x rubies (still shipped by LTS linux distros), we can get around it.

RoxasShadow · 2015-10-17T14:41:17Z

I suggest so squash something if you can and eventually fix ruby1.9.3 does not have Array#bsearch that's not actually written in the git-way.

RoxasShadow · 2015-10-17T14:41:32Z

.travis.yml

@@ -0,0 +1,5 @@
+nguage: ruby


is that a typo?

RoxasShadow · 2015-10-17T14:42:31Z

.travis.yml

+nguage: ruby
+rvm:
+  - 2.0.0
+  - 2.1.2


Aren't version like 2.1.0 and 2.2.0 available? Just to be more standard, you know.

they should be available. will check if something like - latest is available.

Rox, is the next line in this diff ok for you? >2.2.1

RoxasShadow · 2015-10-17T14:44:35Z

lib/yamazaki/database.rb

-			@db.count { |t| t[:filename] == filename } > 0
+			begin
+				@db.bsearch { |t| t[:filename] >= filename }[:filename] == filename
+			rescue


If i recall correctly, Array#bsearch doesn't raise exception. Am I wrong or am I missing something?

if Array#bsearch can't find an item, nil will be returned. NilClass does not have the [] operator.
a different way to do it may be

res = @db.bsearch { |t| t[:filename] >= filename } if res res[:filename] == filename else false end

(update: grammar)

Oh, I got it. Well, let's keep that as it is.

RoxasShadow · 2015-10-17T14:44:57Z

lib/yamazaki/torrent.rb

 			@title       = title
 			@description = description
 			@pub_date    = pub_date
 			@link        = link
 			@index       = index
+                        @filename    = filename


Check the alignment here.

RoxasShadow · 2015-10-17T14:45:27Z

lib/yamazaki/torrent.rb

@@ -15,13 +15,16 @@
 module Yamazaki
 	class Torrent
 		attr_reader :title, :description, :pub_date, :link, :index
+		attr_accessor :filename
+		def initialize(title, description, pub_date, link, index = 0, filename="")


Please leave a blank line between 18 and 19.

RoxasShadow · 2015-10-17T14:46:01Z

specs/database_spec.rb

+  describe '#new' do
+    context 'when track file does not exist' do
+      it 'is created correctly' do
+        File.rm "/tmp/yam.db" if File.exists? "/tmp/yam.db"


I suggest to use the current path to avoid platform compatibility issues.

will do. are you really sure someone will use yamazaki on windows?

Absolutely nope, but why losing the support for a platform just for a path? :D

RoxasShadow · 2015-10-17T14:46:29Z

specs/database_spec.rb

+      it 'is loaded correctly if `track_file` is empty' do
+        track_file = prepare_db([])
+        db = Yamazaki::Database.new(track_file)
+        expect(db.size).to eq(0) 


expect(db).to be_empty should work too.

RoxasShadow · 2015-10-17T14:46:54Z

specs/spec_helper.rb

@@ -1 +1,28 @@
 require 'yamazaki'
+require 'date'
+require 'base64'
+#require 'random'


We can remove this require if we don't need it.

RoxasShadow · 2015-10-17T14:47:15Z

specs/spec_helper.rb

+
+def prepare_db(torrents)
+  db_file_path = new_dbfile_path()
+  if torrents.size > 0


winterthediplomat · 2015-10-17T15:28:21Z

specs/database_spec.rb

+
+
+      it 'does not insert the same filename if already present' do
+        skip 'skip: if needed, a file can be re-downloaded, and we want it to be possible'


I'd like to discuss this, though. Should we remove the test about duplicates filenames? If needed, we may transform it to check the opposite (filenames not considered unique)

I think filenames should be unique. Why we would suppose the contrary?

RoxasShadow · 2015-10-17T15:47:20Z

For me now we're ok 👍 @hydride0?

winterthediplomat · 2015-10-17T15:51:43Z

Please wait until a patch for this issue is added (test has to be un-skipped and code modified to guarantee uniqueness)

winterthediplomat · 2015-10-17T16:34:59Z

Ok, fixed the filename non-uniqueness. Ok for me too.

RoxasShadow · 2015-10-17T16:40:51Z

Good, can you also squash something too? There a lots of commits there. Then let's wait for @hydride0.

Torrent.filename: it'll contain the path a torrent was downloaded to. Add tests: Yamazaki now has more tests, covering the usual behaviour of the library. A travis.yml has been added too. `Database#size` returns the number of saved items inside the database.

…database also refactor spec_helper a bit

Switching to Array#bsearch (with database sorting) makes yamazaki a lot faster (25-56x)

ruby1.9.3 is removed because `Array#bsearch` is not supported from <2.x ruby releases Roxas noticed a lot of imperfections and little problems, they're fixed now.

Filenames should be considered unique in the database, so duplicates are not allowed. If duplicates are present in the source file, they won't be deleted after the JSON load (and it would be very, very time consuming to do this, unless we switch to hashtables).

Fake database files were created under `/tmp`, and it may break compatibility with non-posix platform. Moreover they were not deleted, leaving a lot of random files around: now they are.

winterthediplomat · 2015-10-17T20:25:07Z

squashed and forced things, is it ok for you @RoxasShadow ?

RoxasShadow · 2015-10-17T20:56:13Z

👍

More tests, add `filename` to Torrent, fast Database#include? method

winterthediplomat · 2015-10-19T19:12:51Z

\o/

add some pointers on how to check available commands

7b6a839

RoxasShadow reviewed Oct 17, 2015
View changes

.travis.yml

@@ -0,0 +1,5 @@

nguage: ruby

This comment has been minimized.

Sign in to view

RoxasShadow Oct 17, 2015
Contributor

is that a typo?

This comment has been minimized.

Sign in to view

winterthediplomat Oct 17, 2015
Author Contributor

yes.

RoxasShadow reviewed Oct 17, 2015
View changes

lib/yamazaki/torrent.rb

@title = title

@description = description

@pub_date = pub_date

@link = link

@index = index

@filename = filename

This comment has been minimized.

Sign in to view

RoxasShadow Oct 17, 2015
Contributor

Check the alignment here.

RoxasShadow reviewed Oct 17, 2015
View changes

specs/spec_helper.rb

@@ -1 +1,28 @@

require 'yamazaki'

require 'date'

require 'base64'

#require 'random'

This comment has been minimized.

Sign in to view

RoxasShadow Oct 17, 2015
Contributor

We can remove this require if we don't need it.

RoxasShadow reviewed Oct 17, 2015
View changes

specs/spec_helper.rb

def prepare_db(torrents)

db_file_path = new_dbfile_path()

if torrents.size > 0

This comment has been minimized.

Sign in to view

RoxasShadow Oct 17, 2015
Contributor

any?

winterthediplomat reviewed Oct 17, 2015
View changes

winterthediplomat added 7 commits Oct 13, 2015

Revert the version number change

162562e

add test about negative index and skip test about unique-filename-in-…

901649e

…database also refactor spec_helper a bit

Fasten Database#include?

3dd8b5e

Switching to Array#bsearch (with database sorting) makes yamazaki a lot faster (25-56x)

Remove ruby 1.9.3 from travis, fix Roxas suggestions

d048935

ruby1.9.3 is removed because `Array#bsearch` is not supported from <2.x ruby releases Roxas noticed a lot of imperfections and little problems, they're fixed now.

Improve fake database creation and removal

686ec74

Fake database files were created under `/tmp`, and it may break compatibility with non-posix platform. Moreover they were not deleted, leaving a lot of random files around: now they are.

winterthediplomat force-pushed the winterthediplomat:master branch from 669ef4f to 686ec74 Oct 17, 2015

hydride0 added a commit that referenced this pull request Oct 19, 2015

Merge pull request #14 from alfateam123/master

3b84f9f

More tests, add `filename` to Torrent, fast Database#include? method

hydride0 merged commit 3b84f9f into hydride0:master Oct 19, 2015

hydride0 / yamazaki

More tests, add `filename` to Torrent, fast Database#include? method #14

More tests, add `filename` to Torrent, fast Database#include? method #14

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 19, 2015



		it 'does not insert the same filename if already present' do
		skip 'skip: if needed, a file can be re-downloaded, and we want it to be possible'

hydride0 / yamazaki

More tests, add filename to Torrent, fast Database#include? method #14

More tests, add filename to Torrent, fast Database#include? method #14

Conversation

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

This comment has been minimized.

winterthediplomat Oct 17, 2015 Author Contributor

This comment has been minimized.

RoxasShadow Oct 17, 2015 Contributor

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 17, 2015

RoxasShadow commented Oct 17, 2015

winterthediplomat commented Oct 19, 2015

More tests, add `filename` to Torrent, fast Database#include? method #14

More tests, add `filename` to Torrent, fast Database#include? method #14

RoxasShadow Oct 17, 2015
Contributor

winterthediplomat Oct 17, 2015
Author Contributor

RoxasShadow Oct 17, 2015
Contributor

winterthediplomat Oct 17, 2015
Author Contributor

winterthediplomat Oct 17, 2015
Author Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

winterthediplomat Oct 17, 2015
Author Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

winterthediplomat Oct 17, 2015
Author Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

RoxasShadow Oct 17, 2015
Contributor

winterthediplomat Oct 17, 2015
Author Contributor

RoxasShadow Oct 17, 2015
Contributor