Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Make sure that the cache_path is created for Sqlite3 files #103

Merged
merged 1 commit into from Jan 30, 2013

Conversation

Projects
None yet
2 participants
Contributor

rogerbraun commented Jan 30, 2013

When you try to use the SQLite backend and don't have the necessary paths created, Picky will crash because SQLite can't open (or create) the right file. This is because the directory is only created in #reset, but the file is opened in lazily_initialize_client. The specs ran through because the temporary directory was not cleaned, thus leaving the paths intact. Also, another spec would create the necessary path. This code implements a fix and also fixes the specs.

@floere floere added a commit that referenced this pull request Jan 30, 2013

@floere floere Merge pull request #103 from rogerbraun/sqlite-fix
Make sure that the cache_path is created for Sqlite3 files
58572bc

@floere floere merged commit 58572bc into floere:master Jan 30, 2013

Owner

floere commented Jan 31, 2013

Thanks for the pull request, Roger. The specs were indeed unclean.

A similar problem has cropped up. When running the specs, I get SQLite3::CantOpenException errors.

$ be rspec
  […]
  7) Picky::Backends::SQLite immediately indexing backend (no dump needed) handles dumping and loading
     Failure/Error: data.backend described_class.new(realtime: true)
     SQLite3::CantOpenException:
       unable to open database file
     # ./lib/picky/backends/sqlite/basic.rb:50:in `initialize'
     # ./lib/picky/backends/sqlite/basic.rb:50:in `new'
     # ./lib/picky/backends/sqlite/basic.rb:50:in `lazily_initialize_client'
     # ./lib/picky/backends/sqlite/basic.rb:19:in `initialize'
     # ./lib/picky/backends/sqlite.rb:21:in `new'
     # ./lib/picky/backends/sqlite.rb:21:in `create_inverted'
     # ./lib/picky/bundle.rb:83:in `create_backends'
     # ./lib/picky/bundle.rb:76:in `reset_backend'
     # ./lib/picky/category.rb:122:in `reset_backend'
     # (__FORWARDING__):2:in `block in reset_backend'
     # (__FORWARDING__):2:in `each'
     # (__FORWARDING__):2:in `reset_backend'
     # (__FORWARDING__):2:in `reset_backend'
     # ./lib/picky/index.rb:138:in `backend'
     # ./spec/functional/backends/sqlite_spec.rb:95:in `block (3 levels) in <top (required)>'

Do you get these as well?

If I run the specs in isolation, the Too many open files - getcwd (Errno::EMFILE) error occurs.

$ be rspec ./spec/functional/backends/sqlite_spec.rb
............FFFFFFFF

Failures:

  1) Picky::Backends::SQLite immediately indexing backend (no dump needed) searching for it using partial
/Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/metadata.rb:30:in `expand_path': Too many open files - getcwd (Errno::EMFILE)
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/metadata.rb:30:in `relative_path'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/metadata.rb:77:in `file_and_line_number'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/metadata.rb:49:in `[]'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/example.rb:41:in `block (2 levels) in delegate_to_metadata'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_formatter.rb:123:in `read_failed_line'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:180:in `dump_failure_info'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:175:in `dump_failure'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:19:in `block in dump_failures'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:17:in `each'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:17:in `each_with_index'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/formatters/base_text_formatter.rb:17:in `dump_failures'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:98:in `block in notify'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:97:in `each'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:97:in `notify'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:81:in `finish'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:36:in `report'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/command_line.rb:25:in `run'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/runner.rb:69:in `run'
    from /Users/hanke/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/rspec-core-2.11.1/lib/rspec/core/runner.rb:8:in `block in autorun'

By setting ulimit to eg. 3000, the errors do not occur.

$ ulimit -n 3000
$ be rspec ./spec/functional/backends/sqlite_spec.rb
....................

Finished in 1.02 seconds
20 examples, 0 failures

Have you had similar problems? Is the SQLite lib or a lib it uses opening too many files? (Perhaps Picky is, but I can't find a spot where that would be possible)

Contributor

rogerbraun commented Jan 31, 2013

I get no errors with the current master (except that my PC isn't fast enough... damn perfomance specs ;-)),

Owner

floere commented Jan 31, 2013

:D
Find the performance_ratio.rb file in /spec:

# Remove this file or set PerformanceRatio to 0.5
# if the performance tests fail to often before
# you have changed anything.
#
module Picky; PerformanceRatio = 1.0 end

Set the ratio to 0.5 if your computer is about half as fast as mine.

The performance_ratio.rb is currently checked in. Feel free to send me a pull request where it is in .gitignore :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment