Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 102 lines (82 sloc) 2.426 kB
dc95141 @floere + Namespacing in tests
authored
1 module Picky
fac93d2 @floere + backend cleanup, + to_s
authored
2
536a852 @floere + backend rewrite, towards making it more exchangeable
authored
3 module Backends
fac93d2 @floere + backend cleanup, + to_s
authored
4
794aa49 @floere + experimental file-based index backend prototype
authored
5 class File
ac02b19 @floere + backend refactored, - duplicate code
authored
6
7 def json *args
8 JSON.new *args
9 end
fac93d2 @floere + backend cleanup, + to_s
authored
10
794aa49 @floere + experimental file-based index backend prototype
authored
11 # File-based index files dumped in the JSON format.
ee888e5 @floere + internal rewrite, part 4
authored
12 #
dc95141 @floere + Namespacing in tests
authored
13 class JSON < Basic
14
d199aab @floere + rewritten performance specs
authored
15 attr_writer :mapping
16
d60c667 @floere + notes on the file backend
authored
17 # The in-memory mapping hash, mapping
18 # a Symbol key to [length, offset] of
19 # the JSON data in the file.
20 #
d199aab @floere + rewritten performance specs
authored
21 def mapping
22 @mapping || raise("The File index/category needs to be loaded first.")
23 end
794aa49 @floere + experimental file-based index backend prototype
authored
24
d60c667 @floere + notes on the file backend
authored
25 # See lib/picky/backends/file.rb for what this should return.
794aa49 @floere + experimental file-based index backend prototype
authored
26 #
27 # 1. Gets the length and offset for the key.
28 # 2. Extracts and decodes the object from the file.
29 #
30 def [] key
31 length, offset = mapping[key]
f466fb7 @floere + naive implementation of file based backend, + minimal backend inter…
authored
32 return unless length
b6b58bd @floere + Yajl -> MultiJson
authored
33 result = MultiJson.decode IO.read(cache_path, length, offset)
794aa49 @floere + experimental file-based index backend prototype
authored
34 result
35 end
36
37 # Clears the currently loaded index.
38 #
d60c667 @floere + notes on the file backend
authored
39 # Note: This only clears the in-memory mapping,
40 # but this is enough for the index to not exist
41 # anymore, at least to the application.
42 #
794aa49 @floere + experimental file-based index backend prototype
authored
43 def clear
44 self.mapping.clear
45 end
46
3a232d9 @floere ! analyzer, + phonetics, + category doc
authored
47 # Size of the index is equal to the mapping size.
48 #
49 def size
50 self.mapping.size
51 end
52
794aa49 @floere + experimental file-based index backend prototype
authored
53 # Loads the mapping hash from json format.
dc95141 @floere + Namespacing in tests
authored
54 #
6174246 @floere Change backend interface to add parameter sybmol_keys to load.
authored
55 def load symbol_keys
56 self.mapping = mapping_file.load symbol_keys
794aa49 @floere + experimental file-based index backend prototype
authored
57 self
dc95141 @floere + Namespacing in tests
authored
58 end
794aa49 @floere + experimental file-based index backend prototype
authored
59
dc95141 @floere + Namespacing in tests
authored
60 # Dumps the index hash in json format.
61 #
794aa49 @floere + experimental file-based index backend prototype
authored
62 # 1. Dump actual data.
63 # 2. Dumps mapping key => [length, offset].
64 #
dc95141 @floere + Namespacing in tests
authored
65 def dump hash
794aa49 @floere + experimental file-based index backend prototype
authored
66 offset = 0
10d738e @floere Also use Google Hashes on partial indexes if :no_dump optimisation is…
authored
67 mapping = Hash.new
a5015c2 @floere + Rewrote file backend to use the memory backend with the weights index
authored
68
c6d3d26 @floere + dumping indexes during runtime is possible now
authored
69 create_directory cache_path
794aa49 @floere + experimental file-based index backend prototype
authored
70 ::File.open(cache_path, 'w:utf-8') do |out_file|
71 hash.each do |(key, object)|
b6b58bd @floere + Yajl -> MultiJson
authored
72 encoded = MultiJson.encode object
794aa49 @floere + experimental file-based index backend prototype
authored
73 length = encoded.size
74 mapping[key] = [length, offset]
75 offset += length
76 out_file.write encoded
77 end
78 end
79
80 mapping_file.dump mapping
dc95141 @floere + Namespacing in tests
authored
81 end
794aa49 @floere + experimental file-based index backend prototype
authored
82
dc95141 @floere + Namespacing in tests
authored
83 # A json file does not provide retrieve functionality.
84 #
85 def retrieve
86 raise "Can't retrieve from JSON file. Use text file."
87 end
88
f466fb7 @floere + naive implementation of file based backend, + minimal backend inter…
authored
89 # Uses the extension "json".
90 #
91 def extension
92 :json
93 end
94
2116d5f @floere + Internals restructuring
authored
95 end
fac93d2 @floere + backend cleanup, + to_s
authored
96
2116d5f @floere + Internals restructuring
authored
97 end
fac93d2 @floere + backend cleanup, + to_s
authored
98
2116d5f @floere + Internals restructuring
authored
99 end
ee888e5 @floere + internal rewrite, part 4
authored
100
fb90607 @rogerbraun First steps towards SQLite backend
rogerbraun authored
101 end
Something went wrong with that request. Please try again.