Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Comparing changes

Choose two branches to see what's changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
  • 8 commits
  • 12 files changed
  • 0 commit comments
  • 1 contributor
View
2  .gitignore
@@ -1,4 +1,4 @@
-env26/
+env*/
*.egg
*.egg-info
*.pyc
View
12 CHANGES.txt
@@ -1,3 +1,15 @@
+Next release
+============
+
+- Add ``repoze.catalog.indexes.intfield.CatalogIntFieldIndex`` index type.
+ This is an index that operates just like a FieldIndex but is optimized
+ for integer values.
+
+- Add a ``__ne__`` to the _Range comparator to make ``!=`` comparisons work
+ properly.
+
+- Add a suitable tox.ini.
+
0.8.1 (2011-08-17)
==================
View
12 docs/api.rst
@@ -83,6 +83,18 @@ Other Helpers
.. autoclass:: CatalogFieldIndex
:members:
+.. _api_intfieldindex_section:
+
+:mod:`repoze.catalog.indexes.intfield`
+--------------------------------------
+
+.. automodule:: repoze.catalog.indexes.intfield
+
+ .. autoclass:: CatalogIntFieldIndex
+ :members:
+
+ .. autofunction:: convert_field_to_intfield
+
.. _api_keywordindex_section:
:mod:`repoze.catalog.indexes.keyword`
View
12 docs/glossary.rst
@@ -10,48 +10,60 @@ Glossary
`Setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_
builds on Python's ``distutils`` to provide easier building,
distribution, and installation of packages.
+
Interface
An attribute of a model object that determines its type. It is an
instance of a ``zope.interface`` Interface class.
+
Zope
`The Z Object Publishing Framework <http://zope.org>`_. The granddaddy
of Python web frameworks.
+
ZODB
`The Zope Object Database <http://wiki.zope.org/ZODB/FrontPage>`_
which is a persistent object store for Python.
+
Field index
A type of index that is optimized to index single simple tokenized
values. When a field index is searched, it can be searched for
one or more values, and it will return a result set that includes
these values exacty.
+
Text index
A type of index which indexes a value in such a way that parts of
it can be searched in a non-exact manner. When a text index is
searched, it returns results for values that match based on
various properties of the text indexed, such as omitting
"stopwords" the text might have.
+
Facet index
A type of index which can be used for faceted search.
+
Path index
A type of index that keeps track of documents within a graph;
documents can be searched for by their position in the graph.
+
zope.index
The `underlying indexing machinery
<http://pypi.python.org/pypi/zope.index>`_ that
:mod:`repoze.catalog` uses.
+
zope.app.catalog
The `cataloging implementation
<http://pypi.python.org/pypi/zope.app.catalog>`_ on which
:mod:`repoze.catalog` is based (although it doesn't use any of
its code).
+
Virtualenv
An isolated Python environment. Allows you to control which
packages are used on a particular project by cloning your main
Python. `virtualenv <http://pypi.python.org/pypi/virtualenv>`_
was created by Ian Bicking.
+
CQE
A string representing a Python-like domain-specific-language
expression which is used to generate a query object.
+
Query Object
An object used as an argument to the :meth:`repoze.catalog.Catalog.query`
method's ``queryobject`` parameter.
View
7 docs/overview.rst
@@ -119,9 +119,10 @@ Index Types
Out of the box, ``repoze.catalog`` supports five index types: field indexes,
keyword indexes, text indexes, facet indexes, and path indexes. Field indexes
are meant to index single discrete values. Keys are stored in order, allowing
-for the full suite of range and comparison operators to be used. Keyword
-indexes index sequences of values which can be queried for any of the values
-in each sequence indexed. Text indexes index text using the
+for the full suite of range and comparison operators to be used. IntField
+indexes are like FieldIndexes but are optimized for storing only integers.
+Keyword indexes index sequences of values which can be queried for any of the
+values in each sequence indexed. Text indexes index text using the
``zope.index.text`` index type, and can be queried with arbitrary textual
terms. Text indexes can use various splitting and normalizing strategies to
collapse indexed texts for better querying. Facet indexes are much like
View
28 docs/usage.rst
@@ -107,9 +107,11 @@ This item is the document id for the content we indexed. Your
application is responsible for resolving this document identifier back
to its constituent content.
-.. warning:: The result set is only guaranteed to be an iterable. It
- will always be of a particular type, and *not* always sliceable;
- for example it may be a generator.
+.. warning::
+
+ The result set is only guaranteed to be an iterable. It will always be of
+ a particular type, and *not* always sliceable; for example it may be a
+ generator.
You can also combine query objects, using boolean operations, to search
multiple indexes:
@@ -199,7 +201,9 @@ The above string is a CQE. A "CQE" is a string representing a Python
expression which uses index names and values. It is parsed by the
catalog to create a query object.
-.. warning:: CQE strings are not supported on Python versions < 2.6.
+.. warning::
+
+ CQE strings are not supported on Python versions < 2.6.
Whether a query object is used directly or query objects are generated
as the result of a CQE, an individual query object will be one of two
@@ -228,9 +232,11 @@ as follows:
query = Eq('author', 'crossi') & Contains('body', 'biscuits')
-.. note:: Although it would be more intuitive to use the boolean operators,
- ``or`` and ``and`` for this rather than bitwise operators, Python does not
- allow overloading boolean operators.
+.. note::
+
+ Although it would be more intuitive to use the boolean operators, ``or``
+ and ``and`` for this rather than bitwise operators, Python does not allow
+ overloading boolean operators.
Query objects may also be created by parsing a :term:`CQE` string.
The query parser uses Python's internal code parser to parse CQE query
@@ -498,9 +504,11 @@ CQE::
Search Using the :meth:`repoze.catalog.Catalog.search` Method (Deprecated)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-.. warning:: The :meth:`repoze.catalog.Catalog.search` method is
- deprecated as of :mod:`repoze.catalog` 0.8. Use
- :meth:`repoze.catalog.Catalog.query` instead.
+.. warning::
+
+ The :meth:`repoze.catalog.Catalog.search` method is deprecated as of
+ :mod:`repoze.catalog` 0.8. Use :meth:`repoze.catalog.Catalog.query`
+ instead.
We can pass a query into our catalog's ``search`` method, which is
composed of the name of our index and a value we'd like to find a
View
7 repoze/catalog/catalog.py
@@ -58,9 +58,10 @@ def search(self, **query):
(num, resultseq) based on the merging of results from
individual indexes.
- .. note:: this method is deprecated as of
- :mod:`repoze.catalog` version 0.8. Use
- :meth:`repoze.catalog.Catalog.query` instead.
+ .. note::
+
+ this method is deprecated as of :mod:`repoze.catalog` version
+ 0.8. Use :meth:`repoze.catalog.Catalog.query` instead.
"""
View
194 repoze/catalog/indexes/intfield.py
@@ -0,0 +1,194 @@
+from zope.interface import implements
+
+from zope.index.interfaces import IStatistics
+from repoze.catalog.interfaces import ICatalogIndex
+
+from repoze.catalog.indexes.field import CatalogFieldIndex
+from repoze.catalog import Range
+
+from BTrees.Length import Length
+
+_marker = object()
+
+class CatalogIntFieldIndex(CatalogFieldIndex):
+ """Indexes integer values using multiple granularity levels.
+
+ The multiple levels of granularity make it possible to query large ranges
+ without loading as many IFTreeSets from the forward index as when a
+ regular CatalogFieldIndex is used.
+ """
+ implements(ICatalogIndex, IStatistics)
+
+ def __init__(self, discriminator, levels=(1000,)):
+ """Create an index.
+
+ ``levels`` is a sequence of integer coarseness levels. The default
+ is (1000,).
+ """
+ self._levels = tuple(levels)
+ super(CatalogIntFieldIndex, self).__init__(discriminator)
+
+ def clear(self):
+ """Initialize all mappings."""
+ # The forward index maps an indexed value to IFSet(docids)
+ self._fwd_index = self.family.IO.BTree()
+ # The reverse index maps a docid to its index value
+ self._rev_index = self.family.II.BTree()
+ self._num_docs = Length(0)
+ # self._granular_indexes: [(level, BTree(value -> IFSet([docid])))]
+ self._granular_indexes = [(level, self.family.IO.BTree())
+ for level in self._levels]
+
+ def index_doc(self, docid, obj):
+ if callable(self.discriminator):
+ value = self.discriminator(obj, _marker)
+ else:
+ value = getattr(obj, self.discriminator, _marker)
+
+ if value is _marker:
+ # unindex the previous value
+ self.unindex_doc(docid)
+ return
+
+ if not isinstance(value, int):
+ raise ValueError(
+ 'GranularIndex cannot index non-integer value %s' % value)
+
+ rev_index = self._rev_index
+ if docid in rev_index:
+ if docid in self._fwd_index.get(value, ()):
+ # There's no need to index the doc; it's already up to date.
+ return
+ # unindex doc if present
+ self.unindex_doc(docid)
+
+ # Insert into forward index.
+ set = self._fwd_index.get(value)
+ if set is None:
+ set = self.family.IF.TreeSet()
+ self._fwd_index[value] = set
+ set.insert(docid)
+
+ # increment doc count
+ self._num_docs.change(1)
+
+ # Insert into reverse index.
+ rev_index[docid] = value
+
+ for level, ndx in self._granular_indexes:
+ v = value // level
+ set = ndx.get(v)
+ if set is None:
+ set = self.family.IF.TreeSet()
+ ndx[v] = set
+ set.insert(docid)
+
+ def unindex_doc(self, docid):
+ rev_index = self._rev_index
+ value = rev_index.get(docid)
+ if value is None:
+ return # not in index
+
+ del rev_index[docid]
+
+ self._num_docs.change(-1)
+
+ ndx = self._fwd_index
+ try:
+ set = ndx[value]
+ set.remove(docid)
+ if not set:
+ del ndx[value]
+ except KeyError:
+ pass
+
+ for level, ndx in self._granular_indexes:
+ v = value // level
+ try:
+ set = ndx[v]
+ set.remove(docid)
+ if not set:
+ del ndx[v]
+ except KeyError:
+ pass
+
+ def search(self, queries, operator='or'):
+ sets = []
+ for query in queries:
+ if isinstance(query, Range):
+ query = query.as_tuple()
+ else:
+ query = (query, query)
+
+ set = self.family.IF.multiunion(self.docids_in_range(*query))
+ sets.append(set)
+
+ result = None
+
+ if len(sets) == 1:
+ result = sets[0]
+ elif operator == 'and':
+ sets.sort()
+ for set in sets:
+ result = self.family.IF.intersection(set, result)
+ else:
+ result = self.family.IF.multiunion(sets)
+
+ return result
+
+ def docids_in_range(self, min, max):
+ """List the docids for an integer range, inclusive on both ends.
+
+ min or max can be None, making them unbounded.
+
+ Returns an iterable of IFSets.
+ """
+ for level, ndx in sorted(self._granular_indexes, reverse=True):
+ # Try to fill the range using coarse buckets first.
+ # Use only buckets that completely fill the range.
+ # For example, if start is 2 and level is 10, then we can't
+ # use bucket 0; only buckets 1 and greater are useful.
+ # Similarly, if end is 18 and level is 10, then we can't use
+ # bucket 1; only buckets 0 and less are useful.
+ if min is not None:
+ a = (min + level - 1) // level
+ else:
+ a = None
+ if max is not None:
+ b = (max - level + 1) // level
+ else:
+ b = None
+ # a and b are now coarse bucket values (or None).
+ if a is None or b is None or a <= b:
+ sets = []
+ if a is not None and min < a * level:
+ # include the gap before
+ sets.extend(self.docids_in_range(min, a * level - 1))
+ sets.extend(ndx.values(a, b))
+ if b is not None and (b + 1) * level - 1 < max:
+ # include the gap after
+ sets.extend(self.docids_in_range((b + 1) * level, max))
+ return sets
+
+ return self._fwd_index.values(min, max)
+
+
+def convert_field_to_intfield(index, levels=(1000,)):
+ """Create a IntFieldIndex from a FieldIndex. Returns the new index.
+
+ Copies the data without resolving the objects.
+ """
+ g = CatalogIntFieldIndex(index.discriminator, levels=levels)
+ treeset = g.family.IF.TreeSet
+ for value, docids in index._fwd_index.iteritems():
+ g._fwd_index[value] = treeset(docids)
+ for level, ndx in g._granular_indexes:
+ v = value // level
+ set = ndx.get(v)
+ if set is None:
+ set = treeset()
+ ndx[v] = set
+ set.update(docids)
+ g._rev_index.update(index._rev_index)
+ g._num_docs.value = index._num_docs()
+ return g
View
374 repoze/catalog/indexes/tests/test_intfield.py
@@ -0,0 +1,374 @@
+import unittest
+
+class TestCatalogIntFieldIndex(unittest.TestCase):
+
+ def _class(self):
+ from repoze.catalog.indexes.intfield import CatalogIntFieldIndex
+ return CatalogIntFieldIndex
+
+ def _make(self, *args, **kw):
+ return self._class()(*args, **kw)
+
+ def _make_default(self):
+ def discriminator(value, default):
+ return value
+
+ return self._make(discriminator)
+
+ def test_verifyImplements_ICatalogIndex(self):
+ from zope.interface.verify import verifyClass
+ from repoze.catalog.interfaces import ICatalogIndex
+ verifyClass(ICatalogIndex, self._class())
+
+ def test_verifyProvides_ICatalogIndex(self):
+ from zope.interface.verify import verifyObject
+ from repoze.catalog.interfaces import ICatalogIndex
+ verifyObject(ICatalogIndex, self._make_default())
+
+ def test_verifyImplements_IStatistics(self):
+ from zope.interface.verify import verifyClass
+ from zope.index.interfaces import IStatistics
+ verifyClass(IStatistics, self._class())
+
+ def test_verifyProvides_IStatistics(self):
+ from zope.interface.verify import verifyObject
+ from zope.index.interfaces import IStatistics
+ verifyObject(IStatistics, self._make_default())
+
+ def test_verifyImplements_IInjection(self):
+ from zope.interface.verify import verifyClass
+ from zope.index.interfaces import IInjection
+ verifyClass(IInjection, self._class())
+
+ def test_verifyProvides_IInjection(self):
+ from zope.interface.verify import verifyObject
+ from zope.index.interfaces import IInjection
+ verifyObject(IInjection, self._make_default())
+
+ def test_verifyImplements_IIndexSearch(self):
+ from zope.interface.verify import verifyClass
+ from zope.index.interfaces import IIndexSearch
+ verifyClass(IIndexSearch, self._class())
+
+ def test_verifyProvides_IIndexSearch(self):
+ from zope.interface.verify import verifyObject
+ from zope.index.interfaces import IIndexSearch
+ verifyObject(IIndexSearch, self._make_default())
+
+ def test_verifyImplements_IIndexSort(self):
+ from zope.interface.verify import verifyClass
+ from zope.index.interfaces import IIndexSort
+ verifyClass(IIndexSort, self._class())
+
+ def test_verifyProvides_IIndexSort(self):
+ from zope.interface.verify import verifyObject
+ from zope.index.interfaces import IIndexSort
+ verifyObject(IIndexSort, self._make_default())
+
+ def test_verifyImplements_IPersistent(self):
+ from zope.interface.verify import verifyClass
+ from persistent.interfaces import IPersistent
+ verifyClass(IPersistent, self._class())
+
+ def test_verifyProvides_IPersistent(self):
+ from zope.interface.verify import verifyObject
+ from persistent.interfaces import IPersistent
+ verifyObject(IPersistent, self._make_default())
+
+ def test_ctor(self):
+ obj = self._make_default()
+ self.assertEqual(len(obj._granular_indexes), 1)
+ self.assertEqual(obj._granular_indexes[0][0], 1000)
+ self.assertFalse(obj._granular_indexes[0][1])
+ self.assertEqual(obj._num_docs(), 0)
+
+ def test_index_doc_with_new_doc(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9000])
+ self.assertEqual(sorted(obj._fwd_index[9000]), [5])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5])
+ self.assertEqual(obj._rev_index[5], 9000)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9])
+ self.assertEqual(sorted(ndx[9]), [5])
+ self.assertEqual(obj._num_docs(), 1)
+
+ def test_index_doc_with_attr_discriminator(self):
+ obj = self._make('x')
+ obj.index_doc(5, DummyModel(x=9005))
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9005])
+ self.assertEqual(sorted(obj._fwd_index[9005]), [5])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5])
+ self.assertEqual(obj._rev_index[5], 9005)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9])
+ self.assertEqual(sorted(ndx[9]), [5])
+ self.assertEqual(obj._num_docs(), 1)
+
+ def test_index_doc_with_discriminator_returns_default(self):
+ def discriminator(obj, default):
+ return default
+
+ obj = self._make(discriminator)
+ obj.index_doc(5, DummyModel())
+ self.assertEqual(sorted(obj._fwd_index.keys()), [])
+ self.assertEqual(sorted(obj._rev_index.keys()), [])
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [])
+ self.assertEqual(obj._num_docs(), 0)
+
+ def test_index_doc_with_non_integer_value(self):
+ obj = self._make_default()
+ self.assertRaises(ValueError, obj.index_doc, 5, 'x')
+
+ def test_index_doc_with_changed_doc(self):
+ obj = self._make_default()
+ obj.index_doc(5, 14000)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [14000])
+ self.assertEqual(sorted(obj._fwd_index[14000]), [5])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5])
+ self.assertEqual(obj._rev_index[5], 14000)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [14])
+ self.assertEqual(sorted(ndx[14]), [5])
+
+ obj.index_doc(5, 9000)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9000])
+ self.assertEqual(sorted(obj._fwd_index[9000]), [5])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5])
+ self.assertEqual(obj._rev_index[5], 9000)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9])
+ self.assertEqual(sorted(ndx[9]), [5])
+ self.assertEqual(obj._num_docs(), 1)
+
+ def test_index_doc_with_no_changes(self):
+ obj = self._make_default()
+ for _i in range(2):
+ obj.index_doc(5, 9000)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9000])
+ self.assertEqual(sorted(obj._fwd_index[9000]), [5])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5])
+ self.assertEqual(obj._rev_index[5], 9000)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9])
+ self.assertEqual(sorted(ndx[9]), [5])
+ self.assertEqual(obj._num_docs(), 1)
+
+ def test_index_doc_with_multiple_docs(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9000, 9001, 11005])
+ self.assertEqual(sorted(obj._fwd_index[9000]), [5, 6])
+ self.assertEqual(sorted(obj._fwd_index[9001]), [7])
+ self.assertEqual(sorted(obj._fwd_index[11005]), [8])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5, 6, 7, 8])
+ self.assertEqual(obj._rev_index[5], 9000)
+ self.assertEqual(obj._rev_index[6], 9000)
+ self.assertEqual(obj._rev_index[7], 9001)
+ self.assertEqual(obj._rev_index[8], 11005)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9, 11])
+ self.assertEqual(sorted(ndx[9]), [5, 6, 7])
+ self.assertEqual(sorted(ndx[11]), [8])
+ self.assertEqual(obj._num_docs(), 4)
+
+ def test_unindex_doc_with_normal_indexes(self):
+ obj = self._make_default()
+ obj.index_doc(5, 14000)
+ obj.unindex_doc(5)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [])
+ self.assertEqual(sorted(obj._rev_index.keys()), [])
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [])
+ self.assertEqual(obj._num_docs(), 0)
+
+ def test_unindex_doc_with_incomplete_trees(self):
+ obj = self._make_default()
+ obj.index_doc(5, 14000)
+ del obj._fwd_index[14000]
+ del obj._granular_indexes[0][1][14]
+ obj.unindex_doc(5)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [])
+ self.assertEqual(sorted(obj._rev_index.keys()), [])
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [])
+ self.assertEqual(obj._num_docs(), 0)
+
+ def test_apply_with_one_value(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ self.assertEqual(sorted(obj.apply(9001)), [7])
+
+ def test_apply_with_two_values(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ q = {'query': [9001, 11005]}
+ self.assertEqual(sorted(obj.apply(q)), [7, 8])
+
+ def test_apply_with_small_range(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(9000, 9001)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7])
+
+ def test_apply_with_large_range(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(8000, 10000)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7])
+
+ def test_apply_with_multiple_ranges(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(8000, 10000), Range(11000, 11005)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7, 8])
+
+ def test_apply_with_union_ranges(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(8000, 10000), Range(9001, 11005)],
+ 'operator': 'or'}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7, 8])
+
+ def test_apply_with_intersecting_ranges(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(8000, 10000), Range(9001, 11005)],
+ 'operator': 'and'}
+ self.assertEqual(sorted(obj.apply(q)), [7])
+
+ def test_apply_with_range_that_excludes_9000(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(9001, 12000)]}
+ self.assertEqual(sorted(obj.apply(q)), [7, 8])
+
+ def test_apply_with_range_that_excludes_11006(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ obj.index_doc(9, 11006)
+ from repoze.catalog import Range
+ q = {'query': [Range(9000, 11005)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7, 8])
+
+ def test_apply_without_maximum(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(9001, None)]}
+ self.assertEqual(sorted(obj.apply(q)), [7, 8])
+
+ def test_apply_without_minimum(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+ from repoze.catalog import Range
+ q = {'query': [Range(None, 11004)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7])
+
+ def test_apply_with_unbounded_range(self):
+ obj = self._make_default()
+ obj.index_doc(5, 9000)
+ obj.index_doc(6, 9000)
+ obj.index_doc(7, 9001)
+ obj.index_doc(8, 11005)
+
+ # The operation should use _granular_indexes, not _fwd_index.
+ del obj._fwd_index
+
+ from repoze.catalog import Range
+ q = {'query': [Range(None, None)]}
+ self.assertEqual(sorted(obj.apply(q)), [5, 6, 7, 8])
+
+class Test_convert_field_to_intfield(unittest.TestCase):
+
+ def _call(self, index, levels=(1000,)):
+ from repoze.catalog.indexes.intfield import convert_field_to_intfield
+ return convert_field_to_intfield(index, levels)
+
+ def test_with_empty_index(self):
+ def discriminator(value, default): return value
+ from repoze.catalog.indexes.field import CatalogFieldIndex
+ src = CatalogFieldIndex(discriminator)
+ obj = self._call(src)
+ self.assertEqual(sorted(obj._fwd_index.keys()), [])
+ self.assertEqual(sorted(obj._rev_index.keys()), [])
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [])
+ self.assertEqual(obj._num_docs(), 0)
+
+ def test_with_contents(self):
+ def discriminator(value, default):
+ return value
+
+ from repoze.catalog.indexes.field import CatalogFieldIndex
+ src = CatalogFieldIndex(discriminator)
+ src.index_doc(5, 9000)
+ src.index_doc(6, 9000)
+ src.index_doc(7, 9001)
+ src.index_doc(8, 11005)
+
+ obj = self._call(src)
+
+ self.assertEqual(sorted(obj._fwd_index.keys()), [9000, 9001, 11005])
+ self.assertEqual(sorted(obj._fwd_index[9000]), [5, 6])
+ self.assertEqual(sorted(obj._fwd_index[9001]), [7])
+ self.assertEqual(sorted(obj._fwd_index[11005]), [8])
+ self.assertEqual(sorted(obj._rev_index.keys()), [5, 6, 7, 8])
+ self.assertEqual(obj._rev_index[5], 9000)
+ self.assertEqual(obj._rev_index[6], 9000)
+ self.assertEqual(obj._rev_index[7], 9001)
+ self.assertEqual(obj._rev_index[8], 11005)
+ ndx = obj._granular_indexes[0][1]
+ self.assertEqual(sorted(ndx.keys()), [9, 11])
+ self.assertEqual(sorted(ndx[9]), [5, 6, 7])
+ self.assertEqual(sorted(ndx[11]), [8])
+ self.assertEqual(obj._num_docs(), 4)
+
+class DummyModel(object):
+ def __init__(self, **kw):
+ self.__dict__.update(kw)
+
View
3  repoze/catalog/query.py
@@ -344,6 +344,9 @@ def __eq__(self, other):
self.start_exclusive == other.start_exclusive and
self.end_exclusive == other.end_exclusive)
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
class InRange(_Range):
""" Index value falls within a range.
View
1  setup.py
@@ -15,7 +15,6 @@
__version__ = '0.8.1'
import os
-import sys
from setuptools import setup, find_packages
View
23 tox.ini
@@ -0,0 +1,23 @@
+[tox]
+envlist =
+ py25,py26,py27
+
+[testenv]
+commands =
+ python setup.py test -q
+deps =
+ nose
+ zope.interface
+ zope.index
+ zope.component
+
+[testenv:py25]
+commands =
+ python setup.py test -q
+deps =
+ transaction<=1.1.999
+ nose
+ zope.interface
+ zope.index
+ zope.component
+

No commit comments for this range

Something went wrong with that request. Please try again.