github
Advanced Search
  • Home
  • Pricing and Signup
  • Explore GitHub
  • Blog
  • Login

DanielStutzbach / blist

  • Admin
  • Watch Unwatch
  • Fork
  • Your Fork
  • Pull Request
  • Download Source
    • 13
    • 2
  • Source
  • Commits
  • Network (2)
  • Issues (11)
  • Graphs
  • Branch: master

click here to add a description

click here to add a homepage

  • Branches (2)
    • master ✓
    • sortedlist
  • Tags (12)
    • v1.1.1
    • v1.1.0
    • v1.0.2
    • v1.0.1
    • v0.9.11
    • blist_0_9_6
    • 0.9.4b
    • 0.9.4
    • 0.9.2
    • 0.9.1
    • 0.9
    • 0.7
Sending Request…
Enable Donations

Pledgie Donations

Once activated, we'll place the following badge in your repository's detail box:
Pledgie_example
This service is courtesy of Pledgie.

A list-like type with better asymptotic performance and similar performance on small lists — Read more

  cancel

http://stutzbachenterprises.com/blist/

  cancel
  • Private
  • Read-Only
  • HTTP Read-Only

This URL has Read+Write access

Bumped version 
DanielStutzbach (author)
Sun Jan 31 12:33:00 -0800 2010
commit  627a24e312e1e9f16217ef4e5d2243f9a6d38e0c
tree    b0653674ed013129ac54ebb14da8f73c15309567
parent  caa3b8231dca75ca14d390130a1a7d884887bb60
blist /
name age
history
message
file .gitignore Mon Oct 19 14:32:09 -0700 2009 Added *.pyd and dist/ to .gitignore [DanielStutzbach]
file LICENSE Sun Oct 11 18:50:06 -0700 2009 File permission fixes [hircus]
file MANIFEST.in Sun Jan 31 12:31:28 -0800 2010 Updated MANIFEST.IN [DanielStutzbach]
file Makefile Sun Jan 31 12:32:55 -0800 2010 Fixed username on stutzbachenterprises.com in M... [DanielStutzbach]
file README.rst Sun Jan 31 10:13:53 -0800 2010 Added a note to the README that the extra data ... [DanielStutzbach]
file _blist.c Mon Oct 19 14:32:34 -0700 2009 Fixed compilation error when compiling without ... [DanielStutzbach]
file _btuple.py Sun Jan 31 10:05:33 -0800 2010 Added btuple and sortedict. Fixed bugs in repr(... [DanielStutzbach]
file _sorteddict.py Sun Jan 31 10:10:56 -0800 2010 Fixed sorteddict to work in Python 3.1 [DanielStutzbach]
file _sortedlist.py Sun Jan 31 10:05:33 -0800 2010 Added btuple and sortedict. Fixed bugs in repr(... [DanielStutzbach]
file blist.h Thu Oct 15 09:15:27 -0700 2009 Renamed listobject.h to blist.h to avoid collis... [DanielStutzbach]
file blist.py Sun Jan 31 10:05:33 -0800 2010 Added btuple and sortedict. Fixed bugs in repr(... [DanielStutzbach]
file blist.rst Sun Oct 11 18:50:06 -0700 2009 File permission fixes [hircus]
file distribute_setup.py Sun Jan 31 12:23:37 -0800 2010 Upgraded to distribute_setup.py 0.6.10 [DanielStutzbach]
file fuzz.py Thu Oct 15 08:14:42 -0700 2009 Ported fuzz.py and speed_test.py to work under ... [DanielStutzbach]
directory prototype/ Thu Apr 26 07:22:19 -0700 2007 Prepared for release as an egg darcs-hash:2007... [DanielStutzbach]
file replot.py Fri Apr 27 08:27:13 -0700 2007 Added useful tools for testing darcs-hash:2007... [DanielStutzbach]
file setup.py Sun Jan 31 12:33:00 -0800 2010 Bumped version [DanielStutzbach]
file speed_test.py Thu Oct 15 08:14:42 -0700 2009 Ported fuzz.py and speed_test.py to work under ... [DanielStutzbach]
file speed_test_native.py Sat Oct 10 09:01:47 -0700 2009 Setup speed_test for custom python 2.5 darcs-h... [DanielStutzbach]
file test.c Sun Oct 11 18:50:06 -0700 2009 File permission fixes [hircus]
directory test/ Sun Jan 31 10:10:56 -0800 2010 Fixed sorteddict to work in Python 3.1 [DanielStutzbach]
file test_blist.py Sun Jan 31 10:05:33 -0800 2010 Added btuple and sortedict. Fixed bugs in repr(... [DanielStutzbach]
README.rst

blist: a list-like type with better performance

The blist is a drop-in replacement for the Python list the provides better performance when modifying large lists. Python's built-in list is a dynamically-sized array; to insert or removal an item from the beginning or middle of the list, it has to move most of the list in memory, i.e., O(n) operations. The blist uses a flexible, hybrid array/tree structure and only needs to move a small portion of items in memory, specifically using O(log n) operations.

For small lists, the blist and the built-in list have virtually identical performance.

To use the blist, you simply change code like this:

>>> items = [5, 6, 2]
>>> more_items = function_that_returns_a_list()

to:

>>> from blist import blist
>>> items = blist([5, 6, 2])
>>> more_items = blist(function_that_returns_a_list())

Here are some of the use cases where the blist asymptotically outperforms the built-in list:

Use Case blist list
Insertion into or removal from a list O(log n) O(n)
Taking slices of lists O(log n) O(n)
Making shallow copies of lists O(1) O(n)
Changing slices of lists O(log n + log k) O(n+k)
Multiplying a list to make a sparse list O(log k) O(kn)
Maintain a sorted lists with bisect.insort O(log**2 n) O(n)

So you can see the performance of the blist in more detail, several performance graphs available at the following link: http://stutzbachenterprises.com/blist/

Example usage:

>>> from blist import *
>>> x = blist([0])             # x is a blist with one element
>>> x *= 2**29                 # x is a blist with > 500 million elements
>>> x.append(5)                # append to x
>>> y = x[4:-234234]           # Take a 500 million element slice from x
>>> del x[3:1024]              # Delete a few thousand elements from x

Other data structures

The blist package provides other data structures based on the blist:

  • sortedlist
  • sortedset
  • weaksortedlist
  • weaksorteset
  • sorteddict
  • btuple

These additional data structures are only available in Python 2.6 or higher, as they make use of Abstract Base Classes.

The sortedlist is a list that's always sorted. It's iterable and indexable like a Python list, but to modify a sortedlist the same methods you would use on a Python set (add, discard, or remove).

>>> from blist import sortedlist
>>> my_list = sortedlist([3,7,2,1])
>>> my_list
sortedlist([1, 2, 3, 7])
>>> my_list.add(5)
>>> my_list[3]
5
>>>

The sortedlist constructor takes an optional "key" argument, which may be used to change the sort order just like the sorted() function.

>>> from blist import sortedlist
>>> my_list = sortedlist([3,7,2,1], key=lambda i: -i)
sortedlist([7, 3, 2, 1]
>>>

The sortedset is a set that's always sorted. It's iterable and indexable like a Python list, but modified like a set. Essentially, it's just like a sortedlist except that duplicates are ignored.

>>> from blist import sortedset
>>> my_set = sortedset([3,7,2,2])
sortedset([2, 3, 7]
>>>

The weaksortedlist and weaksortedset are weakref variations of the sortedlist and sortedset.

The sorteddict works just like a regular dict, except the keys are always sorted. The sorteddict should not be confused with Python 2.7's OrderedDict type, which remembers the insertion order of the keys.

>>> from blist import sorteddict
>>> my_dict = sorteddict({1: 5, 6: 8, -5: 9})
>>> my_dict.keys()
[-5, 1, 6]
>>>

The btuple is a drop-in replacement for the built-in tuple. Compared to the built-in tuple, the btuple offers the following advantages:

  • Constructing a btuple from a blist takes O(1) time.
  • Taking a slice of a btuple takes O(n) time, where n is the size of the original tuple. The size of the slice does not matter.
>>> from blist import blist, btuple
>>> x = blist([0])             # x is a blist with one element
>>> x *= 2**29                 # x is a blist with > 500 million elements
>>> y = btuple(x)              # y is a btuple with > 500 million elements

Installation instructions

Python 2.5 or higher is required. If building from the source distribution, the Python header files are also required. In either case, just run:

python setup.py install

The blist module will be installed in the 'site-packages' directory of your Python installation. (Unless directed elsewhere; see the "Installing Python Modules" section of the Python manuals for details on customizing installation locations, etc.).

If you downloaded the source distribution and wish to run the associated test suite, you can also run:

python setup.py test

which will verify the correct installation and functioning of the package. The tests require Python 2.6 or higher.

Feedback

We're eager to hear about your experiences with the blist. You can email me at daniel@stutzbachenterprises.com. Alternately, bug reports and feature requests may be reported on our bug tracker at: http://github.com/DanielStutzbach/blist/issues

How we test

In addition to the tests include in the source distribution, we perform the following to add extra rigor to our testing process:

  1. We use a "fuzzer": a program that randomly generates list operations, performs them using both the blist and the built-in list, and compares the results.
  2. We use a modified Python interpreter where we have replaced the array-based built-in list with the blist. Then, we run all of the regular Python unit tests.
Blog | Support | Training | Contact | API | Status | Twitter | Help | Security
© 2010 GitHub Inc. All rights reserved. | Terms of Service | Privacy Policy
Powered by the Dedicated Servers and
Cloud Computing of Rackspace Hosting®
Dedicated Server