Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial stab at docs for typed-list #7

Merged
merged 5 commits into from
Jul 6, 2019
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions docs/source/reference/pysupported.rst
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,15 @@ The following operations are supported:
list
----


.. warning::
As of version 0.45.0 the internal implementation for the list datatype in
esc marked this conversation as resolved.
Show resolved Hide resolved
Numba is changing. Until recently, only a single implementation of the list
datatype was available, the so-called *reflected-list* (see below).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps reference the documentation on reflection?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation for the that is almost exactly below, I am hoping this relative reference should suffice.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I can do, is reference the deprecation schedule.

However, it was deprecated from version 0.44.0 onwards due to its

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC It was not deprecated, it was scheduled for deprecation, see schedule: http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#schedule

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, I have reworded this and added a link to the deprecation schedule at the end of the warning.

limitations. As of version 0.45.0 a new implementation, the so-called
*typed-list* (see below), is available as an experimental feature.

Creating and returning lists from JIT-compiled functions is supported,
as well as all methods and operations. Lists must be strictly homogeneous:
Numba will reject any list containing objects of different types, even if
Expand Down Expand Up @@ -260,6 +269,78 @@ of this limitation.
List sorting currently uses a quicksort algorithm, which has different
performance characterics than the algorithm used by Python.

Typed List
''''''''''

.. note::
``numba.typed.List`` is an experimental feature, if you encounter any bugs in
functionality or suffer from unexpectedly bad performance, please report
this, ideally by opening an issue on the Numba issue tracker.

As of version 0.45.0 a new implementation of the list datatype is available, the
esc marked this conversation as resolved.
Show resolved Hide resolved
so-called *typed-list*. This is C backed, type-homogeneous list datatype that is
esc marked this conversation as resolved.
Show resolved Hide resolved
an improvement over the *reflected-list* mentioned above. No more reflection is
required which should make it significantly more efficient for larger lists.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not debatable depending on where boxing/unboxing occurs. The location of the penalty has just been moved?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is debatable, I'll scrap the last sentence for now

Additionally, lists can now be arbitrarily nested. Since the implementation is
considered experimental, you will need to import it explicitly::

In [1]: from numba.typed import List

In [2]: from numba import njit

In [3]: @njit
...: def foo(l):
...: l.append(23)
...: return l
...:

In [4]: mylist = List()

In [5]: mylist.append(1)

In [6]: foo(mylist)
Out[6]: ListType[int64]([1, 23])


.. note::
As the typed-list stabilizes it will fully replace the reflected-list and the
constructors `[]` and `list()` will create a typed-list instead of a
reflected one.


Here's an example using ``List()`` to create ``numba.typed.List`` inside a
jit-compiled function and letting the compiler infer the item types:

.. literalinclude:: ../../../examples/typed_list_usage.py
:language: python
:caption: from ``ex_inferred_list_jit`` of ``examples/typed_list_usage.py``
:start-after: magictoken.ex_inferred_list_jit.begin
:end-before: magictoken.ex_inferred_list_jit.end
:dedent: 4
:linenos:

Here's an example of using ``List()`` to create a ``numba.typed.List`` outside of
a jit-compiled function and then using it as an argument to a jit-compiled
function.

.. literalinclude:: ../../../examples/typed_list_usage.py
:language: python
:caption: from ``ex_inferred_list`` of ``examples/typed_list_usage.py``
:start-after: magictoken.ex_inferred_list.begin
:end-before: magictoken.ex_inferred_list.end
:dedent: 4
:linenos:

Lastly, here's an example of using a nested typed-list:
esc marked this conversation as resolved.
Show resolved Hide resolved

.. literalinclude:: ../../../examples/typed_list_usage.py
:language: python
:caption: from ``ex_nested_list`` of ``examples/typed_list_usage.py``
:start-after: magictoken.ex_nested_list.begin
:end-before: magictoken.ex_nested_list.end
:dedent: 4
:linenos:

.. _pysupported-comprehension:

List comprehension
Expand Down
75 changes: 75 additions & 0 deletions examples/typed_list_usage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Contents in this file are referenced from the sphinx-generated docs.
# "magictoken" is used for markers as beginning and ending of example text.


def ex_inferred_list_jit():
# magictoken.ex_inferred_list_jit.begin
from numba import njit
from numba.typed import List

@njit
def foo():
# Instatiate a typed-list
esc marked this conversation as resolved.
Show resolved Hide resolved
l = List()
# Append a value to it, this will set the type to int32/int64
esc marked this conversation as resolved.
Show resolved Hide resolved
l.append(42)
# The usual list operations, getitem, pop and length are supported
print(l[0]) # 42
l[0] = 23
print(l[0]) # 23
print(len(l)) # 1
l.pop()
print(len(l)) # 0
return l

mylist = foo()

# magictoken.ex_inferred_list_jit.end


def ex_inferred_list():
# magictoken.ex_inferred_list.begin
from numba import njit
from numba.typed import List

@njit
def foo(mylist):
for i in range(10, 20):
mylist.append()
return mylist

# Instatiate a typed-list, outside of a jit context
esc marked this conversation as resolved.
Show resolved Hide resolved
l = List()
# Append a value to it, this will set the type to int32/int64
esc marked this conversation as resolved.
Show resolved Hide resolved
l.append(42)
# The usual list operations, getitem, pop and length are supported
print(l[0]) # 42
l[0] = 23
print(l[0]) # 23
print(len(l)) # 1
l.pop()
print(len(l)) # 0

# And you can use the typed-list as an argument for a jit compiled function

l = foo(l)
print(len(l)) # 10

# magictoken.ex_inferred_list.end


def ex_nested_list():
# magictoken.ex_nested_list.begin
from numba.typed import List

# typed-lists can be nested in typed-lists
mylist = List()
for i in range(10):
l = List()
for i in range(10):
l.append(i)
mylist.append(l)
# mylist is now a 10x10 grid
esc marked this conversation as resolved.
Show resolved Hide resolved
print(mylist)

# magictoken.ex_nested_list.end