Skip to content
This repository

Import speedup - Ticket #2181 #334

Closed
wants to merge 9 commits into from

7 participants

Scott Sinclair Don't Add Me To Your Organization a.k.a The Travis Bot Ralf Gommers Charles Harris Travis E. Oliphant Nathaniel J. Smith adalke
Scott Sinclair

Applies the patch submitted at http://projects.scipy.org/numpy/ticket/2181

see mailing list discussion:

http://mail.scipy.org/pipermail/numpy-discussion/2012-July/063130.html

especially - http://mail.scipy.org/pipermail/numpy-discussion/2012-July/063194.html - should polytemplate.py be run at build time instead of requiring it to be run manually when edited.

adalke and others added some commits
adalke adalke Remove the 5 'exec' calls definining polynomial classes.
Currently there are 5 places under numpy/polynomial which do
something like:

  exec polytemplate.substitute(name='Hermite', nick='herm', domain='[-1,1]')

I have changed those so there are start/end markers, like

  ...

I have edited  polytemplate.py so that it implements a __main__
which finds those block markers and replaces the content with
the appropriate "polytemplate" substitution.

This means that if you edit the template then you will need
to manually run 'polytemplate.py' so that the dependent files
are rebuilt based on the new template.

The performance of

  python -c 'import time; t1=time.time(); import numpy; print time.time()-t1'

goes from 0.079 seconds to 0.057 seconds (best of 10), for
an import speedup of ~25%.
2e52dcc
Scott Sinclair scottza STY: Ran reindent tool d59b142
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged d59b142 into 436a28f).

Scott Sinclair

By the way, all tests pass on my 64-bit Ubuntu machine for py24 through py32 using tox. Looks like we got the bad Travis worker machine..

Ralf Gommers
Owner

Yes, that's an unrelated failure.

Not requiring a manual action would indeed be better I think. But I'm okay with whatever option @charris prefers.

Charles Harris
Owner

Looks like a lot of this is a style cleanup that should probably come in as a separate pull request.

On the template, I have mixed feelings. The speed up isn't large and the added amount of code is significant. That said, I was never really happy with the template as is. And I'm not really happy with the current proposal either ;) It would be ideal if the files could be rewritten during the build phase, but that begins to become complicated... I'd like to think about possible solutions a bit more. Any other ideas? Note the main problem I had was the type specific documentation, otherwise the template could have been implemented as a mixin class.

Ralf Gommers
Owner

Rewriting at build time doesn't seem all that complicated; it's what I had in mind in the first place. Whether to do it like in this PR or by simply doing the current exec line at build time is a matter of preference I think. The latter doesn't increase code size at all, besides a few lines in setup.py I'd think.

To be more concrete, what I was thinking was to rename the current files to an underscored version, i.e. chebyshev.py --> _chebyshev.py and then during build do

lines = readlines(_chebyshev.py)
lines.append(exec polytemplate.replace...)
# write lines to chebyshev.py
Charles Harris
Owner

That seems reasonable and should be flexible enough that the template is still usable for other polynomial types. Not sure how this looks in Bento or Numscons.

Charles Harris
Owner

Although the polynomial package will no longer be stand alone. Not sure that that matters.

Ralf Gommers
Owner

Well, it's still stand alone in that it doesn't depend on the rest of Numpy. Don't think it matters much.

Travis E. Oliphant
Owner

I agree with Chuck that a lot of this diff is white-space and other style changes which should be separate from this PR. My $0.02 is that re-writing at build-time is not any more complicated than we are doing elsewhere and if it helps import times, I'm supportive. Everything I've heard from David tells me that this won't affect Bento at all, but David would know more about that.

scottza added some commits
Scott Sinclair scottza Revert "STY: Ran reindent tool"
This reverts commit d59b142.

Revert style related changes which are orthogonal to the purpose of
this branch.
08dadee
Scott Sinclair scottza REF: Rename files in preparation for build time module generation 1739d60
Scott Sinclair scottza ENH: Generate polynomial class modules at build time
* This implementation is based on the outline suggested by Ralf
  Gommers in numpy#334
b53c462
Scott Sinclair

I've reverted the commit with the style clean ups and implemented build time generation of the polynomial class modules. My changes shouldn't cause any problems with the Bento build, but I can't confirm that because the Bento build is currently broken (unrelated).

Scott Sinclair

Actually, my bad on the Bento build. I think that a bscript implementing the build time generation will be required...

Charles Harris
Owner

I just removed import polynomial from numpy/init.py and things work just fine from my point of view.

numpy/polynomial/setup.py
((2 lines not shown))
1 2
  3 +from polytemplate import polytemplate
  4 +
  5 +def generate_from_template(base_file, name, nick, domain):
  6 + """Generate polynomial class from a template."""
  7 + base_path = os.path.join('numpy', 'polynomial')
  8 +
  9 + fp = open(os.path.join(base_path, base_file))
  10 + lines = fp.readlines()
2
Charles Harris Owner
charris added a note

Why not fp.read?

Scott Sinclair
scottza added a note

No good reason. That would be better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/polynomial/setup.py
((2 lines not shown))
1 2
  3 +from polytemplate import polytemplate
  4 +
  5 +def generate_from_template(base_file, name, nick, domain):
  6 + """Generate polynomial class from a template."""
  7 + base_path = os.path.join('numpy', 'polynomial')
  8 +
  9 + fp = open(os.path.join(base_path, base_file))
  10 + lines = fp.readlines()
  11 + fp.close()
  12 +
  13 + lines.append(polytemplate.substitute(name=name,
  14 + nick=nick, domain=domain))
  15 +
  16 + fp = open(os.path.join(base_path, base_file[1:]), 'w')
  17 + for line in lines:
2
Charles Harris Owner
charris added a note

Can then do

fp.write(lines)
fp.write(polytemplate.substitute(...))
Scott Sinclair
scottza added a note

Agreed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/polynomial/_chebyshev.py
@@ -2006,4 +2005,3 @@ def chebpts2(npts):
2006 2005 # Chebyshev series class
2007 2006 #
2008 2007
2009   -exec polytemplate.substitute(name='Chebyshev', nick='cheb', domain='[-1,1]')
2
Charles Harris Owner
charris added a note

A note explaining that code for the Chebyshev class will be adding during the install process would be helpful. Likewise for the other classes.

Scott Sinclair
scottza added a note

Good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Ralf Gommers
Owner

Removing import polynomial probably won't fly, unless we're okay with breaking some user code. Now doing

import numpy as np.
np.polynomial.use_a_func()  

works.

Charles Harris
Owner

True, but I wonder if it's worth it to have it at that level. I put it there originally because that was how other packages were treated, but I wonder if requiring

from numpy.polynomial import Polynomial

Is too much to ask? If we refuse to make these small changes here and there it's hard to get things cleaned up.

Ralf Gommers
Owner

Indeed, it's hard to clean up. The cost of backwards compatibility I'm afraid. I agree it would have been better to not import it by default. I'm all for more and smaller namespaces anyway (I like np.<tab> not to return a list of 600 things) , but I guess that ship has sailed.

Charles Harris
Owner

Sigh. OK, the easy solution to import times isn't available. Last time I tried to deprecate imports didn't work that well eithe, I couldn't make the deprecation messages show up reliably.

I was guessing that the package wasn't used that much until recently due to the missing documentation, but I think that is starting to change...

Charles Harris
Owner

Maybe the lazy import ideas would also be a good way to deprecate imports.

Ralf Gommers
Owner

Please no. I wasted so much time hunting down bugs and then removing the lazy imports from scipy. I think this PR is a much better solution.

Charles Harris
Owner

I was thinking 'and', not exclusive 'or' ;) However, it might be useful to have a way to reliably deprecate imports down the road.

Ralf Gommers
Owner

That would be nice. But still I'd rather not have that than have lazy imports. It will break the most random stuff (like py2exe maybe).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 461d5be into 436a28f).

Scott Sinclair

I've made the changes suggested by Charles.

The idea of removing the polynomial import from numpy/init.py would have avoided breaking the Bento build (I'm pretty sure that this pull request will break polynomial when built with Bento). I can't find any documentation on Bento's hooks, but guessing from it's use elsewhere in Numpy and Scipy, that we need to put in a package level bscript with a pre_build function (or other?) decorator like so:

import os

from bento.commands.hooks import pre_build

from polytemplate import polytemplate

def generate_from_template(base_file, name, nick, domain):
    """Generate polynomial class from the template."""
    base_path = os.path.join('numpy', 'polynomial')

    fp = open(os.path.join(base_path, base_file))
    lines = fp.read()
    fp.close()

    fp = open(os.path.join(base_path, base_file[1:]), 'w')
    fp.write(lines)
    fp.write(polytemplate.substitute(name=name,
                                     nick=nick, domain=domain))
    fp.close()

@pre_build
def class_generation():
    generate_from_template('_chebyshev.py',
                           name='Chebyshev', nick='cheb', domain='[-1,1]')
    generate_from_template('_hermite.py',
                           name='Hermite', nick='herm', domain='[-1,1]')
    generate_from_template('_hermite_e.py',
                           name='HermiteE', nick='herme', domain='[-1,1]')
    generate_from_template('_laguerre.py',
                           name='Laguerre', nick='lag', domain='[-1,1]')
    generate_from_template('_legendre.py',
                           name='Legendre', nick='leg', domain='[-1,1]')
    generate_from_template('_polynomial.py',
                           name='Polynomial', nick='poly', domain='[-1,1]')
Scott Sinclair

I don't understand the Travisbot. Py3 builds work fine on my machine!?

Charles Harris
Owner

Don't worry about it, it might be a Travisbot problem. The same thing happened to my sorting PR after an update but went away when I uploaded the rebased branch with a git push --force. I don't recommend that approach unless you need to rebase, i.e., the merge button turns grey.

@cournape David, what needs to be done for the numscons and bento builds?

Charles Harris
Owner

Although there may be an import problem. Try a clean build, i.e., remove the build directory or the inplace build leftovers.

Scott Sinclair

I thought that I had, but just tested again to make sure

git clean -xfd
tox
....
[TOX] py24: commands succeeded
[TOX] py25: commands succeeded
[TOX] py26: commands succeeded
[TOX] py27: commands succeeded
[TOX] py31: commands succeeded
[TOX] py32: commands succeeded
[TOX] congratulations :)

Looks like a Travis issue..

Nathaniel J. Smith
Owner

How could getting this error message on all the Py3 builds be a Travis issue?

3408 File "/home/vagrant/virtualenv/python3.1/lib/python3.1/site-packages/numpy/__init__.py", line 153, in <module>

3409 from . import polynomial

3410 File "/home/vagrant/virtualenv/python3.1/lib/python3.1/site-packages/numpy/polynomial/__init__.py", line 18, in <module>

3411 from polynomial import Polynomial

3412ImportError: No module named polynomial

Maybe a missing "."?

Charles Harris
Owner

I'm thinking of another modification that might help. If we define an environment variable NPY_STRIP_IMPORTS we could make some of the imports go away with an if statement.

Nathaniel J. Smith
Owner

(And for the record, I'd like a better explanation of why lazy importing won't work before we go breaking the API. I understand it caused problems in Scipy, and sure, likely it will cause problems here. But I can't tell how many of those problems were intrinsic to lazy importing, and how many were caused by _import_tools being incredibly convoluted and over-complicated. Certainly both tab-completion and trac bug #1501 seem to be _import_tools-specific problems. It's true py2exe would need some hand-holding to find numpy libraries, but py2exe needs hand-holding every time you sneeze, and has a whole wiki full of examples of how to do it.)

Nathaniel J. Smith
Owner

On second thought, this is the wrong place for that discussion anyway. I suggest we leave the 'import .polynomial" out of this PR and take that discussion to the list?

Scott Sinclair

@njsmith Hmmm, something odd on my machine then. No errors reported by Tox, but when I install using Python3.2 directly I get the same failure reported by Travisbot:

Traceback (most recent call last):
File "", line 1, in
File "/home/scott/.local/lib/python3.2/site-packages/numpy/init.py", line 153, in
from . import polynomial
File "/home/scott/.local/lib/python3.2/site-packages/numpy/polynomial/init.py", line 18, in
from polynomial import Polynomial
ImportError: No module named polynomial

Nathaniel J. Smith njsmith commented on the diff
numpy/polynomial/polytemplate.py
((17 lines not shown))
10 12
11 13 """
12 14 import string
13   -import sys
14   -
15   -if sys.version_info[0] >= 3:
16   - rel_import = "from . import"
17   -else:
18   - rel_import = "import"
3
Nathaniel J. Smith Owner
njsmith added a note

I bet it's the lack of this code that's causing the import problem on Py3.

Scott Sinclair
scottza added a note
Scott Sinclair
scottza added a note

tl;dr We're confusing 2to3. Py3 builds of this branch against Git result in failure. Py3 builds against a source dist work fine. Regenerate numpy/polynomial/__init__.py at build time for Py3, or use explicit imports everywhere.


I've just had another look at this. This code removal is unrelated to the Py3 errors. The problem is related to the 2to3 tool.

When a Py3 build is run directly from the repository, the 2to3 tool doesn't refactor the relative imports (e.g. from polynomial import Polynomial) in numpy/polynomial/__init__.py (presumably because the files don't exist at the time 2to3 is run and the tool must therefore assume they are meant to be external imports).

When a Py3 build is run from a source distribution, the numpy/polynomial/polynomial.py etc. files already exist and the relative imports can be correctly generated in numpy/polynomial/__init__.py. This explains why testing with tox works on my machine - the build and test cycle for each Py version is based on a single sdist created by the system python and cached at ~/.tox/distshare/

Travis CI farms out the build tasks and runs the build and test for each Py version against a fresh Git checkout - hence the reported Py3 failures.

To fix, numpy/polynomial/__init__.py also needs to be regenerated at build time for Py3, or use explicit imports from numpy.polynomial.polynomial import ... etc. everywhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Ralf Gommers
Owner

@njsmith: unless I'm completely missing the point here, we're not breaking the API here. All we do is reducing import time, by removing an expensive exec statement. Therefore I fail to see the point of adding lazy imports.

Nathaniel J. Smith
Owner

@rgommers: Yeah, that was in response to the discussion between you and @charris about avoiding top-level imports, which I see I didn't read closely enough. I agree it has nothing to do with this pull request.

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged 76b700c into 436a28f).

Scott Sinclair

<High fives travisbot>

Nathaniel J. Smith
Owner

Looks fine to me now. I'll leave @charris make the final determination though.

Charles Harris
Owner

I'm fine with putting this in, but I think the numscons build should work first. We can probably leave the bento build to David. I suspect David is travelling at the moment...

@cournape David, what do we need to do for numscons and bento?

Scott Sinclair

For Bento - I now think that a bento.info file containing at least HookFile: bscript is required in addition to the bscript I outlined earlier, but that's mostly guesswork..

I have zero understanding of Numscons.

Charles Harris
Owner

I'll take a look a numscons if David doesn't chime in.

Charles Harris
Owner

The install puts the generated files in numpy/polynomial instead of in the build directory, that needs to be fixed.

Scott Sinclair
Charles Harris
Owner

Thanks. Looks like it could be a bit of work to figure out where everything goes what with all the different directories that accumulate under build for the different the python versions etc. Somewhere the path gets passed down from the top but I'm not sure how. I've avoided dealing with build stuff over the years.

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged ddaadac into 436a28f).

Scott Sinclair

Not tested on Windows/Mac etc..

Charles Harris
Owner

I think this needs to be brought to completion. I'll take a look this weekend.

Charles Harris
Owner

@scottza needs a rebase. Much has changed since the original PR.

Charles Harris
Owner

@scottza In fact, I suggest you redo this PR starting with current master. That is probably the easiest way to clean up the history, and now that you know what you are doing it probably isn't that much work..

Scott Sinclair
Scott Sinclair

@charris - you're right, it was easier to create a fresh pull request #3639

Scott Sinclair scottza closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 9 unique commits by 2 authors.

Jul 10, 2012
adalke adalke Remove the 5 'exec' calls definining polynomial classes.
Currently there are 5 places under numpy/polynomial which do
something like:

  exec polytemplate.substitute(name='Hermite', nick='herm', domain='[-1,1]')

I have changed those so there are start/end markers, like

  ...

I have edited  polytemplate.py so that it implements a __main__
which finds those block markers and replaces the content with
the appropriate "polytemplate" substitution.

This means that if you edit the template then you will need
to manually run 'polytemplate.py' so that the dependent files
are rebuilt based on the new template.

The performance of

  python -c 'import time; t1=time.time(); import numpy; print time.time()-t1'

goes from 0.079 seconds to 0.057 seconds (best of 10), for
an import speedup of ~25%.
2e52dcc
Scott Sinclair scottza STY: Ran reindent tool d59b142
Jul 11, 2012
Scott Sinclair scottza Revert "STY: Ran reindent tool"
This reverts commit d59b142.

Revert style related changes which are orthogonal to the purpose of
this branch.
08dadee
Scott Sinclair scottza REF: Rename files in preparation for build time module generation 1739d60
Scott Sinclair scottza ENH: Generate polynomial class modules at build time
* This implementation is based on the outline suggested by Ralf
  Gommers in numpy#334
b53c462
Jul 12, 2012
Scott Sinclair scottza REF: Simplify class generation from template ea6b365
Scott Sinclair scottza DOC: Add notes explaining that polynomial classes are auto-generated 461d5be
Jul 16, 2012
Scott Sinclair scottza BUG: Explicit imports to avoid confusing 2to3 tool for Py3 builds 76b700c
Jul 17, 2012
Scott Sinclair scottza BLD: Put generated files in build directory for distutils build ddaadac
This page is out of date. Refresh to see the latest.
12 numpy/polynomial/__init__.py
@@ -15,12 +15,12 @@
15 15 """
16 16 import warnings
17 17
18   -from polynomial import Polynomial
19   -from chebyshev import Chebyshev
20   -from legendre import Legendre
21   -from hermite import Hermite
22   -from hermite_e import HermiteE
23   -from laguerre import Laguerre
  18 +from numpy.polynomial.polynomial import Polynomial
  19 +from numpy.polynomial.chebyshev import Chebyshev
  20 +from numpy.polynomial.legendre import Legendre
  21 +from numpy.polynomial.hermite import Hermite
  22 +from numpy.polynomial.hermite_e import HermiteE
  23 +from numpy.polynomial.laguerre import Laguerre
24 24
25 25 from numpy.testing import Tester
26 26 test = Tester().test
8 numpy/polynomial/chebyshev.py → numpy/polynomial/_chebyshev.py
@@ -91,7 +91,6 @@
91 91 import numpy.linalg as la
92 92 import polyutils as pu
93 93 import warnings
94   -from polytemplate import polytemplate
95 94
96 95 __all__ = ['chebzero', 'chebone', 'chebx', 'chebdomain', 'chebline',
97 96 'chebadd', 'chebsub', 'chebmulx', 'chebmul', 'chebdiv', 'chebpow',
@@ -416,7 +415,7 @@ def cheb2poly(c) :
416 415 array([ -2., -8., 4., 12.])
417 416
418 417 """
419   - from polynomial import polyadd, polysub, polymulx
  418 + from numpy.polynomial.polynomial import polyadd, polysub, polymulx
420 419
421 420 [c] = pu.as_series([c])
422 421 n = len(c)
@@ -2005,5 +2004,8 @@ def chebpts2(npts):
2005 2004 #
2006 2005 # Chebyshev series class
2007 2006 #
  2007 +# Code below this point is generated at build time from the template
  2008 +# in polytemplate.py and combined with _chebyshev.py (see setup.py in
  2009 +# this directory).
  2010 +#
2008 2011
2009   -exec polytemplate.substitute(name='Chebyshev', nick='cheb', domain='[-1,1]')
8 numpy/polynomial/hermite.py → numpy/polynomial/_hermite.py
@@ -63,7 +63,6 @@
63 63 import numpy.linalg as la
64 64 import polyutils as pu
65 65 import warnings
66   -from polytemplate import polytemplate
67 66
68 67 __all__ = ['hermzero', 'hermone', 'hermx', 'hermdomain', 'hermline',
69 68 'hermadd', 'hermsub', 'hermmulx', 'hermmul', 'hermdiv', 'hermpow',
@@ -160,7 +159,7 @@ def herm2poly(c) :
160 159 array([ 0., 1., 2., 3.])
161 160
162 161 """
163   - from polynomial import polyadd, polysub, polymulx
  162 + from numpy.polynomial.polynomial import polyadd, polysub, polymulx
164 163
165 164 [c] = pu.as_series([c])
166 165 n = len(c)
@@ -1740,5 +1739,8 @@ def hermweight(x):
1740 1739 #
1741 1740 # Hermite series class
1742 1741 #
  1742 +# Code below this point is generated at build time from the template
  1743 +# in polytemplate.py and combined with _hermite.py (see setup.py in
  1744 +# this directory).
  1745 +#
1743 1746
1744   -exec polytemplate.substitute(name='Hermite', nick='herm', domain='[-1,1]')
8 numpy/polynomial/hermite_e.py → numpy/polynomial/_hermite_e.py
@@ -63,7 +63,6 @@
63 63 import numpy.linalg as la
64 64 import polyutils as pu
65 65 import warnings
66   -from polytemplate import polytemplate
67 66
68 67 __all__ = ['hermezero', 'hermeone', 'hermex', 'hermedomain', 'hermeline',
69 68 'hermeadd', 'hermesub', 'hermemulx', 'hermemul', 'hermediv', 'hermpow',
@@ -160,7 +159,7 @@ def herme2poly(c) :
160 159 array([ 0., 1., 2., 3.])
161 160
162 161 """
163   - from polynomial import polyadd, polysub, polymulx
  162 + from numpy.polynomial.polynomial import polyadd, polysub, polymulx
164 163
165 164 [c] = pu.as_series([c])
166 165 n = len(c)
@@ -1736,5 +1735,8 @@ def hermeweight(x):
1736 1735 #
1737 1736 # HermiteE series class
1738 1737 #
  1738 +# Code below this point is generated at build time from the template
  1739 +# in polytemplate.py and combined with _hermite_e.py (see setup.py in
  1740 +# this directory).
  1741 +#
1739 1742
1740   -exec polytemplate.substitute(name='HermiteE', nick='herme', domain='[-1,1]')
8 numpy/polynomial/laguerre.py → numpy/polynomial/_laguerre.py
@@ -63,7 +63,6 @@
63 63 import numpy.linalg as la
64 64 import polyutils as pu
65 65 import warnings
66   -from polytemplate import polytemplate
67 66
68 67 __all__ = ['lagzero', 'lagone', 'lagx', 'lagdomain', 'lagline',
69 68 'lagadd', 'lagsub', 'lagmulx', 'lagmul', 'lagdiv', 'lagpow',
@@ -159,7 +158,7 @@ def lag2poly(c) :
159 158 array([ 0., 1., 2., 3.])
160 159
161 160 """
162   - from polynomial import polyadd, polysub, polymulx
  161 + from numpy.polynomial.polynomial import polyadd, polysub, polymulx
163 162
164 163 [c] = pu.as_series([c])
165 164 n = len(c)
@@ -1732,5 +1731,8 @@ def lagweight(x):
1732 1731 #
1733 1732 # Laguerre series class
1734 1733 #
  1734 +# Code below this point is generated at build time from the template
  1735 +# in polytemplate.py and combined with _laguerre.py (see setup.py in
  1736 +# this directory).
  1737 +#
1735 1738
1736   -exec polytemplate.substitute(name='Laguerre', nick='lag', domain='[-1,1]')
8 numpy/polynomial/legendre.py → numpy/polynomial/_legendre.py
@@ -87,7 +87,6 @@
87 87 import numpy.linalg as la
88 88 import polyutils as pu
89 89 import warnings
90   -from polytemplate import polytemplate
91 90
92 91 __all__ = ['legzero', 'legone', 'legx', 'legdomain', 'legline',
93 92 'legadd', 'legsub', 'legmulx', 'legmul', 'legdiv', 'legpow', 'legval',
@@ -191,7 +190,7 @@ def leg2poly(c) :
191 190
192 191
193 192 """
194   - from polynomial import polyadd, polysub, polymulx
  193 + from numpy.polynomial.polynomial import polyadd, polysub, polymulx
195 194
196 195 [c] = pu.as_series([c])
197 196 n = len(c)
@@ -1758,5 +1757,8 @@ def legweight(x):
1758 1757 #
1759 1758 # Legendre series class
1760 1759 #
  1760 +# Code below this point is generated at build time from the template
  1761 +# in polytemplate.py and combined with _legendre.py (see setup.py in
  1762 +# this directory).
  1763 +#
1761 1764
1762   -exec polytemplate.substitute(name='Legendre', nick='leg', domain='[-1,1]')
6 numpy/polynomial/polynomial.py → numpy/polynomial/_polynomial.py
@@ -64,7 +64,6 @@
64 64 import numpy.linalg as la
65 65 import polyutils as pu
66 66 import warnings
67   -from polytemplate import polytemplate
68 67
69 68 polytrim = pu.trimcoef
70 69
@@ -1482,5 +1481,8 @@ def polyroots(c):
1482 1481 #
1483 1482 # polynomial class
1484 1483 #
  1484 +# Code below this point is generated at build time from the template
  1485 +# in polytemplate.py and combined with _polynomial.py (see setup.py in
  1486 +# this directory).
  1487 +#
1485 1488
1486   -exec polytemplate.substitute(name='Polynomial', nick='poly', domain='[-1,1]')
25 numpy/polynomial/polytemplate.py
... ... @@ -1,28 +1,19 @@
1 1 """
2 2 Template for the Chebyshev and Polynomial classes.
3 3
  4 +This is an internal module and should not be imported.
  5 +
4 6 This module houses a Python string module Template object (see, e.g.,
5   -http://docs.python.org/library/string.html#template-strings) used by
6   -the `polynomial` and `chebyshev` modules to implement their respective
7   -`Polynomial` and `Chebyshev` classes. It provides a mechanism for easily
8   -creating additional specific polynomial classes (e.g., Legendre, Jacobi,
9   -etc.) in the future, such that all these classes will have a common API.
  7 +http://docs.python.org/library/string.html#template-strings) used to
  8 +generate the code for the `polynomial`, `chebyshev`, `hermite`,
  9 +`hermite_e`, `laguerre`, and `lengendre` modules, to implement their
  10 +respective `Polynomial`, `Chebyshev`, `Hermite`, `HermiteE`,
  11 +`Laguerre`, and `Lengendre` classes.
10 12
11 13 """
12 14 import string
13   -import sys
14   -
15   -if sys.version_info[0] >= 3:
16   - rel_import = "from . import"
17   -else:
18   - rel_import = "import"
19 15
20 16 polytemplate = string.Template('''
21   -from __future__ import division
22   -import numpy as np
23   -import warnings
24   -REL_IMPORT polyutils as pu
25   -
26 17 class $name(pu.PolyBase) :
27 18 """A $name series class.
28 19
@@ -916,4 +907,4 @@ def cast(series, domain=$domain, window=$domain):
916 907 """
917 908 return series.convert(domain, $name, window)
918 909
919   -'''.replace('REL_IMPORT', rel_import))
  910 +''')
45 numpy/polynomial/setup.py
... ... @@ -1,6 +1,51 @@
  1 +import os
  2 +import sys
  3 +from distutils import util
1 4
  5 +from polytemplate import polytemplate
  6 +
  7 +def get_build_dir():
  8 + """Determine build directory."""
  9 + plat = util.get_platform()
  10 + py_ver = '%s.%s' % (sys.version_info[0], sys.version_info[1])
  11 +
  12 + return os.path.join('build',
  13 + 'lib.%s-%s' % (plat, py_ver), 'numpy', 'polynomial')
  14 +
  15 +def generate_from_template(base_file, name, nick, domain):
  16 + """Generate polynomial class from the template."""
  17 + base_path = os.path.join('numpy', 'polynomial')
  18 +
  19 + fp = open(os.path.join(base_path, base_file))
  20 + lines = fp.read()
  21 + fp.close()
  22 +
  23 + build_path = get_build_dir()
  24 + if not os.path.isdir(build_path):
  25 + # Distutils doesn't create the build directory until after
  26 + # this code has been executed.
  27 + os.makedirs(build_path)
  28 +
  29 + fp = open(os.path.join(build_path, base_file[1:]), 'w')
  30 + fp.write(lines)
  31 + fp.write(polytemplate.substitute(name=name,
  32 + nick=nick, domain=domain))
  33 + fp.close()
2 34
3 35 def configuration(parent_package='',top_path=None):
  36 + generate_from_template('_chebyshev.py',
  37 + name='Chebyshev', nick='cheb', domain='[-1,1]')
  38 + generate_from_template('_hermite.py',
  39 + name='Hermite', nick='herm', domain='[-1,1]')
  40 + generate_from_template('_hermite_e.py',
  41 + name='HermiteE', nick='herme', domain='[-1,1]')
  42 + generate_from_template('_laguerre.py',
  43 + name='Laguerre', nick='lag', domain='[-1,1]')
  44 + generate_from_template('_legendre.py',
  45 + name='Legendre', nick='leg', domain='[-1,1]')
  46 + generate_from_template('_polynomial.py',
  47 + name='Polynomial', nick='poly', domain='[-1,1]')
  48 +
4 49 from numpy.distutils.misc_util import Configuration
5 50 config = Configuration('polynomial',parent_package,top_path)
6 51 config.add_data_dir('tests')

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.