magic interpreter interpretes non magic commands?

[jankatins]

The below code throws a UsageError: Unknown variable '["final_all_"+field]' error when run in a ipython notebook:

%%time
from data_names import (hdf_store_name, hdf_business_authors, hdf_business_articles, hdf_nano_authors, hdf_nano_articles, 
                        graph_business, graph_nano, hdf_journaldata, hdf_final_all_business, hdf_final_all_nano)

combinations = [(hdf_business_authors, hdf_business_articles, graph_business, hdf_final_all_business, "business"), 
                (hdf_nano_authors, hdf_nano_articles, graph_nano, hdf_final_all_nano, "nano")]

store = pandas.HDFStore(hdf_store_name)
journals = store[hdf_journaldata]
for _authors, _articles, _graph, _result, field in combinations:
    progress_init(field)
    articles = store[_articles]
    authors = store[_authors]
    graph = igraph.Graph.Read(_graph)
    author_infos = compute_author_infos(authors, articles, graph, journals, field)
    progress_start("Saving: " + field)
    # Check...
    store["final_all_"+field] = author_infos

    progress_stop()
    progress_destroy()
#    print field
#    print author_infos.head()
#    print author_infos.describe()
#    break
store.close()

I've also seen similar things when I use 'run' as a variable name (if run%5 == 0: ...).

Unfortunately I can't reproduce this error when I try to construct a small reproduceable example.

[takluyver]

What version of IPython are you running? Even if you can't make it much smaller, can you construct a version of that failure that I can run (i.e. without your data_names module)? Is it only a problem when you're using %%time?

[jankatins]

I currently run a version based on dfc7609 ("based on", because I installed #3312).

data_names is only a file with lots of name = "string", so just replacing the import with a list of such definitions should be enough.

I currently only use the %%time magic.

I tried to produce a reproduceable example notebook by just replacing the imports and stubbing out the called functions, but even with that I can't reproduce the error. The only big difference is, that 'compute_author_infos' runs for about 40min and more, the datasets are bigger, and I left out the igraph call completely.

I'm curently running it with s/store/sto_re/ and this seems to work

[jankatins]

Hah, found a way to reproduce the run problem:

Cell1:

%%time
def testmethod():
    run = 0
    for v in xrange(1000):
        if (run % 10) == 0:
            sys.stdout.write("*")
            sys.stdout.flush()
        run += 1
    sys.stdout.write("\n")
    sys.stdout.flush()

Cell2

%%time
testmethod()

Both in a notebook...

[jankatins]

WTF:

%%time
def testmethod():
    run = 0
    for v in xrange(10):
        if (run % 5) == 0:
            sys.stdout.write("*")
            sys.stdout.flush()
        run += 1
    sys.stdout.write("\n")
    sys.stdout.flush()

-> no error, of course

%%time
testmethod()

-> 5 Error lines

%%time
run = 0
for v in xrange(10):
    if (run % 5) == 0:
        sys.stdout.write("*")
        sys.stdout.flush()
    run += 1
sys.stdout.write("\n")
sys.stdout.flush()

-> 5 Error lines

%%time
run = 0
for v in xrange(10):
    if (run % 5) == 0:
        sys.stdout.write("*")
        sys.stdout.flush()
    run += 1
sys.stdout.write("\n")
sys.stdout.flush()

-> NO error lines (was a straight copy from the cell above)

%%time
def testmethod():
    run = 0
    for v in xrange(10):
        if (run % 5) == 0:
            sys.stdout.write("*")
            sys.stdout.flush()
        run += 1
    sys.stdout.write("\n")
    sys.stdout.flush()

-> same content as cell 1, no error again

%%time
testmethod()

-> NO error! WTF?

[takluyver]

It seems to only produce the error if run isn't defined before you run the cell. And it only happens if you're using the %%time magic as well.

[takluyver]

Have a look at pull request #3336.