Flor doesn't work with functions with arguments :(

[rjurney]

Flor won't work with functions with arguments.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-ba0e1c3e4b78> in <module>
     13     epochs=10,
     14     validation_split=0.2,
---> 15     embedding_size=100
     16 ):
     17 

~/flor/flor/interface/input/execution_tracker.py in track(f)
    103     if filename not in os.listdir(secret_dir):
    104         # Needs compilation
--> 105         with open(tru_path, 'r') as sourcefile:
    106             tree = ast.parse(sourcefile.read())
    107             logger.debug(astor.dump_tree(tree))

TypeError: 'OutStream' object is not callable

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelBinarizer

from keras.preprocessing.text import Tokenizer
from keras.models import Sequential
from keras.layers import Flatten, Dense, Embedding, Activation, Dropout, LSTM, GlobalAveragePooling1D

@flor.track
def main(
    sample_size=1000, 
    vocab_size=1000, 
    batch_size=32, 
    epochs=10, 
    validation_split=0.2,
    embedding_size=100
):
    # From Series to lists
    groups = [p for p in patents['group_id']][0:sample_size]
    num_labels = len(set(groups))
    summaries = [s for s in patents['summary']][0:sample_size]

    # Test/train split, summary sequences as X, CPC groups as Y
    train_sums, test_sums, train_labels, test_labels = train_test_split(
        summaries,
        groups
    )

    label_count = len(set(groups))

    # Tokenize the summaries
    tokenizer = Tokenizer(num_words=vocab_size)
    tokenizer.fit_on_texts(summaries) 
    x_train = tokenizer.texts_to_matrix(train_sums, mode='tfidf')
    x_test = tokenizer.texts_to_matrix(test_sums, mode='tfidf')

    encoder = LabelBinarizer()
    encoder.fit(groups)
    y_train = encoder.transform(train_labels)
    y_test = encoder.transform(test_labels)

    print('Label count: {}'.format(label_count))
    print('x_train shape:', x_train.shape)
    print('x_test shape:', x_test.shape)
    print('y_train shape:', y_train.shape)
    print('y_test shape:', y_test.shape)

    model = Sequential()

    # Original
    # model.add(Embedding(vocab_size, embedding_size, input_length=vocab_size))
    # #model.add(Dropout(0.2))
    # model.add(Flatten())
    # model.add(Dense(512, input_shape=(vocab_size,)))
    # model.add(Activation('relu'))
    # # model.add(Dense(64, input_shape=(vocab_size,)))
    # # model.add(Activation('relu'))
    # model.add(Dense(label_count, input_dim=(vocab_size,)))
    # model.add(Activation('softmax'))

    model.add(Embedding(vocab_size, embedding_size, input_length=vocab_size))
    model.add(Dropout(0.2))
    model.add(Flatten())
    model.add(Dense(512, input_shape=(vocab_size,)))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(label_count, input_dim=(vocab_size,)))
    model.add(Activation('sigmoid'))

    model.compile(
        loss='categorical_crossentropy',
        optimizer='adam',
        metrics=['accuracy']
    )
    model.summary()

    history = model.fit(
        x_train, 
        y_train, 
        batch_size=batch_size, 
        epochs=epochs, 
        verbose=1, 
        validation_split=validation_split
    )

    score = model.evaluate(
        x_test,
        y_test,
        batch_size=batch_size, 
        verbose=1
    )
    print('Test accuracy:', score[1])

with flor.Context('patent'):
    main(sample_size=100)

[rjurney]

I don't understand how inspect.getsourcefile can return an OutputStream. How is this possible?

https://docs.python.org/3/library/inspect.html#inspect.getsourcefile

I can't figure out what the fuck an OutputStream is! I don't know how to print(type(tru_path)) in a Jupyter notebook :(

[rlnsanz]

Thanks for bringing this issue to our attention. Are you running this code from a Jupyter Notebook cell? We don't yet support interactive environments (but we're aware of the issue #11 ). If you're running this code from Jupyter, could you try running an ordinary python script instead and see if the issue persists? Thank you!

[rjurney]

Thanks for getting back to me. This is confusing, because so saw a Jupyter demo at the meetup where a student from RISE introduced me to flor. That was the appeal.

On Thu, Dec 6, 2018 at 10:19 AM Rolando Garcia ***@***.***> wrote: Thanks for bringing this issue to our attention. Are you running this code from a Jupyter Notebook cell? We don't yet support interactive environments (but we're aware of the issue #11 <#11> ). If you're running this code from Jupyter, could you try running an ordinary python script instead and see if the issue persists? Thank you! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACkpZKS3V4MwLvxr8Wk_Cn9l69HJmneks5u2V_PgaJpZM4ZFvma> .

-- Russell Jurney @rjurney <http://twitter.com/rjurney> russell.jurney@gmail.com LI <http://linkedin.com/in/russelljurney> FB <http://facebook.com/jurney> datasyndrome.com

[rlnsanz]

Thank you, the demo was for an earlier version of Flor. We've since started doing program analysis (to significantly alleviate the burden of manual specification) so some features that were working in the past need to be re-enabled. We'll make it a priority.

[rjurney]

Cool, thanks!