Added a function for extracting decoding results to python

[halcy]

I am unsure if I am just missing something, but I could not find a simple method for doing a forward pass on some data and then extracting the results back to python. This is a function that does this, though I am not sure that I am entirely happy with how it returns data (as a list of lists of outputs of batches).

(If I did indeed miss the easy way to do this, my suggestion would be that one of the examples should do just that - i.e. say decode some MNIST test data and display prediction vs reference)

[sjoerdvansteenkiste]

The trainer supports a method called 'evaluate' which should do exactly that. I do agree that it may be a good idea to include this functionality in one of the examples.

Another, more involved way, of doing just a forward pass would be to use the provide_external_data method of the network, followed by calling forward_pass. The outputs (layer activations) can be then obtained by querying the network buffer.

[halcy]

Ah, I see - the results are returned as part of the Log object, I guess. Thank you! I'll rewrite my code and, if I have a moment later, add something to the mnist example.

[sjoerdvansteenkiste]

Perhaps also have a look at the evaluate method in tools.py file

[flukeskywalker]

@halcy: It seems that what you'd like is what the extract_and_save tool does, but returns the results instead of saving them to disk.

If you have enough memory to do forward pass on the entire data at once, this is straightforward to do by doing net.provide_external_data(my_data) followed by `net.forward_pass().my_data``is simply a dictionary of named input data items, then you can``get`` any feature from the network.

It'd be usually better to run the network on batches of data and accumulate the results. This might still require too much memory, which is why extract_and_save stores the data to disk. I believe it does make sense to get the extracted features as numpy arrays instead (at the risk of running out of memory), so we can have an extract tool for this. However, I think it should work like the extract_and_save tool, and return a dictionary of data, just like what you'd get if you read the HDF5 file generated by extract_and_save.

[halcy]

On second thought: I either seem to be unable to figure out how to use the the evaluate function to gather output data from the forward pass. Is it actually possible to have the data stored to what the Trainers evaluate function returns?

@flukeskywalker I completely agreee, the commit I had added is essentially that, it just does not return the data in a particularly nice format (but it does have the advantage of cutting time series to the right size for evaluation). It was / is a hack that I wrote because I needed to perform frame-by-frame regression of a few time series and then further process the regression output in python. I'd be very grateful if an 'extract' tool to perform something like this that is less hacky would be added.

[flukeskywalker]

Re: using evaluate(). This will not return extracted features/outputs, but will evaluate scores according to the list of Scorers that you provide (Accuracy, Hamming and MeanSquaredError are available), and return a dictionary of aggregated (over the dataset) scores.

Re: extract(). It's likely that we will add this soon, and you're also welcome to submit a PR. Essentially, it will allocate large Numpy arrays using the number of sequencs from iter.data_shapes and other dim sizes from the shape of the resulting buffers after the first forward pass. Then it will write data to correct parts of these arrays.