Saving a multilayer netwrok as JSON

[gokayhuz]

As described in Issue #1023 , using ModelSerializer.writeModel(), dl4j currently supports saving the network config as a JSON file and saving the parameters in binary format. For a larger network, saving everything in a text format like JSON is not space-efficient and will be slow.

However, there are many cases where it would be desirable to create and train the network using dl4j, and then use the trained network's parameters (weights, layer information, etc) in another environment.

For example, I am working on an Expert Advisor on MetaTrader5, which supports a "simplified C++ like language" (MQL5), and which does not support reading binary files from disk to re-create the network.

It would help greatly if the network information (layers, activation functions, and esp. weights) were saved in a text format, so we can manually transfer this information to an external platform (like MQL5)

[treo]

Since you are going to be loading it manually anyway, you might directly write your parameters with Nd4j.writeTxt to a file. In this case they will be in JSON format.

(If you are in a locale that uses a comma (,) as a decimal separator, this is bugged at the moment, see #6537)

[AlexDBlack]

In theory, we could save the net to a text-based format.
However, for your use case, 99% of your coding would be in essentially reproducing DL4J in MQL5 language - something that would be an absurd amount of work. Maybe implementing a very limited subset (dense layers plus a couple of activation functions) might be feasible.

Put another way: I'm not going to allocate resources to implement tools to export to a non-standard text-based format that can only be parsed and used manually. It's just not useful for 99.9% of users and hence not worth our time.

Now, for saving the network components for inference in a text-based format: there's basically 2 things you need.
First, the parameters. A few different options, Nd4j.writeTxt is one.
Second: the configuration. MultiLayerConfiguration/ComputationGraph already have toJson() methods.

You can then read and parse those in your program. Alternatively, perhaps you should look at a way to call Java program (or some sort of model server via REST etc) from your MQL5 program.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.