Added feature for serialized session options

[AfaqSabirIBEX]

Firstly thank you for creating this library, it has helped us a lot in development.

Recently we have moved to using Tensorflow on the GPU but to get this working properly along side other software we need to be able to modify TF session options, so that we can control the amount of memory TF uses. This PR allows the user to input options if needed when the model is loaded.

[ljn917]

Hi, it seems TF_CreateConfig() is a better solution for your need.

[serizba]

Hi @AfaqSabirIBEX

I think is a great idea and it can be useful. In #3 I answered how to use TF_SetConfig, but I did not add the feature to the main code, I don't know why.

I would suggest you to add a bit of documentation (an example, or maybe in the readme) on how to fill the config_options vector, as it can be a bit tricky. It would be great if you can add it 😊

Thanks!

[AfaqSabirIBEX]

Hi @AfaqSabirIBEX

I think is a great idea and it can be useful. In #3 I answered how to use TF_SetConfig, but I did not add the feature to the main code, I don't know why.

I would suggest you to add a bit of documentation (an example, or maybe in the readme) on how to fill the config_options vector, as it can be a bit tricky. It would be great if you can add it 😊

Thanks!

Hi @serizba

Thanks for the feedback. I have updated the load_model example and added some comments to main.cpp. I also added a script to show how to generate the serialized config options and added a single comment to the model.h to point user in the correct direction.

I figured updating the load_model example is probably the cleanest thing to do and does not over complicate this change. Please let me know if this is correct or any more changes are required, as we are eager to start using this feature in our code.

Thanks again

[AfaqSabirIBEX]

Hi, it seems TF_CreateConfig() is a better solution for your need.

Hi @ljn917,

There are a few reason we did not go down the route of using that function:

• I wanted to keep this change as simple as possible and not include more dependencies.
• Using that function still does not give the desired result as you cannot set the memory fraction. We want to limit TF GPU memory but allow for growth as well. This allows us to keep TF alive whilst running other code and prevents TF from doing unnecessary memory allocations.
• We are not able to use anything experimental in our software.

[ljn917]

Thanks, good to know.

…

On Fri, May 29, 2020, 10:45 AM Afaq Sabir ***@***.***> wrote: Hi, it seems TF_CreateConfig() <https://github.com/tensorflow/tensorflow/blob/ea8e87c8e945c5424b130d66a4fbf5fca7e5c9cd/tensorflow/c/c_api_experimental.h#L93> is a better solution for your need. Hi @ljn917 <https://github.com/ljn917>, There are a few reason we did not go down the route of using that function: - I wanted to keep this change as simple as possible and not include more dependencies. - Using that function still does not give the desired result as you cannot set the memory fraction. We want to limit TF GPU memory but allow for growth as well. This allows us to keep TF alive whilst running other code and prevents TF from doing unnecessary memory allocations. - We are not able to use anything experimental in our software. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALDTHRRCMYUAXYRYBQPNFTRT7C7VANCNFSM4NNFPXPQ> .

[serizba]

Hi @AfaqSabirIBEX
I think is a great idea and it can be useful. In #3 I answered how to use TF_SetConfig, but I did not add the feature to the main code, I don't know why.
I would suggest you to add a bit of documentation (an example, or maybe in the readme) on how to fill the config_options vector, as it can be a bit tricky. It would be great if you can add it blush
Thanks!

Hi @serizba

Thanks for the feedback. I have updated the load_model example and added some comments to main.cpp. I also added a script to show how to generate the serialized config options and added a single comment to the model.h to point user in the correct direction.

I figured updating the load_model example is probably the cleanest thing to do and does not over complicate this change. Please let me know if this is correct or any more changes are required, as we are eager to start using this feature in our code.

Thanks again

Thanks @AfaqSabirIBEX ! That's perfect. Before merging it I just changed the script to create the serialized options to output a C++ like vector so it is easy to copy-paste directly to the cpp file.

Thanks!