how can i use "TF_SetTarget or TF_SetConfig" to set the number of threads in c code

[xunkaixin]

Source code / logs

this is my c code:

TF_SessionOptions* sess_opts = TF_NewSessionOptions();
std::string str("intra_op_parallelism_threads = 4");
TF_SetConfig(sess_opts,(void *)str.c_str(),str.size(),status);
if (TF_GetCode(status) != TF_OK) {
	  printf("ERROR: %s\n", TF_Message(status));
}

this is the debug output:

   Successfully imported graph
   ERROR: Unparseable ConfigProto

[drpngx]

Yes, you'll want to say:

const std::string config_text("introa_op_parallelism_threads: 4")

[somewacko]

Is this correct? I just tried this (trying both "intra_op_parallelism_threads: 4" and ""introa_op_parallelism_threads: 4") and neither seemed to work.

[bgraf422]

I'm also having trouble with this when trying to set the gpu memory via the C API. When I run the code below I get the following error message: "Unparseable ConfigProto".

TF_Status* status = TF_NewStatus();
TF_SessionOptions* opts = TF_NewSessionOptions();
std::string str("per_process_gpu_memory_fraction: 0.5");

TF_SetConfig(opts, (void *)str.c_str(),str.size(), status);

TF_Code code = TF_GetCode(status);
if (code != TF_OK) {
std::cout << TF_Message(status) << std::endl;
}

Are there any documentation/examples/comments that show how to set session options via the C API? I've been referencing the c_api_test.cc (link below) which has many examples of how to use the C API but it does not show how to format string for the TF_SetConfig function.

https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/c/c_api_test.cc

[somewacko]

@bgraf422 I believe TF_SetConfig wants you to provide the config as a serialized protobuf string. You can see what this looks like if you open a Python interpreter and create the config yourself:

>>> import tensorflow as tf
>>> gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.5)
>>> config = tf.ConfigProto(gpu_options=gpu_options)
>>> serialized = config.SerializeToString()
>>> list(map(hex, serialized))
['0x32', '0x9', '0x9', '0x0', '0x0', '0x0', '0x0', '0x0', '0x0', '0xe0', '0x3f']

And you should use this serialized representation in the C code instead, like:

uint8_t config[11] = {0x32, 0x09, 0x09, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, 0x3f};
TF_SetConfig(opts, (void*)config, 11, status);

(Code off the top of my head, may not work but you can get the gist.)

Obviously this is not ideal and you should construct/serialize the Config object instead in your C code, but this seems to be the only solution if you want to rely solely on what's exposed by TensorFlow's C API.

@drpngx do you have any thoughts on this? This seems like a hacky way to set this.

[somewacko]

Also: I've tried this in an Android NDK app to set the number of threads, and while it makes a warning go away I've observed that this had no effect. For me setting the number of threads to 1 or 8 through the C API had no effect on benchmark time, while setting threads using the equivalent code in C++ did. I'd be interested if this works for you or not.

[bgraf422]

Yup that did the trick, thanks for the suggestion! I confirmed that only half of the gpu memory was being used via the nvidia-smi CLI tool.

I agree it would be best to construct the config object but I don't see any way of doing that without being dependent on the C++ API. This will work for me for now, thanks for the quick response!

[somewacko]

Ah, I'm guessing you're on Linux then? Maybe I'm just experiencing an Android problem.

But yeah this would require some sort of wrapper around config in the C API, I'm not sure how much the TF devs want that though (it would definitely add more to the API surface).

[bgraf422]

Yup I'm on Ubuntu 16.04. Makes sense that they don't have all the functionality in the C API since it is mostly meant to be used for building bindings to other languages. I hope they eventually add more features to the C API as it is pretty convenient for deployment. It's a lot easier to just pull down the C API binaries than compiling the source code into a C++ shared library.

[lc82111]

@zo7 I have tried your memory-limitation trick and it works! Thanks!

[lc82111]

c++ code:

 57     TF_SessionOptions* sess_opts = TF_NewSessionOptions();
 58 
 59     uint8_t config[11] = {0x32, 0x09, 0x09, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe0, 0x3f};
 60     TF_SetConfig(sess_opts, (void*)config, 11, s);
 61     assert(TF_GetCode(s) == TF_OK);
 62 
 63     TF_Session* session = TF_NewSession(graph, sess_opts, s);
 64     assert(TF_GetCode(s) == TF_OK);

python2.7 code:

import tensorflow as tf
  
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.5)
config = tf.ConfigProto(gpu_options=gpu_options)
serialized = config.SerializeToString()
  
result = ["0x"+c.encode('hex') for c in serialized]
  
print(result) 

[drpngx]

I need. It's a binary proto. You want to use TextFormat to parse from text, rather than editing the binary strings.

[BerndSchmitt]

no need for hex-encoding, just use
[int(i) for i in serialized]
replace the [] with {}
uint8_t config[] = { ....};
works like a charm

[hoavt-54]

Hey guys, I'm late to the party but is this written anywhere in the API?
I tried to set intra_op_parallelism_threads and inter_op_parallelism_threads but it does not seem to work in C. Here is my code:

config = tf.ConfigProto()
config.intra_op_parallelism_threads = 1
config.inter_op_parallelism_threads = 1
config.use_per_session_threads=False
serialized = config.SerializeToString()
result = ["0x"+c.encode('hex') for c in serialized]

Code in C:

uint8_t config[4] = {0x10, 0x01, 0x28, 0x01};
    TF_SessionOptions* sess_opts = TF_NewSessionOptions();
    TF_SetConfig(sess_opts,(void *)config,4,s);
    if (TF_GetCode(s) != TF_OK) {
      printf("ERROR: %s\n", TF_Message(s));
    }
    TF_Session* session = TF_NewSession(graph, sess_opts, s);
    assert(TF_GetCode(s) == TF_OK);

[mborgerding]

A slightly more advanced version of the code by @hoavt-54 ...

    // reverse engineered the TF_SetConfig protocol from python code like:
    // >> config = tf.ConfigProto();config.intra_op_parallelism_threads=7;config.SerializeToString()
    // '\x10\x07'
    uint8_t intra_op_parallelism_threads = 1;
    uint8_t inter_op_parallelism_threads = 1;
    uint8_t buf[]={0x10,intra_op_parallelism_threads,0x28,inter_op_parallelism_threads};
    TF_SetConfig(sess_opts, buf,sizeof(buf),status);

By default, TF assumes it can take over all CPUs. Unfortunately, its threading is nowhere near linear. On a 16 core machine, processing a model single-threaded changed the /bin/time output from (real,user,sys) = ( 1.61, 20.0, 6.46 ) to (8.36, 8.26, 0.07)
i.e single-threaded took 5x longer wall(real) time, but used less than a third total CPU time (user+sys) as the default. One should seek ways to multiprocess at higher abstractions than this.

[apivovarov]

To get proto parameter for TF_SetConfig we can use tensorflow/core/protobuf/config.pb.h and tensorflow::ConfigProto class
The implementation is included to libtensorflow.so

Code example https://github.com/apivovarov/TF_C_API/blob/master/config.cc

build command:

g++ -std=c++11 -o config.o -c -Itensorflow/bazel-tensorflow/external/com_google_protobuf/src -Itensorflow/bazel-bin -Ilibtensorflow/include config.cc
g++ -std=c++11 -o config config.o -Llibtensorflow/lib -ltensorflow -ltensorflow_framework 
LD_LIBRARY_PATH=libtensorflow/lib: ./config
0x10,0x2,0x28,0x3,0x32,0xb,0x9,0x9a,0x99,0x99,0x99,0x99,0x99,0xb9,0x3f,0x20,0x1

TF_SetConfig API:
https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/c/c_api.h#L147

[fisakhan]

How can I restrict the tensorflow c api to use only and only one core of the cpu?

[fisakhan]

How can I restrict the tensorflow c api to use only and only one core of the cpu?