Error exception when load keras model deep learning by celery

[trangtv57]

Checklist

• I have included the output of celery -A proj report in the issue.
  (if you are not able to do this, then at least specify the Celery
  version affected).
• I have verified that the issue exists against the master branch of Celery.

Steps to reproduce

I want to load model i have trained it before in keras by celery. But in tensorflow( backend of keras) its just accept for load only 1 model on same session( can say is same thread). So problem here is, how can i saparate each thread in celery (can just how to make each thread have it's own each object model keras) to can run.

Expected behavior

Can run api of deep learning( keras) with celery

Actual behavior

[niteeshm]

@trangtv57 Can you post some logs or explain what error is there if any ? Load the model in our main celery app and use it everywhere else required. All workers will have separate object.s

[trangtv57]

Sorry if my question have some confuse. I have object graph: is tensorflow.get_default_graph(). this object need share separate on all threads like this: In Celery I run with mode debugging but nothing so error ( note: I have same problem (with nothing error show, its just stuck when running) when I run load my model on multi process, so I can understand why celery don't show any thing and just stuck).
In my sample when I load model in multi thread, my model can run:
`
def main_worker_dga(queue_domain, n_threads=5):

l_threads = []
model = load_model(path_file_model, path_file_weight)
graph = tf.get_default_graph()
print(id(graph))
for i in range(n_threads):
    o_worker_abc = WorkerABC(queue_domain, model, graph)
    o_worker_abc.start()
    l_threads.append(o_worker_abc)

for e_thread in l_threads:
    e_thread.join()

`

AND File worker thread:

`
class WorkerABC(threading.Thread):
dga_o = None
queue_dm = None
graph = None

def __init__(self, queue_dm, model_dga, graph):
    threading.Thread.__init__(self)
    self.queue_dm = queue_dm
    self.model = model_dga
    self.graph = graph

def run(self):
    self.abc_o = ABC()
    self.abc_o.set_model(self.model)

    # global graph
    # with graph[0].as_default():
    with self.graph.as_default():
        while True:
            if not self.queue_dm.empty():
                dm_next = self.queue_dm.get()
                self.abc_o.set_omain(dm_next)
                print(self.abc_o.predict_probability_domain())
            else:
                break

`

[godelstheory]

The issue is get_session from the TensorFlow library: this will hang across processes. It is not recommended to utilize multiprocess (see the last comment of tensorflow/tensorflow#8220).

A workaround is that all imports to Tensorflow (including keras) occur only in the child, spawned processes. This has allowed me to effectively load and utilize serialized TensorFlow models across child process using keras. see tensorflow/tensorflow#5448

[auvipy]

closing as not a bug, if any documentation or code fixes are suggested plz send a pr