Input shape in ensemble_t0_DNNEstimator1

[weizhenzhao]

Hi dear

I met an error when combine the DNNEstimator
it looks like following things:

` return _get_logits()
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\feature_column\feature_column.py", line 205, in _get_logits
tensor, shape=(batch_size, num_elements))
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 7760, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op
op_def=op_def)
File "C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 196608 values, but the requested shape has 256
[[node adanet/iteration_0/ensemble_t0_DNNEstimator1/weighted_subnetwork_0/subnetwork/dnn/input_from_feature_columns/input_layer/a/Reshape (defined at C:\Users\weizhen.zhao\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\estimator\python\estimator\dnn.py:161) = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](IteratorGetNext/_145, adanet/iteration_0/ensemble_t0_DNNEstimator1/weighted_subnetwork_0/subnetwork/dnn/input_from_feature_columns/input_layer/a/Reshape/shape)]]
[[{{node adanet/iteration_0/ensemble_t0_DNNEstimator1/weighted_subnetwork_0/subnetwork/dnn/zero_fraction_1/Mean/_195}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_227_a...ion_1/Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Process finished with exit code 1`

and my code is as the following:
`

def train(self, training_data, cfg=None, **kwargs):

    intent_dict = self._create_intent_dict(training_data)

    if len(intent_dict) < 2:
        logger.error("Can not train an intent classifier. "
                     "Need at least 2 different classes. "
                     "Skipping training of intent classifier.")
        return

    self.inv_intent_dict = {v: k for k, v in intent_dict.items()}
    self.encoded_all_intents = self._create_encoded_intents(intent_dict)

    X, Y, intents_for_X = self._prepare_data_for_training(training_data, intent_dict)

    num_classes = len(intent_dict)

    head = tf.contrib.estimator.multi_class_head(n_classes=num_classes)

    feature_names = ['a']
    feature_columns = [tf.feature_column.numeric_column(k) for k in feature_names]

    def input_fn():
        def gen1(a, b):
            return {'a': a}, b

        dataset = tf.data.Dataset.from_tensor_slices((X, intents_for_X)).map(gen1)
        dataset = dataset.batch(self.batch_size)
        iterator = dataset.make_one_shot_iterator()
        data, labels = iterator.get_next()
        return data, labels

    estimator = adanet.AutoEnsembleEstimator(head=head,
                                             candidate_pool={
                                                     tf.contrib.estimator.LinearEstimator(
                                                         head = head,
                                                         feature_columns=feature_columns,
                                                         optimizer='Ftrl'
                                                     ),
                                                     tf.contrib.estimator.DNNEstimator(
                                                         head=head,
                                                         feature_columns=feature_columns,
                                                         optimizer='Adagrad',
                                                         hidden_units=[768, 256])},
                                             max_iteration_steps=50
                                             )

    estimator.train(input_fn=lambda: input_fn(), steps=self.epochs)`

when I debug it in the pycharm
the X is ndarray with shape of [2145,767]

intents_for_X is ndarray with shape of [2145,1]

and there is 93 classes in total

Can you give me some advice to solve this problem?

Thanks
Weizhen

[cweill]

You need to explicitly define your feature shapes in tf.feature_column.numeric_column(k, shape=...).

[weizhenzhao]

@cweill
hi dear can you give a little detail instruction , what exactly shapes should I set ?

thanks
weizhen

[cweill]

I'm guessing shape=[2145,1].

[weizhenzhao]

when I changing the shape to
feature_columns = [tf.feature_column.numeric_column(key=k,shape=[1 , 768]) for k in feature_names]
I think the first problem has gone .
But I met another problem , and it looks like

InternalError (see above for traceback): Blas GEMM launch failed : a.shape=(256, 768), b.shape=(768, 93), m=256, n=93, k=768
[[node adanet/iteration_0/ensemble_t0_LinearEstimator0/weighted_subnetwork_0/subnetwork/linear/linear_model/linear_model/a_in/weighted_sum (defined at C:\Users\weizhen\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\contrib\estimator\python\estimator\linear.py:136) = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](IteratorGetNext/_137, adanet/iteration_0/ensemble_t0_LinearEstimator0/weighted_subnetwork_0/subnetwork/linear/linear_model/a_in/weights)]]
[[{{node adanet/iteration_0/ensemble_t0_DNNEstimator1/head_2/labels/assert_equal/All/_243}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_323_a..._equal/All", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Process finished with exit code 1

and when I search that in the google , it writes like that

tensorflow/tensorflow#11812

can you give me some advice ?

thanks
weizhen

[cweill]

I'd recommend first getting this to work with estimator = tf.contrib.estimator.DNNEstimator() before trying AdaNet. This appears to be more related to raw TensorFlow or feature columns.

[weizhenzhao]

I think I've fixed that issue , it because of the per_gpu_memory_fraction config and I should config this parameter to not using all the gpu memory at first