Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: incompatible shapes #37

Closed
VolkerH opened this issue Apr 16, 2019 · 4 comments
Closed

bug: incompatible shapes #37

VolkerH opened this issue Apr 16, 2019 · 4 comments
Labels
bug Something isn't working

Comments

@VolkerH
Copy link
Owner

VolkerH commented Apr 16, 2019 

==========================

Processor for experiment folder:
Matt_Eramo/20190301_dlp2_Matt_image_tests_v1
Output folder:
output_mat

Processing options:
==================:
skip existing output files: False
Perform max intensity projections: False
max intensity projection output: montage
Perform deskew: False
Rotate to coverslip: False
Perform deconvolution: True
Devonvolution backend: flowdec
Number of iterations for deconvolution: 20
Perform deskew after deconvolution: False
Rotate to coverslip after deconvolution: True
Background subtraction value for camera: 100
processing stack number 5
/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/process_llsm_experiment.py:385: UserWarning: Fix write_func stuff to include compression and units
  warnings.warn("Fix write_func stuff to include compression and units")
/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/process_llsm_experiment.py:463: UserWarning: more than one PSF found. Taking first one
  warnings.warn(f"more than one PSF found. Taking first one")
  0%|                                                                                                     | 0/100 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [105,840,625] vs. [105,840,624]
	 [[{{node while/truediv}} = RealDiv[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](while/zeros_like/Enter, while/irfft3d)]]
	 [[{{node result/_149}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_591_result", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vhil0002/anaconda3/envs/newllsm/bin/lls_dd", line 11, in <module>
    load_entry_point('lls-dd==2019.3a1', 'console_scripts', 'lls_dd')()
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/cmdline.py", line 156, in process
    ep.process_stack_subfolder(processcmd.ef.stacks[int(number)])
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/process_llsm_experiment.py", line 517, in process_stack_subfolder
    deconv_functions[wavelength],
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/process_llsm_experiment.py", line 341, in process_file
    deconv_raw = deconv_func(vol_raw)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/deconvolution.py", line 62, in deconv_volume
    result = deconvolver.run(aq, niter=n_iter, session_config=config)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 216, in run
    res = self._run(acquisition, input_kwargs, session_config=session_config)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 83, in _run
    res = sess.run(self.graph.outputs, feed_dict={**data_dict, **args_dict})
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [105,840,625] vs. [105,840,624]
	 [[node while/truediv (defined at /home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py:301)  = RealDiv[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](while/zeros_like/Enter, while/irfft3d)]]
	 [[{{node result/_149}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_591_result", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'while/truediv', defined at:
  File "/home/vhil0002/anaconda3/envs/newllsm/bin/lls_dd", line 11, in <module>
    load_entry_point('lls-dd==2019.3a1', 'console_scripts', 'lls_dd')()
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/cmdline.py", line 156, in process
    ep.process_stack_subfolder(processcmd.ef.stacks[int(number)])
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/process_llsm_experiment.py", line 495, in process_stack_subfolder
    pad_mode=self.flowdec_pad_mode, pad_min=pad_min, real_domain_fft=True
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/lls_dd-2019.3a1-py3.6.egg/lls_dd/deconvolution.py", line 19, in init_rl_deconvolver
    n_dims=3, start_mode="input", **kwargs
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 62, in initialize
    self.graph = self._get_tf_graph()
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 58, in _get_tf_graph
    inputs, outputs = self._build_tf_graph()
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 316, in _build_tf_graph
    result = tf.while_loop(cond, body, [1, decon], parallel_iterations=1)[1]
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
    return_same_structure)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py", line 301, in body
    blur1 = tf.where(conv1 < self.epsilon, tf.zeros_like(datat), datat / conv1, name='blur1')
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 866, in binary_op_wrapper
    return func(x, y, name=name)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 974, in _truediv_python3
    return gen_math_ops.real_div(x, y, name=name)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5989, in real_div
    "RealDiv", x=x, y=y, name=name)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Incompatible shapes: [105,840,625] vs. [105,840,624]
	 [[node while/truediv (defined at /home/vhil0002/anaconda3/envs/newllsm/lib/python3.6/site-packages/flowdec-1.0.7-py3.6.egg/flowdec/restoration.py:301)  = RealDiv[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](while/zeros_like/Enter, while/irfft3d)]]
	 [[{{node result/_149}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_591_result", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
@VolkerH VolkerH added the bug Something isn't working label Apr 16, 2019
@VolkerH
Copy link
Owner Author

VolkerH commented Apr 16, 2019

check for corrupt input file first

@VolkerH
Copy link
Owner Author

VolkerH commented Apr 16, 2019

this appeared after initializing flowdec with real_domain_fft = True option and went away after I set this option to False again. Happens for padding mode 2367 and None. Haven't tested Log2 yet.
Looks like it might be a bug in flowdec.

@VolkerH
Copy link
Owner Author

VolkerH commented Apr 16, 2019

did not appear with padding mode log2 and real_domain_fft

could real_domain_fft have different requirements for input shapes ?

@VolkerH VolkerH mentioned this issue Apr 17, 2019
Open
@VolkerH
Copy link
Owner Author

VolkerH commented Apr 17, 2019

confirmed that shapes for real_domain_fft must be even.
This should be handled in flowdec hammerlab/flowdec#21

closing here

@VolkerH VolkerH closed this as completed Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@VolkerH