Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3D format to .tfrecords #14

Open
dperpen opened this issue May 23, 2019 · 7 comments

Comments

Projects
None yet
4 participants
@dperpen
Copy link

commented May 23, 2019

It'd be great to have a conversion tool to go from any of the usual 3d formats(.fbx, .ply, .obj...) to .tfrecords

@julienvalentin julienvalentin self-assigned this May 24, 2019

@julienvalentin

This comment has been minimized.

Copy link
Collaborator

commented May 24, 2019

Hi,

Would Trimesh (https://github.com/mikedh/trimesh) suit your needs?
I am also happy to start a discussion on this topic if you can share your exact needs.

Best.

@dperpen

This comment has been minimized.

Copy link
Author

commented May 24, 2019

Hi Julien,
Thanks for your reply. I am trying to feed some of our meshes (dancing people) to your mesh segmentation example to see how it goes. We can have our mesh in any 3d format but I don't know how to convert that to .tfrecords. (I believe the one the example uses is: 'data/Dancer_test_sequence.tfrecords', right?)

@avneeshsud

This comment has been minimized.

Copy link

commented May 25, 2019

Hi,

_parse_tfex_proto and _parse_mesh_data examples in mesh_segmentation_io.py (https://github.com/tensorflow/graphics/blob/master/tensorflow_graphics/notebooks/mesh_segmentation_dataio.py#L176) show how to unpack dense vertex coordinate and triangle indices tensors from tf.Example protos. The procedure for packing mesh data is the reverse: convert custom mesh data into tensors, serialize to a string feature and store as TFExamples in TFRecords. Please refer to TFRecords tutorial (https://www.tensorflow.org/tutorials/load_data/tf_records) on creating TFRecords from TFExamples

Best

@dperpen

This comment has been minimized.

Copy link
Author

commented May 28, 2019

Thanks, I will try that.

@julienvalentin

This comment has been minimized.

Copy link
Collaborator

commented May 30, 2019

Let us know if this solved your problem :)

@simofoti

This comment has been minimized.

Copy link

commented Jun 3, 2019

Hi,

as @dperpen I would be interested in using the tensorflow graphics library with data in some usual 3D formats (obj in my case).
I am trying to implement the converter, but I think I am creating the tfrecord in the wrong way. When I try to load the mesh data from the tfrecord (following the same procedure used in your example I have the following error messages:

OP_REQUIRES failed at parse_tensor_op.cc:52 : Invalid argument: Cannot parse tensor from proto
OP_REQUIRES failed at iterator_ops.cc:988 : Invalid argument: Cannot parse tensor from proto
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot parse tensor from proto: 
	 [[{{node ParseTensor}}]] [Op:IteratorGetNextSync]

If anyone can spot the error/errors
Below he/she can find the code I used to generate the tfrecord:

import numpy as np
import tensorflow as tf

def _parse_obj(file_path):
    vertices = []
    triangles = []
    with open(file_path) as fp:
        for line in fp:
            parsed = _parse_vertex_or_triangle(line)
            if parsed[0] == 'v':
                vertices.append(parsed[1])
            if parsed[0] == 'f':
                triangles.append(parsed[1])
    num_vertices = len(vertices)
    num_triangles = len(triangles)

    vertices = tf.constant(np.vstack(vertices))
    triangles = tf.constant(np.vstack(triangles))
    return num_vertices, num_triangles, vertices, triangles


def _parse_vertex_or_triangle(line):
    elem_type = None
    data = np.zeros(3)
    if not line or not line == '\n':  # check if line is empty
        separated = line.split()
        if separated[0] == 'v':
            elem_type = 'v'
            data[0] = float(separated[1])
            data[1] = float(separated[2])
            data[2] = float(separated[3])
        if separated[0] == 'f':
            elem_type = 'f'
            data[0] = int(separated[1].split('/')[0])
            data[1] = int(separated[2].split('/')[0])
            data[2] = int(separated[3].split('/')[0])
    return elem_type, data


def _tensor_feature(values, dtype):
    values = tf.dtypes.cast(values, dtype)
    serialised_values = tf.io.serialize_tensor(values)
    return tf.train.Feature(
        bytes_list=tf.train.BytesList(value=[serialised_values.numpy()]))


def _int64_feature(value):
    """Returns an int64_list from a bool / enum / int / uint."""
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))


def write_tfrecord_from_obj(obj_path, out_path):
    num_vertices, num_triangles, vertices, triangles = _parse_obj(obj_path)
    feature = {
                'num_vertices': _int64_feature(num_vertices),
                'num_triangles': _int64_feature(num_triangles),
                'vertices': _tensor_feature(vertices, tf.float32),
                'triangles': _tensor_feature(triangles, tf.int32)
            }

    example_proto = tf.train.Example(
        features=tf.train.Features(feature=feature))
    serialized_proto = example_proto.SerializeToString()
    with tf.io.TFRecordWriter(out_path) as writer:
        writer.write(serialized_proto)


write_tfrecord_from_obj(obj_path_in, tfrecord_path_out)

Thanks in advance for any help.

@simofoti

This comment has been minimized.

Copy link

commented Jun 6, 2019

Never mind, I figured it out. I was doing two mistakes:

  1. even thought obj files don't come with labels I had to initialise a label tensor (and this fixed the above-mentioned problem)
  2. face/triangle indices in obj files starts from one, while in the library they start from zero

Let me know if you want me to share my code somewhere or to contribute somehow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.