Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3D format to .tfrecords #14

Closed
dperpen opened this issue May 23, 2019 · 13 comments
Closed

3D format to .tfrecords #14

dperpen opened this issue May 23, 2019 · 13 comments
Assignees

Comments

@dperpen
Copy link

@dperpen dperpen commented May 23, 2019

It'd be great to have a conversion tool to go from any of the usual 3d formats(.fbx, .ply, .obj...) to .tfrecords

@julienvalentin julienvalentin self-assigned this May 24, 2019
@julienvalentin
Copy link
Collaborator

@julienvalentin julienvalentin commented May 24, 2019

Hi,

Would Trimesh (https://github.com/mikedh/trimesh) suit your needs?
I am also happy to start a discussion on this topic if you can share your exact needs.

Best.

@dperpen
Copy link
Author

@dperpen dperpen commented May 24, 2019

Hi Julien,
Thanks for your reply. I am trying to feed some of our meshes (dancing people) to your mesh segmentation example to see how it goes. We can have our mesh in any 3d format but I don't know how to convert that to .tfrecords. (I believe the one the example uses is: 'data/Dancer_test_sequence.tfrecords', right?)

@avneeshsud
Copy link

@avneeshsud avneeshsud commented May 25, 2019

Hi,

_parse_tfex_proto and _parse_mesh_data examples in mesh_segmentation_io.py (https://github.com/tensorflow/graphics/blob/master/tensorflow_graphics/notebooks/mesh_segmentation_dataio.py#L176) show how to unpack dense vertex coordinate and triangle indices tensors from tf.Example protos. The procedure for packing mesh data is the reverse: convert custom mesh data into tensors, serialize to a string feature and store as TFExamples in TFRecords. Please refer to TFRecords tutorial (https://www.tensorflow.org/tutorials/load_data/tf_records) on creating TFRecords from TFExamples

Best

@dperpen
Copy link
Author

@dperpen dperpen commented May 28, 2019

Thanks, I will try that.

@julienvalentin
Copy link
Collaborator

@julienvalentin julienvalentin commented May 30, 2019

Let us know if this solved your problem :)

@simofoti
Copy link

@simofoti simofoti commented Jun 3, 2019

Hi,

as @dperpen I would be interested in using the tensorflow graphics library with data in some usual 3D formats (obj in my case).
I am trying to implement the converter, but I think I am creating the tfrecord in the wrong way. When I try to load the mesh data from the tfrecord (following the same procedure used in your example I have the following error messages:

OP_REQUIRES failed at parse_tensor_op.cc:52 : Invalid argument: Cannot parse tensor from proto
OP_REQUIRES failed at iterator_ops.cc:988 : Invalid argument: Cannot parse tensor from proto
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot parse tensor from proto: 
	 [[{{node ParseTensor}}]] [Op:IteratorGetNextSync]

If anyone can spot the error/errors
Below he/she can find the code I used to generate the tfrecord:

import numpy as np
import tensorflow as tf

def _parse_obj(file_path):
    vertices = []
    triangles = []
    with open(file_path) as fp:
        for line in fp:
            parsed = _parse_vertex_or_triangle(line)
            if parsed[0] == 'v':
                vertices.append(parsed[1])
            if parsed[0] == 'f':
                triangles.append(parsed[1])
    num_vertices = len(vertices)
    num_triangles = len(triangles)

    vertices = tf.constant(np.vstack(vertices))
    triangles = tf.constant(np.vstack(triangles))
    return num_vertices, num_triangles, vertices, triangles


def _parse_vertex_or_triangle(line):
    elem_type = None
    data = np.zeros(3)
    if not line or not line == '\n':  # check if line is empty
        separated = line.split()
        if separated[0] == 'v':
            elem_type = 'v'
            data[0] = float(separated[1])
            data[1] = float(separated[2])
            data[2] = float(separated[3])
        if separated[0] == 'f':
            elem_type = 'f'
            data[0] = int(separated[1].split('/')[0])
            data[1] = int(separated[2].split('/')[0])
            data[2] = int(separated[3].split('/')[0])
    return elem_type, data


def _tensor_feature(values, dtype):
    values = tf.dtypes.cast(values, dtype)
    serialised_values = tf.io.serialize_tensor(values)
    return tf.train.Feature(
        bytes_list=tf.train.BytesList(value=[serialised_values.numpy()]))


def _int64_feature(value):
    """Returns an int64_list from a bool / enum / int / uint."""
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))


def write_tfrecord_from_obj(obj_path, out_path):
    num_vertices, num_triangles, vertices, triangles = _parse_obj(obj_path)
    feature = {
                'num_vertices': _int64_feature(num_vertices),
                'num_triangles': _int64_feature(num_triangles),
                'vertices': _tensor_feature(vertices, tf.float32),
                'triangles': _tensor_feature(triangles, tf.int32)
            }

    example_proto = tf.train.Example(
        features=tf.train.Features(feature=feature))
    serialized_proto = example_proto.SerializeToString()
    with tf.io.TFRecordWriter(out_path) as writer:
        writer.write(serialized_proto)


write_tfrecord_from_obj(obj_path_in, tfrecord_path_out)

Thanks in advance for any help.

@simofoti
Copy link

@simofoti simofoti commented Jun 6, 2019

Never mind, I figured it out. I was doing two mistakes:

  1. even thought obj files don't come with labels I had to initialise a label tensor (and this fixed the above-mentioned problem)
  2. face/triangle indices in obj files starts from one, while in the library they start from zero

Let me know if you want me to share my code somewhere or to contribute somehow.

@jajakob
Copy link

@jajakob jajakob commented Jun 20, 2019

Hi guys,
thanks a lot for providing TF graphics.

I also want to apply the mesh segmentation on custom meshes and I was wondering:

What is the maximal number of vertices the input mesh can have?
Because I guess my meshes are currently too large, and according to the FeastNet paper, their experiments have been run with the FAUST dataset where each mesh consists of 6,890 vertices

Best,
Jakob

@avneesh-sud avneesh-sud self-assigned this Jun 21, 2019
@avneesh-sud
Copy link
Collaborator

@avneesh-sud avneesh-sud commented Jun 21, 2019

Hi jajakob - the size of meshes will be dependent on available memory and batch size. One can trade off larger meshes against smaller batch sizes.

@jajakob
Copy link

@jajakob jajakob commented Jun 25, 2019

Hi avneesh-sud,
thanks a lot for the answer! I thought that, anyway I try to decimate the meshes, thanks!

@julienvalentin
Copy link
Collaborator

@julienvalentin julienvalentin commented Jun 25, 2019

Is it fair to assume that all is resolved on this thread? :)

@Anvom
Copy link

@Anvom Anvom commented Dec 12, 2019

Never mind, I figured it out. I was doing two mistakes:

  1. even thought obj files don't come with labels I had to initialise a label tensor (and this fixed the above-mentioned problem)
  2. face/triangle indices in obj files starts from one, while in the library they start from zero

Let me know if you want me to share my code somewhere or to contribute somehow.

It could be very helpful for me to have a look at your code. Is there a way to share your code?

@simofoti
Copy link

@simofoti simofoti commented Dec 12, 2019

Hi @Anvom,

I haven't used that code since then, but it should still run (hopefully). Since there is not much, I will just paste here the latest version I have.

import numpy as np
import os
import tensorflow as tf
# from tensorflow_graphics.notebooks import mesh_segmentation_dataio as dataio


def parse_obj(file_path):
    vertices = []
    triangles = []
    with open(file_path) as fp:
        for line in fp:
            parsed = _parse_vertex_or_triangle(line)
            if parsed[0] == 'v':
                vertices.append(parsed[1])
            if parsed[0] == 'f':
                triangles.append(parsed[1])
    num_vertices = len(vertices)
    num_triangles = len(triangles)

    vertices = tf.constant(np.vstack(vertices))
    triangles = tf.constant(np.vstack(triangles))
    return num_vertices, num_triangles, vertices, triangles


def _parse_vertex_or_triangle(line):
    elem_type = None
    data = np.zeros(3)
    if not line or not line == '\n':  # check if line is empty
        separated = line.split()
        if separated[0] == 'v':
            elem_type = 'v'
            data[0] = float(separated[1])
            data[1] = float(separated[2])
            data[2] = float(separated[3])
        if separated[0] == 'f':
            elem_type = 'f'
            data[0] = int(separated[1].split('/')[0]) - 1
            data[1] = int(separated[2].split('/')[0]) - 1
            data[2] = int(separated[3].split('/')[0]) - 1
    return elem_type, data


def _list_string_feature(values):
    """Returns a bytes_list from a list of strings"""
    return tf.train.Feature(bytes_list=tf.train.BytesList(value=values))


def _tensor_feature(values, dtype):
    values = tf.dtypes.cast(values, dtype)
    serialised_values = tf.io.serialize_tensor(values)
    return tf.train.Feature(
        bytes_list=tf.train.BytesList(value=[serialised_values.numpy()]))


def _int64_feature(value):
    """Returns an int64_list from a bool / enum / int / uint."""
    return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))


def write_tfrecord_from_obj(obj_path, out_path):
    num_vertices, num_triangles, vertices, triangles = parse_obj(obj_path)
    labels = tf.ones(num_vertices, dtype=tf.int32)
    feature = {
                'num_vertices': _int64_feature(num_vertices),
                'num_triangles': _int64_feature(num_triangles),
                'vertices': _tensor_feature(vertices, tf.float32),
                'triangles': _tensor_feature(triangles, tf.int32),
                'labels': _tensor_feature(labels, tf.int32)
              }

    example_proto = tf.train.Example(
        features=tf.train.Features(feature=feature))
    serialized_proto = example_proto.SerializeToString()
    with tf.io.TFRecordWriter(out_path) as writer:
        writer.write(serialized_proto)


def batch_writing_tfrecord_from_obj(objs_folder, out_path, name_depth=1):
    # find all objs in a folder
    obj_files_path = []
    for dirpath, _, fnames in os.walk(objs_folder):
        for f in fnames:
            if f.endswith(".obj"):
                obj_files_path.append(os.path.join(dirpath, f))
    for obj_file_path in obj_files_path:
        tfrecord_file_name = obj_file_path.split('/')[-name_depth]
        if name_depth == 1:
            tfrecord_file_name = tfrecord_file_name.split('.')[0]
        tfrecord_file_path = out_path + tfrecord_file_name + '.tfrecord'
        write_tfrecord_from_obj(obj_file_path, tfrecord_file_path)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants