Skip to content
pybullet grasping with time contrastive network embeddings
Branch: master
Clone or download
Latest commit 9e3f1d4 Jun 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore wip rolling some things back to get working again Jun 2, 2019
LICENSE
README.md update readme May 31, 2019
camera.py export joint positions to {--joint-info-dir}/rNNN/cNNN.tsv Jun 3, 2019
data.py make negative mining strategy for triplet selection configurable Jun 2, 2019
debug_calc_embedding_near_neighbours.sh blog post wip Jun 12, 2019
debug_compare_arm_poses.py exploreing some different approaches to eval Jun 5, 2019
debug_embedding_diversity.py wip rolling some things back to get working again Jun 2, 2019
debug_embedding_near_neighbours.py run01 works! May 25, 2019
debug_frame.py move all refs to width height to one place May 28, 2019
debug_frame_sequence_cameras.py blog post wip Jun 12, 2019
debug_random_frames.py blog post wip Jun 12, 2019
debug_random_triples.py fix refactor bug Jun 5, 2019
embed_imgs.py wip rolling some things back to get working again Jun 2, 2019
gen_runs.py
generate_toy_rgb_data.py generate toy data across 2 runs for consistent testing or N camera, R… May 29, 2019
insert_joint_tsvs_into_redis.py onight run36 param sweeps Jun 4, 2019
kuka_env.py wip rolling some things back to get working again Jun 2, 2019
latest_model_in.py exploreing some different approaches to eval Jun 5, 2019
model.hdf5 up model for run25 May 31, 2019
model.py blog post wip Jun 12, 2019
near_neighbour_egs.gif update readme May 31, 2019
parse_tf_events_into_tsv.py onight run36 param sweeps Jun 4, 2019
random_hacking.ipynb runs 07 to 12 May 29, 2019
run_random_grasps.py add --fixed-camera-seed-offset to run_random_grasps to ensure held ou… Jun 3, 2019
runs_36.sh fix refactor bug Jun 5, 2019
runs_37.sh fix refactor bug Jun 5, 2019
stitch.py higher capacity model from run 38 Jun 5, 2019
train.py add support for --negative-selection-mode ranged_frame Jun 3, 2019
triplet_selection.py add support for --negative-selection-mode ranged_frame Jun 3, 2019
util.py export joint positions to {--joint-info-dir}/rNNN/cNNN.tsv Jun 3, 2019

README.md

pybullet grasping with time contrastive network embeddings

see the blog post for more details

generate images

synthetic r/g/b squares

./generate_toy_rgb_data.py

small set

./run_random_grasps.py --run 1 --num-cameras 4 --num-grasps 5 &
./run_random_grasps.py --run 2 --num-cameras 4 --num-grasps 5 &
wait

simple grasping set

  • 2 cameras
  • 0 objects
time seq 0 99 | parallel './run_random_grasps.py --run {} --render-freq 10 --num-grasps 5 --num-cameras 2 --num-objects 0 --img-dir'

full N set

time seq 0 99 | parallel './run_random_grasps.py --run {} --render-freq 10 --num-grasps 5'

debugging / review images

display a 5x5 sample of random frames

./debug_random_frames.py

display a sample of images across frames / cameras columns are a frame in time, rows are specific cameras

./debug_frame_sequence_cameras.py --run 1 --initial-frame 1 --num-frames 4 --cameras 0,1,2,3

show all camera images for a specific frame

./debug_frame.py --run 1 --frame 10

show random (anchor, positive, negative) triples that will be used for training

./debug_random_triples.py

training model

./train.py --help

  --img-dir IMG_DIR
  --batch-size BATCH_SIZE
                        note: effective batch is x3 (default: 16)
  --embedding-dim EMBEDDING_DIM
                        image embedding dim (default: 8)
  --learning-rate LEARNING_RATE
                        learning rate for adam (default: 0.001)
  --epochs EPOCHS
  --steps-per-epoch STEPS_PER_EPOCH
  --model-output MODEL_OUTPUT
                        where to save model (default: model)

working with embeddings

calc embeddings for all images

./embed_imgs.py

debug near neighbours

./debug_embedding_near_neighbours.py

TODOs:

  • near neighbour debug that checks all images of run from camera_1 and finds NN in images all of camera_2
You can’t perform that action at this time.