log batches of embeddings

[A-Jacobson]

Is there a way to log batches of embeddings then interact with all of the batches together in the projector?

I'm currently logging batches with the following code:

writer.add_embedding(embedding_batch, metadata=label_batch,
                   label_img=train_batch, global_step=current_epoch) 

When I check the embeddings in the projector I have 10 (batch size * dim) tensors and I can only look at one batch at a time. they are named default:0000 - default:0010. Any way to get these all on the same graph?

[lanpa]

How about concatenate those tensors as a big tensor and append different label prefix for each tensor?
Then you can use the filter function in tensorboard projector (at the right panel) to select what you want.

[A-Jacobson]

unfortunately I don't think i can. I'm actually logging these on the fly using a hook. pytorch hooks can't return anything =(. The only thing I was able to think of is to write some code to combine the log files but there should be a better solution than that.

[lanpa]

maybe this is better than dump to a file. XD

class Log():
    def __init__():
        self.tensors = []
        self.labels = []

    def __call__(t, l):
        self.tensors.append(t)
        self.labels.append(l)

    def dump():
        return cancat(...)

[A-Jacobson]

That's the solution I JUST implemented! lol Thanks!