How to measure the embedding/representation difference between layers?

[LifangD]

Can anyone explain how to draw such figure since the L2 distance or cosine similarity is often conducted on the vector rather than the embedding matrix. Besides, how to measure it on the whole dataset?
Thanks for any suggestions or references~

[brightmart]

may be you can take the first position, which is [CLS] token, it represent the sequence of that layer. so that you can have a hidden states as vector to represent that specific layer.
after you trained the model, or you get a checkpoint from pre-trained model, i think you can measure it, as all the parameters are fixed. it may not relate to a downstream dataset.

[LifangD]

may be you can take the first position, which is [CLS] token, it represent the sequence of that layer. so that you can have a hidden states as vector to represent that specific layer.
after you trained the model, or you get a checkpoint from pre-trained model, i think you can measure it, as all the parameters are fixed. it may not relate to a downstream dataset.

OK, thanks~