FC layer information

[chunseoklee]

This issue is for sharing our target model's FC configuration. Due to the confidential issue, we cannot share the model in public repo. Instead, I will share its FC layer's information :

TFLite's FC layer takes two main input tensors: (m,k) input1(a.k.a. weight) and (n,k) input0(a.k.a. input), which will produce (n,m) output

n, k, m
1 1 2 1536
1 128 6144
1 1536 128
1 1536 256
1 1536 64
1 256 6144
1 40 6144
1 64 6144
4 1000 64
4 10137 256
4 2 1536 64
4 2 64 1536
4 256 621
4 3157 384
4 3157 96
4 384 1000
4 384 10137
4 500 384
4 64 1536
4 96 4000

Note that 4 2 64 1536 is interpreted as 8 64 1536

Depicted by tensor shape :

input	weight	output
(1, 128)	(6144,128)	(1,6144)
(1, 1536)	(128,1536)	(1,128)
(1, 1536)	(256,1536)	(1,256)
(1, 1536)	(64,1536)	(1,64)
(1, 2)	(1536,2)	(1,1536)
(1, 256)	(6144,256)	(1,6144)
(1, 40)	(6144,40)	(1,6144)
(1, 64)	(6144,64)	(1,6144)
(4, 1000)	(64,1000)	(4,64)
(4, 10137)	(256,10137)	(4,256)
(4, 256)	(621,256)	(4,621)
(4, 3157)	(384,3157)	(4,384)
(4, 3157)	(96,3157)	(4,96)
(4, 384)	(1000,384)	(4,1000)
(4, 384)	(10137,384)	(4,10137)
(4, 500)	(384,500)	(4,384)
(4, 64)	(1536,64)	(4,1536)
(4, 96)	(4000,96)	(4,4000)
(8, 1536)	(64,1536)	(8,64)
(8, 64)	(1536,64)	(8,1536)

[chunseoklee]

cc @wateret @periannath @hseok-oh

[periannath]

AFAIK, those numbers mean n, k, m instead of n, m, k.

For example, 1 256 6144 means 1x256 shape input, 256x6144 6144x256 shape weight and 1x6144 shape output.

[chunseoklee]

AFAIK, those numbers mean n, k, m instead of n, m, k.

oops! I have updated the label.

For example, 1 256 6144 means 1x256 shape input, 256x6144 shape weight and 1x6144 shape output.

the shape of weight is 6144x256 in tflite schema.

[periannath]

the shape of weight is 6144x256 in tflite schema.

Thanks. It was my mistake. I've fixed it.

[chunseoklee]

It is bit comfusing. AFAIK, Note that gemm library like gemmlowp and ruy takes weight(of TFL's FC) as a LHS(left hand side) and input(of TFL's FC) as a RHS.

[chunseoklee]

I have added tensor shape based table.