Regression in EnumArray fast-avro serialization under Avro 1.4

[volauvent]

Fast-avro serialization is slower than regular avro for EnumArray schema under Avro 1.4. Check out below benchmark results and schema pattern. It was faster before version 0.1.9
The serialization under Avro 1.8 is fine.

Avro 1.4 benchmark resut:

Benchmark                                                                           Mode  Cnt     Score       Error   Units
FastAvroSerdesBenchmark.testAvroSerialization                                       avgt    5  7431.495 ±   906.343   ns/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate                        avgt    5   413.867 ±    48.562  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate.norm                   avgt    5  3384.000 ±     0.001    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space               avgt    5   389.459 ±   865.511  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space.norm          avgt    5  3143.086 ±  6991.629    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space           avgt    5     0.044 ±     0.315  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space.norm      avgt    5     0.359 ±     2.556    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.count                             avgt    5     4.000              counts
FastAvroSerdesBenchmark.testAvroSerialization:·gc.time                              avgt    5    15.000                  ms
FastAvroSerdesBenchmark.testFastAvroSerialization                                   avgt    5  9664.812 ±  1175.680   ns/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate                    avgt    5   315.128 ±    36.635  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate.norm               avgt    5  3352.000 ±     0.001    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space           avgt    5   259.297 ±   933.239  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space.norm      avgt    5  2794.250 ± 10193.706    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space       avgt    5     0.202 ±     1.727  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space.norm  avgt    5     2.272 ±    19.417    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.count                         avgt    5     3.000              counts
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.time                          avgt    5    35.000                  ms

Avro 1.8 benchmark result:

Benchmark                                                                           Mode  Cnt     Score      Error   Units
FastAvroSerdesBenchmark.testAvroSerialization                                       avgt    5  8829.567 ± 4540.217   ns/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate                        avgt    5   350.364 ±  171.892  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate.norm                   avgt    5  3360.000 ±    0.001    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space               avgt    5   363.535 ±  820.904  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space.norm          avgt    5  3451.205 ± 8388.244    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space           avgt    5     0.046 ±    0.344  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space.norm      avgt    5     0.504 ±    3.896    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.count                             avgt    5     4.000             counts
FastAvroSerdesBenchmark.testAvroSerialization:·gc.time                              avgt    5    18.000                 ms
FastAvroSerdesBenchmark.testFastAvroSerialization                                   avgt    5  6423.710 ± 3154.787   ns/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate                    avgt    5   476.969 ±  243.782  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate.norm               avgt    5  3328.000 ±    0.001    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space           avgt    5   491.521 ±  218.395  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space.norm      avgt    5  3467.017 ± 2125.681    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space       avgt    5     0.198 ±    1.649  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space.norm  avgt    5     1.595 ±   13.316    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.count                         avgt    5     5.000             counts
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.time                          avgt    5    27.000                 ms

Schema (Test record array contains 200 enum elements)

{
  "type": "record",
  "name": "BenchmarkSchema",
  "namespace": "com.linkedin.avro.fastserde.generated.avro",
  "doc": "Enum array",
  "fields": [
    {
      "name": "EnumArray",
      "type": {
        "type": "array",
        "items": {
          "type": "enum",
          "name": "Method",
          "symbols": [
            "OPTIONS",
            "GET",
            "HEAD",
            "POST",
            "PUT",
            "DELETE",
            "TRACE",
            "CONNECT",
            "EXTENSION"
          ],
          "doc": "HTTP method of the request.",
          "compliance": "NONE"
        }
      }
    }
  ]
}

[gaojieliu]

Interesting, did you see any related changes after 0.1.9?

[volauvent]

No, I have no idea what has happened so far. @gaojieliu

[gaojieliu]

Could you share the generated fast-serializers with 0.1.9 and the latest version?
From the generated classes, it should be straightforward to tell the differences.

[volauvent]

I have made a similar change for Enum Schema (Avro 1.4) in FastSerializerGenerator like FastDeserializerGenerator in #49 .
This is benchmark results. It has become a little better.

Benchmark                                                                           Mode  Cnt     Score      Error   Units
FastAvroSerdesBenchmark.testAvroSerialization                                       avgt    5  7560.036 ±  690.972   ns/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate                        avgt    5   406.696 ±   36.323  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.alloc.rate.norm                   avgt    5  3384.000 ±    0.001    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space               avgt    5   389.452 ±  865.498  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Eden_Space.norm          avgt    5  3209.487 ± 7137.196    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space           avgt    5     0.043 ±    0.313  MB/sec
FastAvroSerdesBenchmark.testAvroSerialization:·gc.churn.PS_Survivor_Space.norm      avgt    5     0.358 ±    2.583    B/op
FastAvroSerdesBenchmark.testAvroSerialization:·gc.count                             avgt    5     4.000             counts
FastAvroSerdesBenchmark.testAvroSerialization:·gc.time                              avgt    5    14.000                 ms
FastAvroSerdesBenchmark.testFastAvroSerialization                                   avgt   10  7464.840 ±  185.288   ns/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate                    avgt   10   407.911 ±    9.827  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.alloc.rate.norm               avgt   10  3352.000 ±    0.001    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space           avgt   10   400.763 ±  324.879  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Eden_Space.norm      avgt   10  3281.003 ± 2660.225    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space       avgt   10     0.103 ±    0.477  MB/sec
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.churn.PS_Survivor_Space.norm  avgt   10     0.852 ±    3.946    B/op
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.count                         avgt   10     8.000             counts
FastAvroSerdesBenchmark.testFastAvroSerialization:·gc.time                          avgt   10    36.000                 ms

[gaojieliu]

That sounds great, but it is a separate improvement, which would like to have anyway.

But I think it is still important to understand the difference between 0.1.9 and the latest version by looking at the difference of the generated serializers.

[volauvent]

I have checked the generated serializers between 0.1.9 and the latest version under Avro 1.4. They are exactly the same BenchmarkSchemaGenericSerializer.

I found the old Avro-1.4 EnumArray benchmark result was not correct since the Enum value index used in regular avro-1.4 and fast-avro were different... The value string used in fast-avro benchmark happen to be shorter than the one used in regular avro-1.4. So fast-avro result was faster than regular avro 1.4 in the old test.

I re-benchmarked the EnumArray using 0.1.9 under Avro 1.4, the result was the same as above benchmark result. Fast-avro 0.1.9 is slower.

Benchmark                                             Mode  Cnt      Score      Error   Units
FastAvroSerdesBenchmark.testFastAvroSerialization     avgt   50  9855.346 ±   77.602   ns/op