Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SCB-918] Codec based on proto #930

Merged
merged 9 commits into from Oct 10, 2018
Merged

Conversation

wujimin
Copy link
Contributor

@wujimin wujimin commented Sep 27, 2018

need to merge #928 first

ProtobufRoot.java is generated by protobufRoot.proto, 5000 lines.
it's only for unit test

  • compare to jackson protobuf:

    • can parse protobuf 3 proto file
    • support protobuf 3: map/any
  • compare to protoStuff runtime:

    • for a proto message type, not only support strong type(Pojo), but alse support weak type(Map)
    • support "any" type
    • support generic pojo type, eg:CustomGeneric<User>
    • NOT support List<List>/List<Map<X, Y>> any more, because protobuf specification not support it, and the parser can not parse the proto file
  • compare to standard protobuf:

    • extend "any" type, for standard not support cases, use "json" schema to codec it.
  • compare to all:

    • just pojo, no need any code generation and annotation
    • one pojo can serialize to different version proto file to support different version server
    • support text data come from http,can serrialize from different data type
      • number fields (int32/int64 and so on)
        • number
        • String
        • String[]
      • string fields
        • string
        • string[]
      • bool fields
        • boolean
        • string
        • string[]
      • enum fields
        • enum
        • number
        • string
        • string[]
  • performance compare:

1.protobuf
  in our real scenes
  business model never bind to transport, and can switch between different transports dynamically
  that means if we choose standard protobuf, must build protobuf models from business models each time
  so should be much slower than the test results
2.protoStuff
  some scenes, there is no field but have getter or setter, so we can not use unsafe to access field
  so we disable protoStuff unsafe feature
3.jackson
  not support map, so skip map in map/mixed test
4.serialize result size
  ScbStrong/ScbWeak/Protobuf have the same and smaller size, because skip all default/null value
Empty:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :250        250        235        156        437        
ser len       :36         0          0          0          56         
deser time(ms):125        15         0          257        483        
deser-ser len :36         0          0          0          56         

Scalars:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :235        264        218        156        413        
ser len       :53         21         21         21         73         
deser time(ms):156        63         94         225        469        
deser-ser len :53         21         21         21         73         

SimpleList:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :266        250        220        172        440        
ser len       :68         32         32         32         88         
deser time(ms):234        94         109        265        499        
deser-ser len :68         32         32         32         88         

PojoList:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :297        343        235        187        543        
ser len       :56         20         20         20         76         
deser time(ms):211        126        168        298        610        
deser-ser len :56         20         20         20         76         

Map:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :404        512        424        533        403        
ser len       :92         54         54         54         56         
deser time(ms):500        343        406        750        359        
deser-ser len :92         54         54         54         56         

Mixed:
               Protostuff ScbStrong  ScbWeak    Protobuf   Jackson    
ser time(ms)  :579        704        547        579        625        
ser len       :161        127        127        127        125        
deser time(ms):736        623        766        1015       798        
deser-ser len :161        127        127        127        125        

@coveralls
Copy link

coveralls commented Sep 28, 2018

Coverage Status

Coverage increased (+0.09%) to 86.289% when pulling 388cc81 on wujimin:codec-based-on-proto into 6aadae0 on apache:master.

@wujimin wujimin force-pushed the codec-based-on-proto branch 3 times, most recently from 6f1a259 to 00533ac Compare September 29, 2018 17:00
@liubao68 liubao68 merged commit 179de78 into apache:master Oct 10, 2018
@wujimin wujimin deleted the codec-based-on-proto branch October 11, 2018 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants