Extract Kryo interface

[brianfromoregon]

I have a number of extensions of Kryo in my code base, either to fix bugs or add checks. Right now I have to override a number of methods and it feels brittle. If you had a Kryo interface I could implement instead and then delegate to the real Kryo, I'd sleep better at night :)

[serverperformance]

What's wrong with inheritance + factory? (I'm old school).

I also have my customized XKryo subclass with overriden behaviour in some methods.

:)

-----Original Message-----
From: Brian Harris notifications@github.com
Date: Wed, 13 Aug 2014 13:41:21
To: EsotericSoftware/kryokryo@noreply.github.com
Reply-To: EsotericSoftware/kryo reply@reply.github.com
Subject: [kryo] Extract Kryo interface (#235)

I have a number of extensions of Kryo in my code base, either to fix bugs or add checks. Right now I have to override a number of methods and it feels brittle. If you had a Kryo interface I could implement instead and then delegate to the real Kryo, I'd sleep better at night :)

---

Reply to this email directly or view it on GitHub:
#235

[brianfromoregon]

It's something that only became clear to me after my continual use of inheritance at work was burning me more often than not in terms of making future refactoring of the base class difficult, because base and (potential) children were more tightly coupled than I realized. A final class, or else a class with mostly final/private methods is a pleasure to refactor. It cannot be extended via inheritance but can be extended via composition. Think about what change could be made to the internals of the Kryo class today without potentially breaking users who are extending/overriding. There are very few of them! With Kryo devs giving increased attention to backward compatibility lately, working toward having an interface as the extension point would untie their hands for future refactoring. And I carry less risk because it's less likely that a Kryo version upgrade down the road will break my XKryo. For more reading google "favor composition over inheritance". It's not a java thing specifically, tho bloch does have two items (see 16 and 17) in Effective Java which cover it.

[romix]

@NathanSweet @brianfromoregon The idea of using interfaces in Kryo was suggested more than once. The usual concern was, IIRC, that it could introduce a performance hit. Nate should correct me, if I'm wrong. But eventually it is not the case anymore with modern JVMs. I think the easiest way to prove it is to try it out. Basically, decouple the Kryo class into an interface and implementation. Make sure all tests still work. Run some performance benchmarks, e.g. https://github.com/RuedigerMoeller/serialization-benchmark and see if any measurable overheads are introduced by this re-factoring.

@brianfromoregon Would you like to conduct this experiment and report back the results? It should be pretty straight-forward, hopefully.

[NathanSweet]

Performance is not an issue with interfaces. The issue is that same as when any additional layers are added, the overall complexity is higher. I generally don't use interfaces until it can be seen that the abstraction is useful because I dislike the clutter.

Back in the good old days, Kryo was extremely simple to follow, now we have all sorts of factories and implementations. 😄 There are reasons for this of course, but the cost is that it is more complex.

I doubt a Kryo interface really reduces the chance that a future version of Kryo breaks something. Any significant refactoring probably breaks something anyway.

[romix]

Nate, thanks for your explanation. A small remark from my side: After having looked at the implementation of many Java serialization libs (Avro, Hazelcast serialization layer, fast-serialization, protostuff-runtime and many others), I'd say that Kryo is still rather simple and small in comparison with most/all of them. But yes, it is getting more complex with a time...

[brianfromoregon]

An example from today. I wish Output was an interface because I'm writing an impl which writes to direct memory with Unsafe, and some of the methods I must inheirit doesn't make sense for me like setOutputstream, setBuffer, setPosition. An Output interface wouldn't have such methods I suppose.

[NathanSweet]

An Output interface would probably have those methods. It doesn't hide that it uses a buffer.

[romix]

BTW, you know that you can write using Unsafe IO streams into direct memory already? Just use this constructor:
public UnsafeMemoryOutput (long address, int maxBufferSize), where address is the address in the off-heap memory.

Why do you need to implement an alternative solution?

[brianfromoregon]

Hot dog! Thanks for the pointer I'll start with this!

A quick glance shows two things I might need to adjust

• writeInt and others should do bounds checking with assert keyword so no penalty in prod
• this memory is a region of a mmaped file, and I have that ByteBuffer being (privately) managed and written to elsewhere, so having UnsafeMemoryOutput create and write to its own buffer over the same address space might cause a problem? hmmm.. need to try it.

[romix]

Let me know if it worked for you. Also if you have any improvement proposals for those UnsafeMemoryInput/Output APIs, please let me know. I'm interested in making it even better.

[romix]

@brianfromoregon BTW, how did it work for you? Have you seen any improvements using Unsafe IO streams?

[brianfromoregon]

Hi, did not adopt the unsafe streams because of issue discovered with read ops performing writes. May revisit again later.

[magro]

Closing, reopen if still relevant.