Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Remove mesos dep from SingularityBase #1648
This works towards solving a problem that has plagued us for a while now. We've been storing the mesos protos objects as json in zk/sql. With each mesos update, these can change shape and cause us issues with (de)serialization. Additionally, the update to mesos 1 pulled in protobuf 2.6.1, causing issues with ObjectMappers and jackson in SingularityClient for anything running a different protobuf version.
This PRs aims to recreate POJOs for most of the important mesos task info inside of SingularityBase, with the result of removing any dependency on org.apache.mesos from SingularityBase and the SingularityClient.
All mesos operations are currently moved to a mesos utils module (might move to SingularityService if it is the only one using it).
This is part way done, MesosOfferObject is finished, but still need to finish the MesosTaskObejct
Changed up the strategy on this. I realized if we try and keep our own POJOs for the entire mesos protos, we fall victim to the same breakage for updates. However, we only use a very small subset of the fields, and just store the rest for reference.
So, I've made POJOs for the fields in the mesos protos that we use, and utilized jackson's AnyGetter/AnySetter for the rest of the fields. This way everything will still be stored in the json for possible future usage, but we don't need to explicitly update beyond the fields that we actually need. With this the dep on mesos and on protobuf are now gone from SingularityBase. I'll be adding a release note to docs on the fact that the new objects mimic the protos for the most part, but the remaining fields would need to be accessed in different ways (or just using objectMapper.convert to go back to protos)