Skip to content

Remove mesos dep from SingularityBase#1648

Merged
ssalinas merged 6 commits into
masterfrom
base_no_mesos
Nov 17, 2017
Merged

Remove mesos dep from SingularityBase#1648
ssalinas merged 6 commits into
masterfrom
base_no_mesos

Conversation

@ssalinas
Copy link
Copy Markdown
Contributor

@ssalinas ssalinas commented Nov 7, 2017

This works towards solving a problem that has plagued us for a while now. We've been storing the mesos protos objects as json in zk/sql. With each mesos update, these can change shape and cause us issues with (de)serialization. Additionally, the update to mesos 1 pulled in protobuf 2.6.1, causing issues with ObjectMappers and jackson in SingularityClient for anything running a different protobuf version.

This PRs aims to recreate POJOs for most of the important mesos task info inside of SingularityBase, with the result of removing any dependency on org.apache.mesos from SingularityBase and the SingularityClient.

All mesos operations are currently moved to a mesos utils module (might move to SingularityService if it is the only one using it).

This is part way done, MesosOfferObject is finished, but still need to finish the MesosTaskObejct

@ssalinas
Copy link
Copy Markdown
Contributor Author

ssalinas commented Nov 8, 2017

Changed up the strategy on this. I realized if we try and keep our own POJOs for the entire mesos protos, we fall victim to the same breakage for updates. However, we only use a very small subset of the fields, and just store the rest for reference.

So, I've made POJOs for the fields in the mesos protos that we use, and utilized jackson's AnyGetter/AnySetter for the rest of the fields. This way everything will still be stored in the json for possible future usage, but we don't need to explicitly update beyond the fields that we actually need. With this the dep on mesos and on protobuf are now gone from SingularityBase. I'll be adding a release note to docs on the fact that the new objects mimic the protos for the most part, but the remaining fields would need to be accessed in different ways (or just using objectMapper.convert to go back to protos)

@ssalinas ssalinas changed the title (WIP) remove mesos dep from SingularityBase Remove mesos dep from SingularityBase Nov 15, 2017
@ssalinas ssalinas added this to the 0.18.0 milestone Nov 15, 2017
@matush-v
Copy link
Copy Markdown
Contributor

🚢

@ssalinas ssalinas merged commit 418374f into master Nov 17, 2017
@ssalinas ssalinas deleted the base_no_mesos branch November 17, 2017 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants