Remove Fog::XML and Fog::JSON to own gems

[tokengeek]

Ooooo new repo!

In line with #1 we should remove the two parsing systems into their own gems, allowing fog to be dependent on them.

fog-xml (or similar) would then manage the nokogiri dependency and fog-json the multijson dependency.

From fog user perspective it shouldn't change but when providers break out into modules then they can just include the relevant one.

I think this is already possible, just the requires in lib/fog/core.rb need removing and relevant modules deleting for fog-core to function.

[geemus]

This hadn't even occurred to me (though I know I'd like this to happen, I hadn't realized it was near at hand). Good call!

[tokengeek]

I think the biggest logistics are upstream. Creating new repos and gems for it.

Will have a quick look at deleting them today or tomorrow so we can reduce core down even further.

[tokengeek]

Yeah - I thought most of the problems had been covered. #3 just deletes the code and tests still pass.

We just need to be careful to re-require them in fog and not delete them when clearing it out.

[elight]

This brings up a whole different question: how careful do we want to be about extracting new gems and thereby creating the appearance of new dependencies?

[tokengeek]

@elight I think a lot of it comes from either the dependencies or possible reuse for other people.

The XML code, brings in nokogiri, which requires build tools to install the native gem etc. and it creates a big burden for tools using fog that don't require it.

Historically @geemus has extracted bits (like Excon) where there was potential for reuse but I'm not sure how much is going to be left.

There was mention before about splitting things by service as well in some capacity (compute, storage etc) since that could benefit some use cases (something like Paperclip which manages uploads to S3 but doesn't care about servers, DNS or other things).

However the management of that would create a bit of work and is probably the step too far.

So I think we do want to be careful and not do too much but there might not been much left.

Ideally I'd like us to clean up a few bits of code and use something like https://github.com/rom-rb/rom to replace the attributes code and https://github.com/hoxworth/json-schema to replace the schema stuff (fog/fog#2626) so maybe adding a few choice dependencies for some of the code that was needed but is now filling out core.

Where we do have dependencies it would be nice to get them as standardised in fog as possible to avoid everyone just wrapping a bunch of SDK gems each depending on a different HTTP library.

[elight]

My concern is for potential version conflicts down the line for consumers of said extracted gems. Fog will depend on the latest version whereas other consumers of these extracted gems may not. This is a possibility for any gem we extract that is consumed beyond fog.

Re: attributes, Virtus?

On Tue, Feb 4, 2014 at 2:48 PM, Paul Thornthwaite
notifications@github.com wrote:

@elight I think a lot of it comes from either the dependencies or possible reuse for other people.
The XML code, brings in nokogiri, which requires build tools to install the native gem etc. and it creates a big burden for tools using fog that don't require it.
Historically @geemus has extracted bits (like Excon) where there was potential for reuse but I'm not sure how much is going to be left.
There was mention before about splitting things by service as well in some capacity (compute, storage etc) since that could benefit some use cases (something like Paperclip which manages uploads to S3 but doesn't care about servers, DNS or other things).
However the management of that would create a bit of work and is probably the step too far.
So I think we do want to be careful and not do too much but there might not been much left.
Ideally I'd like us to clean up a few bits of code and use something like https://github.com/rom-rb/rom to replace the attributes code and https://github.com/hoxworth/json-schema to replace the schema stuff (fog/fog#2626) so maybe adding a few choice dependencies for some of the code that was needed but is now filling out core.

Where we do have dependencies it would be nice to get them as standardised in fog as possible to avoid everyone just wrapping a bunch of SDK gems each depending on a different HTTP library.

Reply to this email directly or view it on GitHub:
#2 (comment)

[geemus]

versioning is always terrible. I'm not sure it is that much worse here though (at least in the xml/json cases). We have already had many headaches around both and probably will continue to, I guess maybe this way we at least reduce the number of users likely to have said headaches at any given time? Certainly worth considering carefully before we decide one way or the other.

[tokengeek]

I think for versioning (at least XML/JSON stuff) that we just depend on the latest gem in fog and/or fog-#{provider}

So fog won't declare a dependency to any version of nokogiri and so on...

Providers should only use fog-xml and any issues with nokogiri can be dealt with within there.

If a third party library or a users app is in dependency hell, we should try more optimistic versioning so ultimately the user's app, third party library and/or Bundler have to make the decision somewhere about the correct version to use.

Most of our versions are overly pessimistic for a library but more in line with fog as a binary tool.

For example is it fog's responsibility to ensure that an app is using the latest version of nokogiri with security fixes? Even if possibly forcing the app developers into conflicting versions between another library in their app?

We should be as optimistic as possible in versions and only bump them up when old versions are known to break things.

When a library we depend on breaks something we'll have to see what impact it causes. Can we fix it in code quickly? Do we have to lock down the version? Is it an upstream issue we raise there?

We had to deal with this when nokogiri dropped 1.8.7 support between 1.5 and 1.6 and even then our pessimistic versioning couldn't catch a major change in a minor release.

[tokengeek]

re: attributes. I was probably thinking Virtus when I said ROM. I struggle to keep up with what @solnic is working on!

Not looked too much at the details but there's almost a case for mapping the API response data to the fog models as well.

So there's two more possible dependencies that could end up conflicting so we'd probably declare the loosest version dependency possible and just have to manage changes as they arose.

As to if we actually want to use them is a completely different issue but links nicely to dependencies. The less code we have to maintain, test and debug allows us to be free for the higher level stuff like standard interfaces to models.

[solnic]

Who summoned me? ;)

What is it that you need?

[tokengeek]

Heh! I'll create a new issue since it's only partially related here!

[tokengeek]

Original issue now resolved. The JSON and XML parsers remain in fog for extraction soon.

[geemus]

Thanks!

On Wed, Feb 12, 2014 at 11:13 AM, Paul Thornthwaite <
notifications@github.com> wrote:

Original issue now resolved. The JSON and XML parsers remain in fog for
extraction soon.

Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-34891519
.