chore(Add Library Standards)

[Pablosan]

A master list of external (third-party) packages consistently used
across our projects. The first pass at this list is not considered
comprehensive and this list is expected to change frequently as we
discover better tools than the ones we currently use.

[ziadsawalha]

👍

[ziadsawalha]

This PR (and a few others like it we are discussing now) feel like they need more than one +1 before merging...

[stavxyz]

If we are going to include tornado in this privileged list, I will ask what it accomplishes that is not covered by the following:

https://docs.python.org/3/library/asyncio.html

or

One of the many asynchronous servers with built-in bottle support. Since bottle supports eventlet, gevent, diesel, fapws, tornado, twisted, meinheld, and bjoern servers, I would vote that we pick one and stick to it. I know of our team using eventlet and tornado, but.... lets choose one?

bottlepy/bottle really should cut releases more often- there hasn't been a release since December 2014 ( 0.12.8 ). A pull request for an asyncio based server was merged in on March 24.

[Pablosan]

@samstav The only reason for choosing Tornado over another async package is that I introduced Tornado when implementing Patrick as our first Python 3 project. The only reason Tornado was used instead of our team standard of eventlet was because, at the time, eventlet wasn't working with bottle in Python 3.

@ziadsawalha has since mentioned that Tornado also supports bottle's reloader feature which is convenient during development.

I don't have a good answer here. I've thrown away several versions of this final paragraph. I will gladly consider recommendations and change the PR to match our decision.

[stavxyz]

@Pablosan, makes sense. This clarifies the inclusion of everything in the list for me. Thanks.

[Pablosan]

@samstav The more I think about this, the more I dislike having both. This definitely sets a precedent and it is much easier to omit something than it is to remove it later. The fact that the XTornadoServer class is already included is... problematic, I think.

[ziadsawalha]

Maybe the question to ask is whether simpl reflects what we aspire to or where we currently are. We may aspire to having one, and only one, way to do things, but we run both Tornado and Eventlet in production today.

[Pablosan]

My answer to that question is "neither" and "both". The difficulty is that, once we include something (in this case, Tornado), we are in essence saying "you should feel free to use this and have at least some confidence that it will be here tomorrow." As I said elsewhere, omitting something is much easier than removing it later.

I'm fine leaving both eventlet and tornado in. I just want us to make that decision with our eyes wide opened: for all practical purposes this decision will be cast in stone.

[graywizardx]

I think inclusion of these packages moves Simpl out of the category of Library and into Framework. There is a major difference between arrow/requests and Tornado and Bottle.

Its a significant line for me.

What is the reason for including these libraries as our recommended/dependency ones? Is it just that we have significant portions of Simpl that were dependent on them?

[Pablosan]

@graywizardx the reason we are including libraries like bottle, eventlet, tornado, pymongo, etc. is because when we set out to build a new REST API these are the tools we reach for and when we implement them there are certain patterns we have found useful.

One example: Tornado was introduced by the Patrick project because, at the time, eventlet wasn't playing nice with bottle in Python 3. Unfortunately, tornado had X-Headers disabled by default and we routinely use X-Forwarded-For in our debugging. So we introduced our own XTornadoServer class with X-Headers enabled so we didn't have to re-remember that lesson.

I'm not sure whether that places simpl firmly in the Framework camp or not, but I take your point. What concerns you about crossing that line?

[stavxyz]

I agree with @graywizardx that it's a significant line.

@Pablosan you didn't ask me, but.. I'm going to say that it doesn't bother me if it is (or we call it) a framework.

[stavxyz]

Relevant issue: #68

[ziadsawalha]

Where are we with this? Is anyone averse to merging it in as is and iterating on it? @graywizardx, specifically?

[graywizardx]

So immediate issues first, dont block on merging because of me. I dont want to be in the habit of blocking things because of opinion.

As to @Pablosan question about my concern about frameworks, is that they (frameworks) become a "way of life" or "an anchor" depending on how they go down. I think its really hard to build a framework, probably harder than anything else I have ever worked on, and I have bad memories of frameworks as an impediment to trying to get simple things done (I just want to spin up a simple REST service to do X).

If we are going the framework route, I would like to be clear about when and how we intend to use Simpl. For instance why Bottle and not Flask? Why Tornado and not Gunicorn? Why not CherryPy? Whats our reasons for using the technologies we picked, and not others? When can we diverge and what do we expect from Simpl if we do diverge?

To me there is a big difference between patterns/reference implementations and a framework. A pattern shows me how to do things, and I make changes to fit what I am doing (here is an example of how to do linq expressions...) and a framework takes away my freedom in favor of accelerating my workflow (use bottle, and tornado and this specific version of pygments and in return I will give you these prebuilt libraries that make these operations one line calls).

For me frameworks work well in large shared infrastructures, where interoperability of software stacks is key (we always build software this same way) and flexibility is less important (we always build the same kind of software) and libraries are some where in the middle (here is a library use it if it makes sense, otherwise build the behavior yourself). Reference implementations are on the far opposite side, showing the basics of how to do something but leaving the meat to be re-implemented.

I see a lot of discussion about reducing code duplication and "how we did it previously" but not about what we want to build now. I think Simpl is in limbo until we decide that. Deciding that might have large impacts for us as a team as well.

For my opinion, I dont mind code dupes it keeps projects and stacks independent and separated it does mean divergence overtime, so as with anything it means trade-offs. I would rather see us build several libraries of functionality and avoid 3p dependencies as much as possible. I'd like to see us move away from building cap-stone tools (i.e. that combine specific functionality from other libraries like bottle and Tornado). Build simple tools that bootstrap common problems/design best practices and leave things for specific technologies in a reference library area where people can cut/paste as necessary.

That is all IMHO and YMMV statements.