Discussion on project direction: Upgrade to Python 3 or Focus on Next Gen project?

[AWegnerGitHub]

This discussion has come up several times in the last month about where we should focus our efforts as a development team.

One option is to upgrade to Python 3.

A second option is to write the solution and focus on the Next Gen (NG) project.

---

The purpose of this issue is to get the entire Charcoal community to discuss the differences between the two options outside of chat, so that we can all see the discussion and respond. The timings of the discussions have excluded different aspects of the community due to when the conversations occurred.

[Undo1]

Here's a relevant quote:

Is there an alternative? The consensus seems to be that the old Netscape code base was really bad. Well, it might have been bad, but, you know what? It worked pretty darn well on an awful lot of real world computer systems.

Really, all of that post is great reading (classic Joel). Here's what I see: We have one project that works and runs in production, and one project that might eventually. There's a huge amount of work to be done to make -ng work, and even more to certify that it actually does work. While there are some awesome people working on it, there's still significant risk that -ng just... dies. If that happens, we lose whatever work we put into it.

In comparison, Classic Smokey is pretty safe; it's already stable enough that we run in prod. Any work done here has near-immediate (comparatively) impact. Considering all of that, I'd vote that we, as a general guideline, focus on Charcoal-SE/SmokeDetector.

There are three groups here - the "I want to work on Classic" people, and the "I want to work on something new" people. Those folks can work on whatever they want, of course - it's none of my business to tell them to not work on something. The folks in the middle, though, the "what should I work on" people - those are the people this question is about.

[AWegnerGitHub]

I prefer a straight upgrade to Python 3.

This option is simplest and requires the least about of re-validation that the entire solution works. Smoke Detector has earned a lot of trust across the entire network among users, moderators and employees of Stack Exchange. As far as I know, it is the only project to utilize an increased API quota and this is due to that trust level. We have more than a handful of mods that pop into the chatroom and have discussions with Smoke Detector developers about ways to improve spam prevention on their sites and we have community managers make regular trips through to discuss the project and it's future. This is a huge amount of trust across every significant user group on Stack Exchange.

A rewrite has to establish that same level of trust on day one.

A upgrade to Python 3 will involve some work. We have at least one third party library that needs to be validated against Python 3 (ChatExchange). We also need to ensure that everything else we depend on is compatible. We have several areas that behave differently between Py2 and Py3 (example: iterating over dictionaries). But, the differences between Py2 and Py3 are well documented. Additionally, with a straight split to Py3 and not working about Py2 compatibility, we reduce the amount of "compatibility" we have to maintain.

The Next Gen project has the goal of upgrading to Python 3 and fixing many other problems that Smokey has. This isn't a bad goal, but I think it's misplaced. It's doing to much at once.

I argue that we should take these issues that the NG project is working on, and fix those within the existing application after we've upgraded to Python 3. NG has identified areas that are...problematic (findspam, being dependent on certain rooms - GlobalVars.charcoal_hq, etc) and these should be fixed.

Fixed. Refactored. Not rewritten. Doing it piecemeal allows us to focus on single areas at a time. This keeps Smokey stable and in turn keeps the trust of users across the network.

[angussidney]

Here's a list of arguments for and against each option that I can think of at the moment.

Existing Smokey upgrade

Pros

• We don't need to rewrite the whole thing, including the bits that have
• Past development work will not be 'wasted'
• We may not have the energy to recreate several years worth of work, and hence making incremental changes to the current Smokey will be debateably easier

Cons

• Existing 'stale' code means that making changes will be harder than it should be
• We really will only fix a single issue, which is the Python 3 upgrade
• Difficulty in transitioning from Python 2 to 3 in production?
• Testing changes will most likely result in extra messages being posted in CHQ/SOCVR/Tavern and duplicate entries on MS, since the current code has quite a few assumptions in relation to where it should post etc, which will be very hard to change for testing purposes.

SmokeDetector-NG

Pros

• It will allow us to fix some issues that we have been dealing with for quite a while that we'd like to fix, but haven't been able to due to assumtions in older code, e.g.
  • Design Smokey to work with metasmoke, rather than it just being something which is haphazardly nailed on the side
  • Changing assumptions on which rooms we post in (use a room config file, which will make testing much easier - see the last con of the Smokey upgrade section)
  • Better cross-platform compatiability (to make testing/running on windows easier)
  • Of course, Python 3 compatiability (or even cross-version compatiability?)
  • Add more statistics e.g. how long it takes for a post to be passed through findspam, queue stats that are sent directly to MS (instead of only being able to be retrieved by the current hoster)
  • Refactor findspam and other parts of the code so that they are designed to work with OOP, rather than the newer Post object being interfaced with the old 'bucket of strings' code (Note: I'm not entirely sure whether or not this has acutally been fully fixed in the current Smokey yet)
• Rewriting from scratch may be easier than trying to refactor certain parts of the code, especially when compatiability with older components must be maintained

Cons

• Rewriting means a lot of work - do we have the energy to follow this through and complete it?
• As Andy says, we need to maintain trust with SE and the rest of our userbase - but can't we just test it extensively? Have the smokeys run side-by side for a week or so, and we'll pick up any problems. Tell everyone from CHQ, SOCVR etc to try their best to break it. Either way, a major change will need testing, and our trust must be maintained.
• Past dev work could be considered to be 'wasted'

Those are the things that I can think of at the moment - I'll add more as I think about it. I'm leaning towards Smokey-NG, but I'm open to see what other people think.

[ArtOfCode-]

A thing to keep in mind: certifying NG will likely require a month or two of side-by-side running, possibly some metasmoke changes, and close monitoring.

I'll go with +1 for Undo's last paragraph. This is still a volunteer projects, which is actually one of its strengths. We don't want people feeling obligated to work on one project, nor forbidden from working on another. If there are people who are "meh" about working on Classic, but would love to work on a new Smokey, I absolutely don't want to discourage that - I'd rather keep people around Charcoal than say "no you can't work on that" and lose a good team member.

From what I've seen, there's a (relatively small) group of people who want to work on NG, or on both. If you're one of those people, go ahead and do that, and keep the room updated on how it's going. Likewise people who want to work on just Classic. For the inbetweeners, I fall on the side of upgrading Classic because of its more immediate gratification. Working on things that will only see results a year down the line isn't for everyone.

[teward]

I'm inclined to work with both, starting with the Python 3 direct upgrade, and then long-term supporting the NG development.

If you really care, here's my thought processes:

---

Direct to Python3

With Python2 officially dying within the next handful of years (yes, I can count them on my fingers), we should be moving the existing code base to Python 3 to make sure it works without issue going into the future when Python 2 is officially killed off. While that point is years in the future, we should start moving things over now, and fixing Py2 -> Py3 bugs. This will be a hard cut - all Python 2 support will be killed off by doing this - but moves us a little more towards a modern direction.

We know the existing code-base is battle-tested and is 'trusted' for the most part as is. Moving the existing code to Python 3 will let us be able to work proper with the existing codebase for years to come, in the event the -ng project never sees the light of day fully.

We also know that the existing system won't need as much heavy testing once Python 3 upgrades roll out. While I already have a Python 3 'refactor' already completed, it's not tested yet, but it should function as is since the core of the functionality remains the same - a tried-and-battle-tested-true approach to Smokey which we know has some flaws but is more or less usable for what it is, without a complete rewrite.e

---

Go with the SmokeDetector-ng project

There's some headaches in the existing code base. This is a given. When I first started contributing here, that's one of the first things I noticed, with limited error handling at times, "odd" approaches to handling items, unnecessary self references for some classes, etc. A rewrite will allow us to 'start over' and create a new SmokeDetector and fix the issues we've discovered along the way.

However, as was stated by others, even if we get a completed SmokeDetector-ng project, we'll need well over a month of testing and vetting to compare with the existing system. And, SmokeDetector-ng is nowhere near ready to go, even as a 'trial' run.

---

Looking at this logically...

... the first thing to consider is the Python 2 'Death' that is coming up in three years (2020, end of life date, see PEP 0373). Given that, and given that I do not believe NG will be fully viable by that point, we need to support both a direct-to-Python3 upgrade, and then consider the long-term rewrite.

Firstly, I think in the shorter term we should cut direct to Python 3 with the existing codebase, so as to make sure SmokeDetector will run on Python 3 and operate even after Python kills off Python 2. This way, we have a code base that works. This should be done ASAP, and I've done a lot of the 'rebase' work short of test running. That said, I think it'll work without issue - I'm running the test suites now first to determine whether or not that is the case. (This will also take less time to develop and make working)

Secondly, I believe we should still look into a rewrite of Smokey for the very long term. This would allow us to rework the code to eliminate the issues we've noticed over time, but still have the existing code base for now usable in Python 3, so that if/when NG is viable, we can start testing that against the tried-and-true code-base.

---

But, because I don't want to be on the fence about things...

...the critical point is that we should start with the Python 3 upgrade of the existing code, and then decide if we want to continue on with the rewrite.

[AWegnerGitHub]

Question for those that have worked on NG: Is there a reason that some of that work can't be ported over the Smokey once the Py3 upgrade is completed?

[teward]

@AWegnerGitHub NG is a complete rewrite from scratch of Smokey in Py3 to begin with. "some of that work" would be long down the line, since it's nowhere near 'feasible' with any code changes being used here. It's literally a complete rewrite, with completely different code structures.

[Undo1]

Would it be possible to run the rewrite in a way that it's easier to backport improvements back to Classic? That'd be a way to lessen the potential impact of NG dying.

[AWegnerGitHub]

I'm going to close this. The migration to Py3 has been completed. Activity on NG can continue as developers like, smokey development will continue too.