Support for multi-threaded run

[breznak]

• Each HTM pipeline composes of SP and TM(TP) computation; these 2 components take approximately same time now, see Sp profile review nupic-legacy#3131

The full pipeline is Encoder -> SP -> TM -> Classifier/Anomaly, but the SP,TM are the most computationally intensive routines.

• All modern systems are multithreaded

Therefore the proposed simple optimization would halve the time of practical HTM applications.

Requirement:

• the 2 regions should be synced on a spOutput variable, which is then input to TM

Implementation:

• show a multithreaded use in Low-level Anomaly example #889
• create a parallel/synced region that wraps any other region inside it
• two possible approaches:
  • optimize "HTM application", consisting of SP,TM,...
  • easier
  • run SP in one thread, TM in another,...
  • optimize individual classes (SP,TM,..)
  • many-core systems can gain even more
  • more complex
  • run SP.method1 in one thread, SP.method2 in other,...
  • gain even if used only a single class (e.g. SP only)

Threads are new feature of C++11: https://solarianprogrammer.com/2011/12/16/cpp-11-thread-tutorial/

[rhyolight]

Each HTM pipeline composes of SP and TM(TP) computation; these 2 components take approximately same time now, see numenta/nupic-legacy#3131

I don't think this is true after reading @mrcslws 's discussion on numenta/nupic-legacy#3131

Also, @breznak, I don't want to describe implementation details in Github Issues. I think those should be discussed on the NuPIC Developer Forum and subsequent pull request(s).

Each Github Issue should clearly define one proposal or feature, not the implementation. Please update this ticket to remove implementation details, and open the appropriate discussions in a new NuPIC Developer Forum topic.

[cogmission]

@rhyolight I don't have any opinion on the matter, but I was wondering what you might think of having references to implementation discussions from issues?

[breznak]

I don't think this is true after reading @mrcslws 's discussion on numenta/nupic-legacy#3131

Well, Marcus said the random code uncovered problematic/unoptimized code-paths, while the "well behaved" data benchmarks another paths. It's still trivial to see that the time would be reduced to max(TM, SP time) from SP + TM time. Let's wait to the updated benchmark in the aforementioned PR to see how much impact the optimization would have, but my guess is that even when TM/SP are optimize, the gain would still be relatively high in percentual performance.

Each Github Issue should clearly define one proposal or feature, not the implementation...

@rhyolight isn't the bureaucratic bar for contribution rising recently? What functionality would the issues have then? Are there many developers/reviewers who monitor a forum and not a GH issues/PRs?

[rhyolight]

@breznak I've been advocating for this approach of splitting up features and implementation for years now. Here's a message from April 2014:

An issue defines a problem that needs to be solved, not necessarily the solution that solves it.

Not trying to add bureaucracy, but the simple truth is that we don't have the manpower to keep up with you. Keeping issues extremely lean and clean helps. Keeping details about the implementation out of the feature / problem description decreases the cognitive dissonance involved in understanding the issue.

@breznak said:

Are there many developers/reviewers who monitor a forum and not a GH issues/PRs?

Issues get very hard to follow when there is a large amount of discussion about implementation details on them. The larger an issue gets, the harder it is to grok it. The less likely it is that another contributor will bother to read through it. I like the idea of trying to keep Github Issues as tidy as possible, discussing the messy details on the forum. Maybe that's not the right way to do things, but I'm not happy with how slowly things are moving with your issues and PRs today, so what else can I do but try something new?

I am the project manager, not a core developer on this project. I usually can't validate issues and review PRs, and I can't force reviewers to discuss issues and review PRs. I'm still trying to make issues and PRs more approachable. Let me know if you have any other ideas.

@cogmission said:

@rhyolight I don't have any opinion on the matter, but I was wondering what you might think of having references to implementation discussions from issues?

Sure, fine with me.

[rhyolight]

@breznak I don't want to continue this conversation here on Github. Let's continue it on HTM Forum.

[rhyolight]

We will continue discussion of this issue here until it is better-defined: https://discourse.numenta.org/t/multi-threading-optimization-in-nupic/524.