New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'htm.java' detector #237

Merged
merged 24 commits into from Nov 29, 2016

Conversation

Projects
None yet
8 participants
@lscheinkman
Contributor

lscheinkman commented Apr 2, 2016

Add htm.java detector to NAB by modifying NumentaDetector to use htm.java model instead of OPF model to get the raw anomaly values. All other logic remains the same as the original NumentaDetector. In the future we should also replace the anomaly likelihood logic to use the htm.java implementation.

Follow the README instructions to build and run this new detector.

In this initial implementation I've mapped OPF model parameters directly to htm.java parameters, however we may need to fine tune htm.java parameters to get better results.

@cogmission Could you please review and if possible help me find out what is best set of parameters? I think I am missing some basic settings because the results I am getting are very different from the python version.

@BoltzmannBrain Please review as well.

CC: @subutai , @rhyolight

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 2, 2016

@lscheinkman This is AWESOME! 👍

Nice to meet you!

I left a line note here: https://github.com/numenta/NAB/pull/237/files#diff-d53aa77236428fba395772cc18304a64R59

In general the code looks very good... I'm curious about piping your input from standard in, and whether you will be entering the data from the command line? So depending on your source, there is also the FileSensor which can read a file in directly (obviating the need for a Publisher)? As I said, I'm not sure what your intentions are and also I'm not at all familiar with NAB.

I'm sure we can bounce ideas back and forth and arrive at the best treatment... But overall I'm very impressed with the use of the API - What were your experiences getting in to it? Did you find it difficult to figure things out? How was the documentation? Any feedback you can give me to improve things would be greatly appreciated!

Regarding the Parameters. The parameters used in HTM.java are with few exceptions directly taken from the NuPIC algorithms (i.e. no NetworkAPI or OPF variables were used). So as far as performance or output goes, the engineers at Numenta should be able to help you come up with the best parameters to elicit the best overall output.

Let me know how else I can help?

cogmission commented Apr 2, 2016

@lscheinkman This is AWESOME! 👍

Nice to meet you!

I left a line note here: https://github.com/numenta/NAB/pull/237/files#diff-d53aa77236428fba395772cc18304a64R59

In general the code looks very good... I'm curious about piping your input from standard in, and whether you will be entering the data from the command line? So depending on your source, there is also the FileSensor which can read a file in directly (obviating the need for a Publisher)? As I said, I'm not sure what your intentions are and also I'm not at all familiar with NAB.

I'm sure we can bounce ideas back and forth and arrive at the best treatment... But overall I'm very impressed with the use of the API - What were your experiences getting in to it? Did you find it difficult to figure things out? How was the documentation? Any feedback you can give me to improve things would be greatly appreciated!

Regarding the Parameters. The parameters used in HTM.java are with few exceptions directly taken from the NuPIC algorithms (i.e. no NetworkAPI or OPF variables were used). So as far as performance or output goes, the engineers at Numenta should be able to help you come up with the best parameters to elicit the best overall output.

Let me know how else I can help?

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Apr 2, 2016

Contributor

@cogmission We've met before when you came to our office during the "HTM Challenge" last year. I'm the Numenta engineer who was sitting next to you in the office.

Great job on the Network API!
The API is simple, intuitive and really easy to use. I was able to get a model up and running really quick just by looking at some of the examples. My overall experience with htm.java was great. I was very impressed on how quick and easy I was able to create a full HTM Java project from zero.

Thank you for the quick feedback, I will simplify the network by removing the extra layers. I was just trying to mimic the OPF network, but now I understand htm.java does not need the extra regions and layers for this simple network.

As far as piping the stdin/stdout, it was done to simplify the communication with the NAB framework. This class was written so it could be called by NAB from a python process. See htmjava_detector.py for more details.

Regarding the parameters, I am using the model parameters extracted from nupic via getScalarMetricWithTimeOfDayAnomalyParams. Based on what you said, my guess is that OPF must be changing the default values for other algorithm parameters not returned by this function. I will double check OPF parameters and updated any missing algorithm parameter value with OPF default value and let you know if that improves the results.

Thank you for your help.

Contributor

lscheinkman commented Apr 2, 2016

@cogmission We've met before when you came to our office during the "HTM Challenge" last year. I'm the Numenta engineer who was sitting next to you in the office.

Great job on the Network API!
The API is simple, intuitive and really easy to use. I was able to get a model up and running really quick just by looking at some of the examples. My overall experience with htm.java was great. I was very impressed on how quick and easy I was able to create a full HTM Java project from zero.

Thank you for the quick feedback, I will simplify the network by removing the extra layers. I was just trying to mimic the OPF network, but now I understand htm.java does not need the extra regions and layers for this simple network.

As far as piping the stdin/stdout, it was done to simplify the communication with the NAB framework. This class was written so it could be called by NAB from a python process. See htmjava_detector.py for more details.

Regarding the parameters, I am using the model parameters extracted from nupic via getScalarMetricWithTimeOfDayAnomalyParams. Based on what you said, my guess is that OPF must be changing the default values for other algorithm parameters not returned by this function. I will double check OPF parameters and updated any missing algorithm parameter value with OPF default value and let you know if that improves the results.

Thank you for your help.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 3, 2016

@lscheinkman Thanks for the feedback! I just thought of another potential deviation between versions, which could be if the NAB's configuration includes the old TP (htm.java doesn't use the old TP but rather the new TemporalMemory)?

cogmission commented Apr 3, 2016

@lscheinkman Thanks for the feedback! I just thought of another potential deviation between versions, which could be if the NAB's configuration includes the old TP (htm.java doesn't use the old TP but rather the new TemporalMemory)?

Show outdated Hide outdated README.md
encoderParams["value"]["fieldname"] = "value"
encoderParams["value"]["name"] = "value"
self.sensorParams = encoderParams["value"]

This comment has been minimized.

@BoltzmannBrain

BoltzmannBrain Apr 11, 2016

Contributor

Please add newline at end of files.

@BoltzmannBrain

BoltzmannBrain Apr 11, 2016

Contributor

Please add newline at end of files.

@BoltzmannBrain

This comment has been minimized.

Show comment
Hide comment
@BoltzmannBrain

BoltzmannBrain Apr 11, 2016

Contributor

@lscheinkman I left a few comments, but overall looks good.
We still will hold off merging until we can validate htmjava yields the same results as the NuPIC detector.

Contributor

BoltzmannBrain commented Apr 11, 2016

@lscheinkman I left a few comments, but overall looks good.
We still will hold off merging until we can validate htmjava yields the same results as the NuPIC detector.

@subutai

This comment has been minimized.

Show comment
Hide comment
@subutai

subutai Apr 11, 2016

Member

You'll never get the exact same result as we don't share RNG implementations

That's fine. Right now the NAB score of the implementation using HTM.Java is terrible - it is basically random. We just need to debug, figure out the issues, and get the score in the same ballpark. It's probably something simple - I'm confident @lscheinkman or @cogmission will figure it out!

Member

subutai commented Apr 11, 2016

You'll never get the exact same result as we don't share RNG implementations

That's fine. Right now the NAB score of the implementation using HTM.Java is terrible - it is basically random. We just need to debug, figure out the issues, and get the score in the same ballpark. It's probably something simple - I'm confident @lscheinkman or @cogmission will figure it out!

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 11, 2016

Sorry... I hit "delete" accidentally - this comment belongs above @subutai 's ^^

You're never going to get the same results in HTM.java as we don't share RNG implementations. While the Mersenne algorithm purports to be the same, one has to use the language neutral methods (the general nextInt() method) or else we can't get the same result. You guys use the more "fancy" methods and so there is no compatibility to be had.

Also, there is a faster algorithm which is easily portable to Python or C++ here... - you guys might consider using it?

cogmission commented Apr 11, 2016

Sorry... I hit "delete" accidentally - this comment belongs above @subutai 's ^^

You're never going to get the same results in HTM.java as we don't share RNG implementations. While the Mersenne algorithm purports to be the same, one has to use the language neutral methods (the general nextInt() method) or else we can't get the same result. You guys use the more "fancy" methods and so there is no compatibility to be had.

Also, there is a faster algorithm which is easily portable to Python or C++ here... - you guys might consider using it?

@subutai

This comment has been minimized.

Show comment
Hide comment
@subutai

subutai Apr 11, 2016

Member

You're never going to get the same results in HTM.java

I agree with you (please read my comment). However we do want to get decent results in HTM.java.

Member

subutai commented Apr 11, 2016

You're never going to get the same results in HTM.java

I agree with you (please read my comment). However we do want to get decent results in HTM.java.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 11, 2016

Awesome! :-) I'll never turn down care and lovin' :-P

cogmission commented Apr 11, 2016

Awesome! :-) I'll never turn down care and lovin' :-P

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 12, 2016

@lscheinkman any news on what's going on with HTM.java and NAB yet? No rush... I'm just anxious to see what the problem is? :-)

cogmission commented Apr 12, 2016

@lscheinkman any news on what's going on with HTM.java and NAB yet? No rush... I'm just anxious to see what the problem is? :-)

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Apr 12, 2016

Contributor

@cogmission Sorry for going slow on this PR, got busy on other tasks. I will try to debug this week and keep you posted.
On thing I've noticed was using KEY.MAX_BOOST parameter default value (10.0) instead of NAB's model value (1.0) improves performance but still not comparable to TM_CPP yet.

Contributor

lscheinkman commented Apr 12, 2016

@cogmission Sorry for going slow on this PR, got busy on other tasks. I will try to debug this week and keep you posted.
On thing I've noticed was using KEY.MAX_BOOST parameter default value (10.0) instead of NAB's model value (1.0) improves performance but still not comparable to TM_CPP yet.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Apr 12, 2016

@lscheinkman Thanks! Please keep me posted (I'm rooting for ya') 👍

cogmission commented Apr 12, 2016

@lscheinkman Thanks! Please keep me posted (I'm rooting for ya') 👍

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman May 27, 2016

Contributor

Once numenta/nupic#3138 is merged we should try the new TM parameters and see if we get similar results

Contributor

lscheinkman commented May 27, 2016

Once numenta/nupic#3138 is merged we should try the new TM parameters and see if we get similar results

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission commented May 28, 2016

@BoltzmannBrain

This comment has been minimized.

Show comment
Hide comment
@BoltzmannBrain

BoltzmannBrain Jun 1, 2016

Contributor

@lscheinkman @cogmission would one of you be able to try the new TM params?

Contributor

BoltzmannBrain commented Jun 1, 2016

@lscheinkman @cogmission would one of you be able to try the new TM params?

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman
Contributor

lscheinkman commented Jun 1, 2016

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Jun 3, 2016

Contributor

@cogmission @BoltzmannBrain @subutai
Here are the results with the new parameters:

    "htmjava": {
        "reward_low_FN_rate": 12.80037041235632,
        "reward_low_FP_rate": 3.7401232426724107,
        "standard": 9.28676251491379
    }
Contributor

lscheinkman commented Jun 3, 2016

@cogmission @BoltzmannBrain @subutai
Here are the results with the new parameters:

    "htmjava": {
        "reward_low_FN_rate": 12.80037041235632,
        "reward_low_FP_rate": 3.7401232426724107,
        "standard": 9.28676251491379
    }
@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Jun 3, 2016

@lscheinkman What do these mean? :-P

cogmission commented Jun 3, 2016

@lscheinkman What do these mean? :-P

@BoltzmannBrain

This comment has been minimized.

Show comment
Hide comment
@BoltzmannBrain

BoltzmannBrain Jun 3, 2016

Contributor

@cogmission these are to be compared with the results of the new TM detector (#240):

    "numentaTM": {
        "reward_low_FN_rate": 66.09146564859769,
        "reward_low_FP_rate": 52.389446324344824,
        "standard": 61.20616399012932
    },

We would expect the HTMJava detector to match these scores.

Contributor

BoltzmannBrain commented Jun 3, 2016

@cogmission these are to be compared with the results of the new TM detector (#240):

    "numentaTM": {
        "reward_low_FN_rate": 66.09146564859769,
        "reward_low_FP_rate": 52.389446324344824,
        "standard": 61.20616399012932
    },

We would expect the HTMJava detector to match these scores.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Jun 3, 2016

@lscheinkman So then, what's the conclusion? Do new TM parameters correspond to new TM code? or are they just new parameters? If they correspond to new code, then HTM.java doesn't have it... Otherwise, I don't know what to say - if HTM.java is unit test and integration test identical - then what gives? I wonder...?

Also, is the Python version using the old TP or new TM algorithm?

Just trying to account for any differences...

cogmission commented Jun 3, 2016

@lscheinkman So then, what's the conclusion? Do new TM parameters correspond to new TM code? or are they just new parameters? If they correspond to new code, then HTM.java doesn't have it... Otherwise, I don't know what to say - if HTM.java is unit test and integration test identical - then what gives? I wonder...?

Also, is the Python version using the old TP or new TM algorithm?

Just trying to account for any differences...

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Jun 3, 2016

Contributor

@cogmission AFAIK there is no new TM code, numenta/nupic#3138 only updates TM parameters.
Unit and integration tests validates the code implementation not the algorithm performance.

To test algorithm performance you need to compare the results from different implementation, and that's what NAB is doing.

One way to test the algorithm is to compare hotgym results and see if they are similar.
Currently nupic runs benchmarks tests where it validates the prediction results as part of its regression tests.
Another example can be found in HTM Studio where the test uses NAB results as ground truth.

CC: @subutai

Contributor

lscheinkman commented Jun 3, 2016

@cogmission AFAIK there is no new TM code, numenta/nupic#3138 only updates TM parameters.
Unit and integration tests validates the code implementation not the algorithm performance.

To test algorithm performance you need to compare the results from different implementation, and that's what NAB is doing.

One way to test the algorithm is to compare hotgym results and see if they are similar.
Currently nupic runs benchmarks tests where it validates the prediction results as part of its regression tests.
Another example can be found in HTM Studio where the test uses NAB results as ground truth.

CC: @subutai

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Jun 3, 2016

Unit and integration tests validates the code implementation not the algorithm performance.

It seems to me that the same algorithm written in different languages, if 100% determinant, should perform identically? ;-)

Sent from my iPhone

On Jun 3, 2016, at 5:41 PM, Luiz Scheinkman notifications@github.com wrote:

@cogmission AFAIK there is no new TM code, numenta/nupic#3138 only updates TM parameters.
Unit and integration tests validates the code implementation not the algorithm performance.

To test algorithm performance you need to compare the results from different implementation, and that's what NAB is doing.

One way to test the algorithm is to compare hotgym results and see if they are similar.
Currently nupic runs benchmarks tests where it validates the prediction results as part of its regression tests.
Another example can be found in HTM Studio where the test uses NAB results as ground truth.

CC: @subutai


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

cogmission commented Jun 3, 2016

Unit and integration tests validates the code implementation not the algorithm performance.

It seems to me that the same algorithm written in different languages, if 100% determinant, should perform identically? ;-)

Sent from my iPhone

On Jun 3, 2016, at 5:41 PM, Luiz Scheinkman notifications@github.com wrote:

@cogmission AFAIK there is no new TM code, numenta/nupic#3138 only updates TM parameters.
Unit and integration tests validates the code implementation not the algorithm performance.

To test algorithm performance you need to compare the results from different implementation, and that's what NAB is doing.

One way to test the algorithm is to compare hotgym results and see if they are similar.
Currently nupic runs benchmarks tests where it validates the prediction results as part of its regression tests.
Another example can be found in HTM Studio where the test uses NAB results as ground truth.

CC: @subutai


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Jun 3, 2016

Contributor

Assuming 100% test coverage and no differences between the implementations.

Basically this difference in the results indicates a bug somewhere not covered by the tests, either on my use of HTM Java in NAB or some implementation differences between java and python that would only manifest in certain situations.

We can start investigating by comparing hotgym results using both python and java implementation and later on move to other NAB results.

Contributor

lscheinkman commented Jun 3, 2016

Assuming 100% test coverage and no differences between the implementations.

Basically this difference in the results indicates a bug somewhere not covered by the tests, either on my use of HTM Java in NAB or some implementation differences between java and python that would only manifest in certain situations.

We can start investigating by comparing hotgym results using both python and java implementation and later on move to other NAB results.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Sep 29, 2016

@lscheinkman I'll defer to @subutai because it means NuPIC's scores have to be lowered too, no?

cogmission commented Sep 29, 2016

@lscheinkman I'll defer to @subutai because it means NuPIC's scores have to be lowered too, no?

@mrcslws

This comment has been minimized.

Show comment
Hide comment
@mrcslws

mrcslws Sep 29, 2016

Contributor

The "numenta" scores have also fallen. I'm investigating which commit caused it to drop. I think it was one of my recent changes to the SP.

Contributor

mrcslws commented Sep 29, 2016

The "numenta" scores have also fallen. I'm investigating which commit caused it to drop. I think it was one of my recent changes to the SP.

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 1, 2016

@mrcslws @lscheinkman I pushed changes to the build; which integrates the latest HTM.Java release. The new release internally sets the potentialRadius variable within the Network API, making the local changes to HTMModel which also set that variable, unnecessary.

cogmission commented Oct 1, 2016

@mrcslws @lscheinkman I pushed changes to the build; which integrates the latest HTM.Java release. The new release internally sets the potentialRadius variable within the Network API, making the local changes to HTMModel which also set that variable, unnecessary.

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Oct 1, 2016

Contributor

@cogmission The merge commit ddf61ee removed all NAB results from the repo.
Could you revert that one and submit only changes related to nab/detectors/htmjava and results/htmjava directories?

Contributor

lscheinkman commented Oct 1, 2016

@cogmission The merge commit ddf61ee removed all NAB results from the repo.
Could you revert that one and submit only changes related to nab/detectors/htmjava and results/htmjava directories?

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 2, 2016

@lscheinkman Done!

(I had to first learn how to do it!) 😉

cogmission commented Oct 2, 2016

@lscheinkman Done!

(I had to first learn how to do it!) 😉

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 16, 2016

@lscheinkman @everyone :-)

Aren't we going to merge this after the long bout in hell we spent getting this to work?

cogmission commented Oct 16, 2016

@lscheinkman @everyone :-)

Aren't we going to merge this after the long bout in hell we spent getting this to work?

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Oct 16, 2016

Contributor

@cogmission LGTM, I believe we are just waiting on #276. There is an open PR fixing it already. See #277

Let me know If you are ok with the current results in the README so we could merge this PR now and not wait for #276.

CC: @subutai

Contributor

lscheinkman commented Oct 16, 2016

@cogmission LGTM, I believe we are just waiting on #276. There is an open PR fixing it already. See #277

Let me know If you are ok with the current results in the README so we could merge this PR now and not wait for #276.

CC: @subutai

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 16, 2016

The scores should match very closely with NuPIC's. Can you re-run it using 0.6.10 (build.gradle version) and see if they match. They should be within 0.2 points I would say?

cogmission commented Oct 16, 2016

The scores should match very closely with NuPIC's. Can you re-run it using 0.6.10 (build.gradle version) and see if they match. They should be within 0.2 points I would say?

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Oct 19, 2016

Contributor

@cogmission The current scores already match the latest NumentaTM scores. See #276 for updated values.
The current values for NumentaTM in the README reflects the scores taken from nupic (numenta/nupic@42f701d) and nupic.core (numenta/nupic.core@c030b84) versions. PR #277 will update these values to the latest version.

I will leave up to you whether we should merge this PR before #277

Contributor

lscheinkman commented Oct 19, 2016

@cogmission The current scores already match the latest NumentaTM scores. See #276 for updated values.
The current values for NumentaTM in the README reflects the scores taken from nupic (numenta/nupic@42f701d) and nupic.core (numenta/nupic.core@c030b84) versions. PR #277 will update these values to the latest version.

I will leave up to you whether we should merge this PR before #277

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 20, 2016

@lscheinkman

I will leave up to you whether we should merge this PR before #277

Let's wait for #277 so that there is no unexplained difference in the scores, ok?

cogmission commented Oct 20, 2016

@lscheinkman

I will leave up to you whether we should merge this PR before #277

Let's wait for #277 so that there is no unexplained difference in the scores, ok?

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Oct 20, 2016

@rhyolight @mrcslws @lscheinkman Just a friendly reminder that Marcus was going to look into why the scores dropped (possibly due to a change in the SP?) - I don't think we should lose track of this task as its important to figure out... Not sure if this "reminder" is warranted or not because Marcus may have already looked into this?

cogmission commented Oct 20, 2016

@rhyolight @mrcslws @lscheinkman Just a friendly reminder that Marcus was going to look into why the scores dropped (possibly due to a change in the SP?) - I don't think we should lose track of this task as its important to figure out... Not sure if this "reminder" is warranted or not because Marcus may have already looked into this?

@mrcslws

This comment has been minimized.

Show comment
Hide comment
@mrcslws

mrcslws Nov 2, 2016

Contributor

@cogmission Here was the result of that investigation: #245 (comment)

The solution is probably to post the mean score across different random seeds.

Contributor

mrcslws commented Nov 2, 2016

@cogmission Here was the result of that investigation: #245 (comment)

The solution is probably to post the mean score across different random seeds.

@scottpurdy

This comment has been minimized.

Show comment
Hide comment
@scottpurdy

scottpurdy Nov 28, 2016

Contributor

See #287 for updated numentaTM results. There is a spatial detection change to the inherited NumentaDetector class that will need to be done in order to get similar results:

if spatialAnomaly:
finalScore = 1.0

Contributor

scottpurdy commented Nov 28, 2016

See #287 for updated numentaTM results. There is a spatial detection change to the inherited NumentaDetector class that will need to be done in order to get similar results:

if spatialAnomaly:
finalScore = 1.0

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Nov 29, 2016

Contributor

@cogmission I've updated the detector with @scottpurdy changes and now htm.java score is:

    "htmjava": {
        "reward_low_FN_rate": 70.42407766520115,
        "reward_low_FP_rate": 53.25694971610345,
        "standard": 65.54990960125
    }

I've updated the README.md with the latest values.

Please review and let me know if it is Ok to merge.

Contributor

lscheinkman commented Nov 29, 2016

@cogmission I've updated the detector with @scottpurdy changes and now htm.java score is:

    "htmjava": {
        "reward_low_FN_rate": 70.42407766520115,
        "reward_low_FP_rate": 53.25694971610345,
        "standard": 65.54990960125
    }

I've updated the README.md with the latest values.

Please review and let me know if it is Ok to merge.

@subutai

This comment has been minimized.

Show comment
Hide comment
@subutai
Member

subutai commented Nov 29, 2016

@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Nov 29, 2016

@lscheinkman @scottpurdy Indeed, very nice! I do believe a merge is in order!

cogmission commented Nov 29, 2016

@lscheinkman @scottpurdy Indeed, very nice! I do believe a merge is in order!

@lscheinkman lscheinkman merged commit 600a380 into numenta:master Nov 29, 2016

3 checks passed

Contributor Validator lscheinkman signed the Contributor License
Details
Fixes Issue Validator support repos don't require issues for PRs.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@cogmission

This comment has been minimized.

Show comment
Hide comment
@cogmission

cogmission Nov 29, 2016

@lscheinkman Sorry for the late comment, but shouldn't the code beginning at line 95 be above the call to the anomaly likelihood code at line 92 to avoid executing the likelihood code for efficiency (since the logScore = 1.0 value isn't affected by the call to the likelihood code?

cogmission commented Nov 29, 2016

@lscheinkman Sorry for the late comment, but shouldn't the code beginning at line 95 be above the call to the anomaly likelihood code at line 92 to avoid executing the likelihood code for efficiency (since the logScore = 1.0 value isn't affected by the call to the likelihood code?

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman

lscheinkman Nov 29, 2016

Contributor

@cogmission Good catch!
I will add new PR fixing this issue here and the numenta_detector.py

Contributor

lscheinkman commented Nov 29, 2016

@cogmission Good catch!
I will add new PR fixing this issue here and the numenta_detector.py

@lscheinkman

This comment has been minimized.

Show comment
Hide comment
@lscheinkman
Contributor

lscheinkman commented Nov 29, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment