CogALg REVIEW

[helalyrabiy]

Hello Boris, the last few days I’ve been looking at your github repo. I really love the work and ideas you are trying to implement. Here is my review on both code and algorithm.

1. Algorithm

• I really admire the idea of finding patterns and deep patterns through the calculation of form match and loss, after that those patterns will be introduced to some clustering algos (though here the clustering will be based on what? I mean will it be a density based? Spacial based? …), but here the first question that came up, how exactly the algorithm will behave in complex tasks? I would really know more about this.
• From my understanding, every pattern is not like a feature (please correct me if I’m wrong) it does not represent some characteristic on the studied problem but it represents a way of behaving or pattern of behaving (I don’t know exactly how to say it), and every pattern will be cross-compared, here what I love is that every physical and non-physical problem could be represented based on your algorithm as a series of pattern. Theoretically if we understand that pattern we will be able to predict the way it behaves, here what I want to know is how you are planning to represent non-quantitative things, (an image is a set of pixels, a sound is a set of frequencies, even text could be vectorized into sequence of vectors…). I’m very curious about this so it will be so great if you explain it to me more.
• There are indeed some details that I didn’t understand quite good such as the clustering methods you are using….

2. Code

• After a run through the code, I found it a bit difficult to keep up with the logic used, its well documented but with the big number of variable and params I found myself every time obliged to reread every parameter notation, wouldn’t be more efficient to use a pep of python so everyone can understand the datils more easily?
• I didn’t find any tests on the repo, are you planning to make some maybe Unit testing?
• Do you think that using only python will make the application slow? If yes why not using numpy arrays instead of lists and dicts to gain more performance and less memory usage
• Do you thought of integrating multithreading or why not cuda to gain more speed?
  Again I really admire the work you guys are doing and I look further to discuss more details about the algorithm and the implementation.
  Thank you very much for this opportunity.

[boris-kz]

Thank a lot, appreciate your interest! Sorry, don't quite know how to address you?

Re clustering, it will be by nearest above-average match, computed in cross-comp. Nearest positionally, in the order of input, not in terms of similarity. So 1st level patterns are contiguous, and higher-level patterns will be increasingly sparse graphs.
This is connectivity clustering: https://en.wikipedia.org/wiki/Cluster_analysis#Connectivity-based_clustering_(hierarchical_clustering), forming structured representations, vs. effectively centroid-based clustering in ANN.
Basically lateral vs. vertical clustering, please see comparison to ANN and BNN section.

Re tasks, this algorithm has only one: pattern discovery. Where complexity is incremental with elevation: higher-level inputs are lower-level patterns. So, both cross-comp and clustering will become more complex and selective.

Re features, well, it's how you define them. I don't use the term because it's too ambiguous, and I think in terms inputs not problems. The only problem I care about is unsupervised learning. They define it through the negative (unsupervised) because they don't know what the positive is. I think I do: pattern discovery.

Re non-quantitative things, I don't think they exist. It's just quantitative things that have been encoded beyond recognition.
I will reply to code questions latter.

[boris-kz]

About code,

Sorry about the number of variables, I don't know how to reduce it without losing critical info. The easiest way to remember them is through understanding how and why they were formed, mainly in comparison process. My notation is consistent across the project, so it will get easier to follow. I did a version of line_patterns in conventional style long time ago, but to me it just looks ridiculous: https://github.com/boris-kz/CogAlg/blob/master/line_1D_alg/alternative%20versions/line_introductory_old.py

Re tests, we have images folder for 2D alg, but it doesn't show much. The alg is far from complete, testing won't be very meaningful for some time.

We do use numpy in 2D alg, especially frame_blobs and intra_comp. It doesn't make any difference in 1D alg. Performance is not a priority yet, we need to define functionality first. So, we will stick with Python for predicable future. And then it should be easy to convert it to C with PyPy, we already tried.

Thanks again!