Mining - topic detection

[mjy]

• identify what's needed from the initial stream
• identify libraries or tools that will be used to isolate topics
• stream back topics (by definition this means a format for returning the results)

Bonus:

• topics are geographical entities
• topics can be classified as biological or not

Both these require text, and therefor space and bandwidth.

[herbherbherb]

Implemented topic model methods using tf-idf and LDA sklearn library and set up another grpc server example for topic modeling. One python client can now receives stream of documents/pages from BHL server and stream those pages to topic model server and get the corresponding topics back.
Code: https://github.com/SpeciesFileGroup/cs_492_fall/tree/topic_model_Herbert

Next Step: Working on extracting topics that are geographical entities. Play with Geo-entity extraction tool and attempt to extract geo-entity out of the document stream.
Geo-entity tool link: https://nlpforhackers.io/named-entity-extraction/