Add FARMClassifier node for Document Classification #1265
Conversation
julian-risch
changed the title
haystack/classifier/base.py
Outdated
| return wrapper | ||
|
|
||
| def print_time(self): | ||
| print("Ranker (Speed)") |
haystack/classifier/base.py
Outdated
| return_preds: bool = False, | ||
| ) -> dict: | ||
| """ | ||
| Performs evaluation of the Ranker. |
There was a problem hiding this comment.
Let's remove the eval method from the base class if we don't have it implemented in any child class
| p = Pipeline() | ||
| p.add_node(component=retriever, name="ESRetriever", inputs=["Query"]) | ||
| p.add_node(component=classifier, name="Classifier", inputs=["ESRetriever"]) | ||
|
|
There was a problem hiding this comment.
Would it actually also work to use this node in an indexing pipeline? I mean something like FileConverter->Preprocessor->Classifier->DocStore
So we would basically append meta data to the docs at indexing time...
There was a problem hiding this comment.
Doesn't have to be part of this PR if it requires bigger changes, but maybe you can document what's missing for that use case and create a separate issue
There was a problem hiding this comment.
created an issue here: #1281
|
|
||
| - Take a plain language model (e.g. `bert-base-cased`) and train it for TextClassification | ||
| - Take a TextClassification model and fine-tune it for your domain | ||
|
|
There was a problem hiding this comment.
Please add some info about the expected format of the train file (csv, what columns ...)
haystack/classifier/farm.py
Outdated
| dev_split=dev_split, | ||
| test_filename=test_filename, | ||
| data_dir=Path(data_dir), | ||
| delimiter="\t" |
There was a problem hiding this comment.
Maybe we should also add the delimiter as an option to the init?
haystack/classifier/farm.py
Outdated
| """ | ||
| Use loaded classification model to classify the supplied list of Document. | ||
|
|
||
| Returns list of Document enriched with classification. |
There was a problem hiding this comment.
Please add here the info where the classification result is stored (Document.meta["classification"])
Proposed changes:
metafield**Status **: