ADAM -- Question Answering System
A question answering system that extracts answers from Wikipedia to questions posed in natural language. Inspired by IBM Watson and START. We are currently focused on improving the accuracy of the extracted answers. Follow the creator's blog at shirishkadam.com for updates on progress.
Elasticsearch is being used to store and index the scrapped and parsed texts from Wikipedia.
Elasticsearch 7.X installation guide can be found at Elasticsearch Documentation.
You might have to start the elasticsearch search service.
$ git clone https://github.com/5hirish/adam_qas.git $ cd adam_qas $ pip install -r requirements.txt $ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"
Note: The above installation downloads the best-matching default english language model for spaCy. But to improve the model's accuracy you can install other models too. Read more at spaCy docs.
$ python -m spacy download en_core_web_md
Running with Docker
$ git clone https://github.com/5hirish/adam_qas.git $ cd adam_qas $ docker-compose up
Now both conntainers are up and running. Next step is to enter in the python container and run Adam:
$ docker exec -it $(docker ps -a -q --filter ancestor=adam_qas_adam) bash $ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"
Find more in depth documentation about the system with its research paper and system architecture here
Python Package dependencies listed in requirements.txt Upgrading Elasticsearch 6.X:
- Extract information from Wikipedia
- Classify questions with regular expression (default)
- Classify questions with a SVM (optional)
- Vector space model used for answer extraction
- Rank candidate answers
- Merge top 5 answers into one response
Current Project State ?
- Replace Wikipedia APIs with custom scraper
- Storing extracted data in database (elasticsearch)
- SQLite test input data storage
- Anaphora resolution in both questions and answers
- Machine learning query constructor rather than rule-based
- Improve vector space language model for answer extraction
Please see our contributing documentation for some tips on getting started.
- @5hirish - Shirish Kadam