Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Wit.ai document type determiner #63

Merged
merged 23 commits into from Oct 28, 2017

Conversation

Projects
None yet
3 participants
@Awinja-Andela
Copy link
Contributor

commented Oct 13, 2017

We are currently using keywords to determine which document type the application should query from, which limits to the application to some words. Wit.ai will identify key words in queries and determine which document type it should query.

This PR therefore introduces:

  • wit.ai message API which returns the extracted meaning from a sentence, based on the app data.

This task involved:

  • opening a wit.ai account
  • Training wit.ai to check for the 8 entities that health tools uses which are:
    doctors, nurses, clinical-officers,health-facilities,nhif-inpatient,nhif-outpatient,nhif-outpatient-cs and query

wit.ai is suitable for the /search endpoint and further SMS function

It will enable users do searches like:
http://127.0.0.1:5000/search?q=is Doctor Jane Wairimu registered?

screen shot 2017-10-16 at 11 49 28 am

or send texts like:
where are civil servants outpatient services offered in kerugoya?

screen shot 2017-10-16 at 11 50 04 am

Relevant pivotal tracker stories

#51

This checks the doc type against the doc list an determines whether it should be elastic search or nurses search.
If the doc type is empty,search type is None
"""
doc = ['nhif-outpatient', 'nhif-inpatient', 'nhif-outpatient-cs', 'doctors', 'health-facilities', 'clinical-officers']

This comment has been minimized.

Copy link
@RyanSept

RyanSept Oct 14, 2017

Contributor

A bit of a nitpick but lines 30-33 aren't pep8

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

Nitpick is so important in these uncertain times.. 💯

'result': {'hits': [], 'total': 0},
'doc_type': doc_type,
'status': 'FAILED',
'msg': '' # TODO: Pass run_query message here.

This comment has been minimized.

Copy link
@RyanSept

RyanSept Oct 15, 2017

Contributor

Why do we have this TODO here?

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

That's me. We should pass an actual message if this fails. Currently we don't have any message being returned so we only know it's failed but not why.

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

Wait, @Awinja-Andela why are these changes reflecting here? I thought I had already written this code.

This comment has been minimized.

Copy link
@Awinja-Andela

Awinja-Andela Oct 16, 2017

Author Contributor

@DavidLemayian i am reusing this code for wit.ai. I am using it just so that we can have two options for searching for the document name.
using wit.ai or using document.py
It is either i do this, or i remove document.py completely.

'msg': '' # TODO: Pass run_query message here.
})

# TODO: Log event here (send to Google Analytics)

This comment has been minimized.

Copy link
@RyanSept

RyanSept Oct 15, 2017

Contributor

This one too

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

This is me again. Notes in code to get back to. Showing where we should do stuff.



blueprint = Blueprint('search_api', __name__)


@blueprint.route('/search', methods=['GET'], strict_slashes=False)
def search_index(doc_type=None):

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

@Awinja-Andela Why are we creating a separate function for what's already below as "index"?

This comment has been minimized.

Copy link
@Awinja-Andela

Awinja-Andela Oct 16, 2017

Author Contributor

@DavidLemayian notice that i have introduced :
from healthtools.search.wit_ai import wit_run_query
while still retaining:
from healthtools.search import run_query

@blueprint.route('/wit_search', methods=['GET'], strict_slashes=False)
that uses:
result, doc_type = wit_run_query(query, doc_type)

and
@blueprint.route('/search', methods=['GET'], strict_slashes=False) @blueprint.route('/search/<doc_type>', methods=['GET'], strict_slashes=False)
that still uses:
result, doc_type = run_query(query, doc_type)


#wit.ai access token

access_token = os.getenv("access_token")

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 15, 2017

Member

Formating and naming on these changes do not follow the rest of the project..

@Awinja-Andela Awinja-Andela changed the title Wit.ai Added Wit.ai document type determiner Oct 16, 2017

Awinja-Andela added some commits Oct 16, 2017

resp = client.message(message_text)
query = ''.join(nested_lookup('value', resp['entities']['query']))
doc_type = ''.join([var for var in (resp['entities'].keys()) if var != 'query'])
#wit returns the entity with an underscore,this changes it to hyphen

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 18, 2017

Member

When you see it.

search_type = 'nurses'
else:
search_type = 'elastic'
return search_type

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 18, 2017

Member

This is already defined in the DOCUMENTS. Why an entire function for it? https://github.com/CodeForAfricaLabs/HealthTools.API/blob/develop/healthtools/documents.py


################################################################################

#Wit.ai Access Token

This comment has been minimized.

Copy link
@DavidLemayian

DavidLemayian Oct 18, 2017

Member

When you see it.

@DavidLemayian

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

@Awinja-Andela I can kind of see what's happening here now.

So as to avoid replication, let's approach this as a doc-type.

When you search using https://api.healthtools/search/wit?q=Dr+Ouma it then determines the doc type using wit.ai. Make sense?

@Awinja-Andela

This comment has been minimized.

Copy link
Contributor Author

commented Oct 19, 2017

@DavidLemayian yes. i see. let me try it out.

@Awinja-Andela

This comment has been minimized.

Copy link
Contributor Author

commented Oct 19, 2017

@DavidLemayian check it out!!!

@DavidLemayian

This comment has been minimized.

Copy link
Member

commented Oct 25, 2017

This is shaping up nicely @Awinja-Andela. Now what I'd like is for determine_doc_type be the only function wit_ai runs. It should return only the query and doc_type. Makes sense?

@Awinja-Andela

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2017

@DavidLemayian Makes Sense. I have changed it to return query and doc_type

@Awinja-Andela

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2017

@DavidLemayian you will see a lot has changed now.
wit_ai.py is no more.
there's a new function in query.py
and run_query fucntion has changed to accomodate the new function
And search_api.py no longer checks if doc_type is 'wit'

Awinja-Andela and others added some commits Oct 27, 2017

@DavidLemayian DavidLemayian merged commit a6aeeb9 into CodeForAfrica:develop Oct 28, 2017

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.