Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A minimal REST API for Qiita #2094

Merged
merged 46 commits into from
Mar 30, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
b806436
TST: Add initial test cases for study handler
ElDeveloper Mar 22, 2017
dabad2f
ENH: Add initial study rest api
ElDeveloper Mar 22, 2017
a9f4eeb
API: test if a study exists
wasade Mar 22, 2017
5e73446
ENH: oauth2 forced
wasade Mar 23, 2017
ab8a067
Get back basic study deets
wasade Mar 23, 2017
64862f2
TST: test for samples collection
wasade Mar 23, 2017
748a49f
API: rest get sample IDs from a study
wasade Mar 23, 2017
346fd2f
ENH: samples/info handler
wasade Mar 23, 2017
dc0f7c6
broken routes
wasade Mar 23, 2017
553f816
API: request sample metadata
wasade Mar 24, 2017
6df3420
ENH/API: Add methods to check for a study person
ElDeveloper Mar 24, 2017
9981b06
ENH/API: Add POST methods for study person
ElDeveloper Mar 24, 2017
99a8eb8
TST: Add tests for from_name_and_affiliation
ElDeveloper Mar 24, 2017
3888571
TST: study creation
wasade Mar 24, 2017
a77736c
BUG: Add headers to tests
ElDeveloper Mar 24, 2017
69c63c3
Merge branch 'death-march' of github.com:eldeveloper/qiita into death…
wasade Mar 24, 2017
6b2e3f6
ENH: create study
wasade Mar 24, 2017
cee5dce
Adjust GET on study description
wasade Mar 27, 2017
e19e6fa
API: Add endpoints for preparation creation
ElDeveloper Mar 28, 2017
6ad2ef2
TST: 200 :D
ElDeveloper Mar 28, 2017
a0f7463
TST: Correctly verify study instantiation
ElDeveloper Mar 28, 2017
9218611
TST: prep artifact creation
wasade Mar 28, 2017
d3fb608
ENH/API: associate artifacts with a preparation
wasade Mar 28, 2017
cfe09d6
TST: test study statys
wasade Mar 28, 2017
9c3d42a
ENH: study status
wasade Mar 28, 2017
ca34017
Removed trailing whitespace
wasade Mar 28, 2017
fd54712
STY: PEP8
ElDeveloper Mar 28, 2017
43e133e
Merge branch 'death-march' of github.com:ElDeveloper/qiita into death…
ElDeveloper Mar 28, 2017
a18b5d8
MAINT: refactor, centralize setup boilerplate
wasade Mar 28, 2017
b32d906
REFACTOR: Remove repeated code
ElDeveloper Mar 28, 2017
13eb32c
DOC: Remove unnecessary comments
ElDeveloper Mar 28, 2017
0872833
REFACTOR: Missing removal of pattern
ElDeveloper Mar 28, 2017
09f06bd
STY: Fix PEP8 errors
ElDeveloper Mar 29, 2017
d4e010c
BUG: Incorrectly changed error code
ElDeveloper Mar 29, 2017
53b3803
BUG/TST: Fix typo in tests
ElDeveloper Mar 29, 2017
1a3808c
Addressing an @antgonza comment
wasade Mar 29, 2017
23c9993
Another @antgonza comment
wasade Mar 29, 2017
e460390
Merge branch 'death-march' of github.com:eldeveloper/qiita into death…
wasade Mar 29, 2017
e76e82a
RVW: Address review comments
ElDeveloper Mar 30, 2017
3f58dcd
Merge branch 'death-march' of github.com:ElDeveloper/qiita into death…
ElDeveloper Mar 30, 2017
f243a1d
ENH: Cleanup webserver and name-spaces
ElDeveloper Mar 30, 2017
595d9a1
ENH: Improve error messages
ElDeveloper Mar 30, 2017
e337632
ENH: Add more descriptive error message
ElDeveloper Mar 30, 2017
9f528ea
TST: Exercise different argument types
ElDeveloper Mar 30, 2017
e970392
DOC: Add documentation for REST API
ElDeveloper Mar 30, 2017
133b9f8
ENH: Remove extra comma
ElDeveloper Mar 30, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions qiita_db/study.py
Original file line number Diff line number Diff line change
Expand Up @@ -1213,6 +1213,33 @@ def exists(cls, name, affiliation):
qdb.sql_connection.TRN.add(sql, [name, affiliation])
return qdb.sql_connection.TRN.execute_fetchlast()

@classmethod
def from_name_and_affiliation(cls, name, affiliation):
"""Gets a StudyPerson object based on the name and affiliation

Parameters
----------
name: str
Name of the person
affiliation : str
institution with which the person is affiliated

Returns
-------
StudyPerson
The StudyPerson for the name and affiliation
"""
with qdb.sql_connection.TRN:
if not cls.exists(name, affiliation):
raise qdb.exceptions.QiitaDBLookupError(
'Study person does not exist')

sql = """SELECT study_person_id FROM qiita.{0}
WHERE name = %s
AND affiliation = %s""".format(cls._table)
qdb.sql_connection.TRN.add(sql, [name, affiliation])
return cls(qdb.sql_connection.TRN.execute_fetchlast())

@classmethod
def create(cls, name, email, affiliation, address=None, phone=None):
"""Create a StudyPerson object, checking if person already exists.
Expand Down
13 changes: 13 additions & 0 deletions qiita_db/test/test_study.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,19 @@ def test_delete(self):
self.assertFalse(
qdb.study.StudyPerson.exists('SomeDude', 'affil'))

def test_retrieve_non_existant_people(self):
with self.assertRaises(qdb.exceptions.QiitaDBLookupError):
qdb.study.StudyPerson.from_name_and_affiliation('Boaty McBoatFace',
'UCSD')

p = qdb.study.StudyPerson.from_name_and_affiliation('LabDude',
'knight lab')
self.assertEqual(p.name, 'LabDude')
self.assertEqual(p.affiliation, 'knight lab')
self.assertEqual(p.address, '123 lab street')
self.assertEqual(p.phone, '121-222-3333')
self.assertEqual(p.email, 'lab_dude@foo.bar')

def test_iter(self):
"""Make sure that each and every StudyPerson is retrieved"""
expected = [
Expand Down
35 changes: 35 additions & 0 deletions qiita_pet/handlers/rest/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# -----------------------------------------------------------------------------
# Copyright (c) 2014--, The Qiita Development Team.
#
# Distributed under the terms of the BSD 3-clause License.
#
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

from .study import StudyHandler, StudyCreatorHandler, StudyStatusHandler
from .study_samples import (StudySamplesHandler, StudySamplesInfoHandler,
StudySamplesCategoriesHandler)
from .study_person import StudyPersonHandler
from .study_preparation import (StudyPrepCreatorHandler,
StudyPrepArtifactCreatorHandler)


__all__ = ['StudyHandler', 'StudySamplesHandler', 'StudySamplesInfoHandler',
'StudySamplesCategoriesHandler', 'StudyPersonHandler',
'StudyCreatorHandler', 'StudyPrepCreatorHandler',
'StudyPrepArtifactCreatorHandler', 'StudyStatusHandler']


ENDPOINTS = (
(r"/api/v1/study$", StudyCreatorHandler),
(r"/api/v1/study/([0-9]+)$", StudyHandler),
(r"/api/v1/study/([0-9]+)/samples/categories=([a-zA-Z\-0-9\.:,_]*)",
StudySamplesCategoriesHandler),
(r"/api/v1/study/([0-9]+)/samples", StudySamplesHandler),
(r"/api/v1/study/([0-9]+)/samples/info", StudySamplesInfoHandler),
(r"/api/v1/person(.*)", StudyPersonHandler),
(r"/api/v1/study/([0-9]+)/preparation/([0-9]+)/artifact",
StudyPrepArtifactCreatorHandler),
(r"/api/v1/study/([0-9]+)/preparation(.*)", StudyPrepCreatorHandler),
(r"/api/v1/study/([0-9]+)/status$", StudyStatusHandler)
)
31 changes: 31 additions & 0 deletions qiita_pet/handlers/rest/rest_handler.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# -----------------------------------------------------------------------------
# Copyright (c) 2014--, The Qiita Development Team.
#
# Distributed under the terms of the BSD 3-clause License.
#
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------
from qiita_db.study import Study
from qiita_db.exceptions import QiitaDBUnknownIDError
from qiita_pet.handlers.util import to_int
from qiita_pet.handlers.base_handlers import BaseHandler


class RESTHandler(BaseHandler):
def fail(self, msg, status, **kwargs):
out = {'message': msg}
out.update(kwargs)

self.write(out)
self.set_status(status)
self.finish()

def safe_get_study(self, study_id):
study_id = to_int(study_id)
s = None
try:
s = Study(study_id)
except QiitaDBUnknownIDError:
self.fail('Study not found', 404)
finally:
return s
154 changes: 154 additions & 0 deletions qiita_pet/handlers/rest/study.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# -----------------------------------------------------------------------------
# Copyright (c) 2014--, The Qiita Development Team.
#
# Distributed under the terms of the BSD 3-clause License.
#
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------
import warnings

from tornado.escape import json_decode

from qiita_db.handlers.oauth2 import authenticate_oauth
from qiita_db.study import StudyPerson, Study
from qiita_db.user import User
from .rest_handler import RESTHandler
from qiita_db.metadata_template.constants import SAMPLE_TEMPLATE_COLUMNS


class StudyHandler(RESTHandler):

@authenticate_oauth
def get(self, study_id):
study = self.safe_get_study(study_id)
if study is None:
return

info = study.info
pi = info['principal_investigator']
lp = info['lab_person']
self.write({'title': study.title,
'contacts': {'principal_investigator': [
pi.name,
pi.affiliation,
pi.email],
'lab_person': [
lp.name,
lp.affiliation,
lp.email]},
'study_abstract': info['study_abstract'],
'study_description': info['study_description'],
'study_alias': info['study_alias']})
self.finish()


class StudyCreatorHandler(RESTHandler):

@authenticate_oauth
def post(self):
try:
payload = json_decode(self.request.body)
except ValueError:
self.fail('Could not parse body', 400)
return

required = {'title', 'study_abstract', 'study_description',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will not require efo I will just hardcode it to 1 as it is done everywhere else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I've just removed this.

'study_alias', 'owner', 'contacts'}

if not required.issubset(payload):
self.fail('Not all required arguments provided', 400)
return

title = payload['title']
study_abstract = payload['study_abstract']
study_desc = payload['study_description']
study_alias = payload['study_alias']

owner = payload['owner']
if not User.exists(owner):
self.fail('Unknown user', 403)
return
else:
owner = User(owner)

contacts = payload['contacts']

if Study.exists(title):
self.fail('Study title already exists', 409)
return

pi_name = contacts['principal_investigator'][0]
pi_aff = contacts['principal_investigator'][1]
if not StudyPerson.exists(pi_name, pi_aff):
self.fail('Unknown principal investigator', 403)
return
else:
pi = StudyPerson.from_name_and_affiliation(pi_name, pi_aff)

lp_name = contacts['lab_person'][0]
lp_aff = contacts['lab_person'][1]
if not StudyPerson.exists(lp_name, lp_aff):
self.fail('Unknown lab person', 403)
return
else:
lp = StudyPerson.from_name_and_affiliation(lp_name, lp_aff)

info = {'lab_person_id': lp,
'principal_investigator_id': pi,
'study_abstract': study_abstract,
'study_description': study_desc,
'study_alias': study_alias,

# TODO: we believe it is accurate that mixs is false and
# metadata completion is false as these cannot be known
# at study creation here no matter what.
# we do not know what should be done with the timeseries.
'mixs_compliant': False,
'metadata_complete': False,
'timeseries_type_id': 1}
study = Study.create(owner, title, [1], info)

self.set_status(201)
self.write({'id': study.id})
self.finish()


class StudyStatusHandler(RESTHandler):
@authenticate_oauth
def get(self, study_id):
study = self.safe_get_study(study_id)
if study is None:
return

public = study.status == 'public'
st = study.sample_template
sample_information = st is not None
if sample_information:
with warnings.catch_warnings():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these warnings are stored in redis at the template generation time if it is done through the GUI.
I think this code is fine as it is not ensured that these warnings exist somewhere. I've created an issue so we can avoid code duplication. Can you add a comment with the issue ID so it is easy to search for later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for creating that issue, I just added a link to it.

try:
st.validate(SAMPLE_TEMPLATE_COLUMNS)
except Warning:
sample_information_warnings = True
else:
sample_information_warnings = False
else:
sample_information_warnings = False

preparations = []
for prep in study.prep_templates():
pid = prep.id
art = prep.artifact is not None
# TODO: unclear how to test for warnings on the preparations as
# it requires knowledge of the preparation type. It is possible
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what this refers to? data_type, information_type, other?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to validate requires knowledge of what the preparation actually is, and we didn't want to replicate a bunch of functionality for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it! Depending on the data_type is how the validation happen. One simply option is to return all the validations avaialble? Not pretty but this will allow us to test and figure out a better solution. Other option will be to force the user to pass that, I guess a similar issue than artifact_type/id. BTW I think this is important cause the first use case will be either target gene or shotgun sequencing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need to hardcode the variable names of all the prep template constants which can be passed to validate, unless there is a mapping somewhere already? It would also make the response from this API variable as different requirements are added in which could easily cause issue with consumers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be englobed here: #2096

# to tease this out, but it replicates code present in
# PrepTemplate.create, see:
# https://github.com/biocore/qiita/issues/2096
preparations.append({'id': pid, 'has_artifact': art})

self.write({'is_public': public,
'has_sample_information': sample_information,
'sample_information_has_warnings':
sample_information_warnings,
'preparations': preparations})
self.set_status(200)
self.finish()
49 changes: 49 additions & 0 deletions qiita_pet/handlers/rest/study_person.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# -----------------------------------------------------------------------------
# Copyright (c) 2014--, The Qiita Development Team.
#
# Distributed under the terms of the BSD 3-clause License.
#
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

from qiita_db.handlers.oauth2 import authenticate_oauth
from qiita_db.study import StudyPerson
from qiita_db.exceptions import QiitaDBLookupError
from .rest_handler import RESTHandler


class StudyPersonHandler(RESTHandler):
@authenticate_oauth
def get(self, *args, **kwargs):
name = self.get_argument('name')
affiliation = self.get_argument('affiliation')

try:
p = StudyPerson.from_name_and_affiliation(name, affiliation)
except QiitaDBLookupError:
self.fail('Person not found', 404)
return

self.write({'address': p.address, 'phone': p.phone, 'email': p.email,
'id': p.id})
self.finish()

@authenticate_oauth
def post(self, *args, **kwargs):
name = self.get_argument('name')
affiliation = self.get_argument('affiliation')
email = self.get_argument('email')

phone = self.get_argument('phone', None)
address = self.get_argument('address', None)

if StudyPerson.exists(name, affiliation):
self.fail('Person already exists', 409)
return

p = StudyPerson.create(name=name, affiliation=affiliation, email=email,
phone=phone, address=address)

self.set_status(201)
self.write({'id': p.id})
self.finish()
Loading