Annotation api #65

ir4y · 2017-02-16T05:51:00Z

No description provided.

Add regex field Allow blank column field

Suor · 2017-02-17T08:34:37Z

starapi/serializers.py

+        samples = data['series'].samples.all()
+
+        gsm_to_id = {s.gsm_name.upper(): s.id for s in samples}
+        id_to_gsm = {v: k for k, v in gsm_to_id.iteritems()}


Remove support for ids, only work with gsms. Do not play with case.

There is also funcy.flip() function for this. But you won't need this code at all )

Suor · 2017-02-17T08:38:10Z

starapi/serializers.py

+            raise serializers.ValidationError(unicode(err))
+
+        tagged_samples_ids = set(values.keys())
+        all_samples_ids = set(data['series'].samples.values_list('id', flat=True))


Totally unneeded query. Use samples from above. Or use .values_list() above.

Suor · 2017-02-17T08:39:07Z

starapi/viewsets.py

@@ -2,24 +2,28 @@

 from cacheops import cached_as
 from django.http import JsonResponse, HttpResponse
-from funcy import walk_keys
+from django.db import transaction
+from funcy import walk_keys, first


Return empty line.

Suor · 2017-02-17T08:41:45Z

starapi/viewsets.py

+        series = serializer.validated_data['series']
+        tag = serializer.validated_data['tag']
+
+        series_tag = first(SeriesTag.objects.filter(series=series, tag=tag))


Use Django's QuerySet.first().

Also specify .order_by('id'), now this code is unstable.

Also MAJOR error not filtering by platform here. SeriesTag is defined by these 3 things: serie, platform, tag. And filter by is_active=True.

Suor · 2017-02-17T08:43:33Z

tags/models.py

@@ -83,6 +83,8 @@ class SeriesTag(models.Model):
    # (with whatever previous one, not necessarily initial)
    agreed = models.IntegerField(blank=True, null=True)
    fleiss_kappa = models.FloatField(blank=True, null=True)
+    comment = models.CharField(max_length=512, blank=True, null=True)


Should be TextField. Never use CharField when you are not sure about max length.

And change "comment" -> "note".

Suor · 2017-02-17T09:16:39Z

tags/annotate_views.py

+        data = serializer.validated_data
+
+        with transaction.atomic():
+            old_annotation = first(SeriesTag.objects.filter(series=data['series'], tag=data['tag']))


use QuerySet.first() and .order_by('id').

Note that while we allow tag several platforms at once in UI, I don't want to allow that in API.

Also this code should be in save_annotation() not duplicated here.

Suor · 2017-02-17T09:23:15Z

tags/misc.py

@@ -0,0 +1,61 @@
+from funcy import group_by


rename to annotate_core.py

Suor · 2017-02-17T09:30:09Z

tags/misc.py

+        sa.fill_samples(sample_tags)
+
+
+def save_validation(user_id, series_tag, data, from_api=False):


Incorporate user_id and series_tag_id into data, same with save_annotation().

Do not query series_tag. Use serie_id, platform_id and tag_id passed from user/API.

Suor · 2017-02-17T09:30:39Z

tags/misc.py

+            series_id=series_tag.series_id,
+            platform_id=series_tag.platform_id, tag_id=series_tag.tag_id,
+            from_api=from_api, comment=data['comment'],
+        )


This could be simple project(data, [...]).

Suor · 2017-02-17T09:32:22Z

starapi/viewsets.py

+                raise ValidationError(
+                    {'non_field_errors': ["You can't validate your own annotation"]})
+
+            res, err = save_validation(user_id, series_tag, serializer.validated_data, True)


You get orphan ValidationJob here, clean it up.

Use GSM for consistency calculation. Use id instead of objects as annotate_core functions params Use exceptons instead of function pair result on_demand_validate is now usung save_validation from annotate_core

ir4y · 2017-03-03T09:59:16Z

Hi @iDrDex
We have released sample annotations API.

You can create sample annotations by POST query to the http://stargeo.org/api/sample_annotations/ endpoint.

Also, you need to sign your request with the token. You can find token at the header of swagger docs http://stargeo.org/docs/

The example of annotation

{
  "series": "GSE77818",
  "platform": "GPL10787",
  "tag": 22,
  "note": "This annotation was created bu API request",
  "annotations": {
    "GSM2060029": "1",
    "GSM2060030": "1",
    "GSM2060031": "1",
    "GSM2060032": "1",
    "GSM2060033": "1",
    "GSM2060034": "1",
    "GSM2060035": "1",
    "GSM2060036": "1"
  }
}

It is an example how you can send annotation with curl

curl -H "Content-Type: application/json" -H "Authorization: Token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -X POST -d '{
  "series": "GSE77818",
  "platform": "GPL10787",
  "tag": 22,
  "note": "This annotation was created bu API request",
  "annotations": {
    "GSM2060029": "1",
    "GSM2060030": "1",
    "GSM2060031": "1",
    "GSM2060032": "1",
    "GSM2060033": "1",
    "GSM2060034": "1",
    "GSM2060035": "1",
    "GSM2060036": "1"
  }
}'  http://stargeo.org/api/sample_annotations/

It is an example how you can send annotation with requests

In [1]: import requests

In [2]: data = {
   ...:   "series": "GSE77818",
   ...:   "platform": "GPL10787",
   ...:   "tag": 22,
   ...:   "note": "ololol",
   ...:   "annotations": {
   ...:     "GSM2060029": "1",
   ...:     "GSM2060030": "1",
   ...:     "GSM2060031": "1",
   ...:     "GSM2060032": "1",
   ...:     "GSM2060033": "1",
   ...:     "GSM2060034": "1",
   ...:     "GSM2060035": "1",
   ...:     "GSM2060036": "1"
   ...:   }
   ...: }

In [3]: headers = {'Content-Type': 'application/json', 'Authorization': 'Token XXXXXXXXXX
   ...: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'}

In [4]: requests.post('http://stargeo.org/api/sample_annotations/', data=data, header
   ...: s=headers)

iDrDex · 2017-03-03T22:26:01Z

Awesome. Thx!

iDrDex · 2017-04-30T05:44:01Z

@ir4y this code fails with <Response [400]>:

import requests

data = {
    "series": "GSE77818",
    "platform": "GPL10787",
    "tag": 22,
    "note": "ololol",
    "annotations": {
        "GSM2060029": "1",
        "GSM2060030": "1",
        "GSM2060031": "1",
        "GSM2060032": "1",
        "GSM2060033": "1",
        "GSM2060034": "1",
        "GSM2060035": "1",
        "GSM2060036": "1"
    }
}

headers = {
    'Content-Type': 'application/json',
    'Authorization': 'Token XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
}

print requests.post(
    'http://stargeo.org/api/sample_annotations/', 
    data=data,
    headers=headers
)

ir4y · 2017-04-30T08:25:39Z

Hi @iDrDex !

The token is a confidential information.
With this token anybody can perform actions from your name. Please don't share it at public posts anymore.
I have revoked it, so you need to get a new one from http://stargeo.org/docs/
Please don't share it at public posts, you can provide me a user name and I will get a token from the database.

ir4y · 2017-04-30T08:40:39Z

About the issue.

There was an issue with my code sample.
You need to dump python dict to JSON before sending a request.

res = requests.post(
    'http://stargeo.org/api/sample_annotations/', 
    data=json.dumps(data),
    headers=headers
)

FYI
The res object contains all information about response, including error message in case of error.

print(res.status_code, res.content)

iDrDex · 2017-04-30T21:00:52Z

Awesome. Thx @ir4y. I can definitely push annotations now. However, the counter on the home page does not reflect the API pushes. Can we fix that?

ir4y · 2017-05-01T07:27:02Z

@iDrDex
You need to wait for a while.
Updating home page counters run periodically.

ir4y added 3 commits February 10, 2017 17:13

Enable token authentication

e979cff

WIP Implementing API for creating series annoation

6611aec

Implement annotatation validation by API

a78d2fb

ir4y requested a review from Suor February 16, 2017 05:51

ir4y added 9 commits February 16, 2017 13:25

Add validation check that annotation must be full

59f81d1

Format validation errors

d91ce99

Rename create tokens migration

890b18e

Add comment and from_api fields

8fe7ed8

Move create logic from serializer to view

3352d3e

Format errors

0c7825d

Use serializer for annotate form validation

af5e9d5

Add regex field Allow blank column field

Hide token information for anonymous users

16a970a

Use GSM as sample id

0b5c855

Suor requested changes Feb 17, 2017

View reviewed changes

ir4y added 8 commits March 2, 2017 12:55

Refactor after review, ref #65

326e158

Filter samples by platform, ref #65

64a73ca

Revactor save_validation, ref #65

a229e54

Style fix, ref #65

85c2161

Set default value of note to empty string, ref #65

aa1d7a1

Refactor core annotate methods, ref #65

7753ed8

Use GSM for consistency calculation. Use id instead of objects as annotate_core functions params Use exceptons instead of function pair result on_demand_validate is now usung save_validation from annotate_core

Cleanup imports

c60582c

Fixes after review, ref #65

423af03

ir4y merged commit 576bcaa into master Mar 3, 2017

ir4y deleted the annotation-api branch July 25, 2017 06:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Annotation api #65

Annotation api #65

ir4y commented Feb 16, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

Suor Feb 17, 2017

ir4y commented Mar 3, 2017

iDrDex commented Mar 3, 2017

iDrDex commented Apr 30, 2017 •

edited

ir4y commented Apr 30, 2017 •

edited

ir4y commented Apr 30, 2017

iDrDex commented Apr 30, 2017

ir4y commented May 1, 2017

		sa.fill_samples(sample_tags)


		def save_validation(user_id, series_tag, data, from_api=False):

Annotation api #65

Annotation api #65

Conversation

ir4y commented Feb 16, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ir4y commented Mar 3, 2017

iDrDex commented Mar 3, 2017

iDrDex commented Apr 30, 2017 • edited

ir4y commented Apr 30, 2017 • edited

ir4y commented Apr 30, 2017

iDrDex commented Apr 30, 2017

ir4y commented May 1, 2017

iDrDex commented Apr 30, 2017 •

edited

ir4y commented Apr 30, 2017 •

edited