Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

location data quick fix #151

Merged
merged 4 commits into from Jun 17, 2020
Merged

location data quick fix #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e7dca77
add json2str option
czue Jun 17, 2020
43d273c
update readme
czue Jun 17, 2020
f8a52eb
support json-formatted location data
czue Jun 17, 2020
d035636
update tests
czue Jun 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Expand Up @@ -391,6 +391,7 @@ List of builtin functions:
| selected-at | Returns the Nth word in a string. N is zero-indexed. | selected-at(3) - return 4th word |
| selected | Returns True if the given word is in the value. | selected(fever) |
| count-selected | Count the number of words | |
| json2str | Convert a JSON object to a string |
| template | Render a string template (not robust) | template({} on {}, state, date) |
| attachment_url | Convert an attachment name into it's download URL | |

Expand Down
2 changes: 1 addition & 1 deletion commcare_export/builtin_queries.py
View file
Open in desktop
Expand Up @@ -98,7 +98,7 @@ def sql_column_name(code):
Column('external_id', 'external_id'),
Column('last_modified', 'last_modified', 'str2date'),
Column('latitude', 'latitude'),
Column('location_data', 'location_data'),
Column('location_data', 'location_data', 'json2str'),
Column('location_id', 'location_id'),
Column('location_type', 'location_type'),
Column('longitude', 'longitude'),
Expand Down
12 changes: 12 additions & 0 deletions commcare_export/env.py
View file
Open in desktop
@@ -1,6 +1,7 @@
from __future__ import unicode_literals, print_function, absolute_import, division, generators, nested_scopes

import hashlib
import json
from datetime import datetime
import operator
import pytz
Expand Down Expand Up @@ -344,6 +345,16 @@ def count_selected(val):
return len(val.split())


@unwrap('val')
def json2str(val):
if isinstance(val, six.string_types):
return val
try:
return json.dumps(val)
except ValueError:
return


def join(*args):
args = [unwrap_val(arg)for arg in args]
try:
Expand Down Expand Up @@ -430,6 +441,7 @@ def __init__(self, d=None):
'bool2int': bool2int,
'str2num': str2num,
'str2date': str2date,
'json2str': json2str,
'selected': selected,
'selected-at': selected_at,
'count-selected': count_selected,
Expand Down
8 changes: 4 additions & 4 deletions tests/test_cli.py
View file
Open in desktop
Expand Up @@ -109,13 +109,13 @@ def mock_hq_client(include_parent):
{'id': 'id1', 'created_at': '2020-04-01T21:57:26.403053',
'domain': 'd1', 'external_id': 'eid1',
'last_modified': '2020-04-01T21:58:23.88343',
'latitude': '11.2', 'location_data': 'ld1',
'latitude': '11.2', 'location_data': {'p1': 'ld1'},
'location_id': 'lid1', 'location_type': 'lt1',
'longitude': '-20.5', 'name': 'n1',
'resource_uri': 'ru1', 'site_code': 'sc1'},
{'id': 'id2', 'created_at': '2020-04-01T21:58:47.627371',
'domain': 'd2', 'last_modified': '2020-04-01T21:59:16.018411',
'latitude': '-56.3', 'location_data': 'ld2',
'latitude': '-56.3', 'location_data': {'p1': 'ld2'},
'location_id': 'lid2', 'location_type': 'lt2',
'longitude': '18.7', 'name': 'n2',
'parent': 'ru1' if include_parent else None,
Expand Down Expand Up @@ -207,11 +207,11 @@ def get_expected_locations_results(include_parent):
],
"rows": [
["id1", "2020-04-01 21:57:26", "d1", "eid1",
"2020-04-01 21:58:23", "11.2", "ld1", "lid1", "lt1",
"2020-04-01 21:58:23", "11.2", '{"p1": "ld1", "id": "id1.location_data"}', "lid1", "lt1",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an example of the id property that gets injected

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a surprise to me, but the 'id' comes from this line:
https://github.com/dimagi/commcare-export/blob/master/commcare_export/env.py#L203

It seems to be recording the path to the dictionary within its enclosing object. I would leave it as is for now because I don't know how to avoid it for location_data without affecting its intended use.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for chasing this down!

"-20.5", "n1", None, "ru1", "sc1", True, "hq", "HQ", None,
None, "lid1"],
["id2", "2020-04-01 21:58:47", "d2", None,
"2020-04-01 21:59:16", "-56.3", "ld2", "lid2", "lt2",
"2020-04-01 21:59:16", "-56.3", '{"p1": "ld2", "id": "id2.location_data"}', "lid2", "lt2",
"18.7", "n2", ("ru1" if include_parent else None), "ru2",
"sc2", False, "local", "Local", "lt1",
"lid2", ("lid1" if include_parent else None)]
Expand Down