exception with body_fat data type

[rickysarraf]

Hi, Thanks for creating this bridge.
When trying to run app.py, I've run into the following exception. I did setup the fitbit/google authentication successfully, as you've documented.

rrs@learner:/var/tmp/lxc/fitbit-googlefit (master)$ python3 app.py 
Traceback (most recent call last):
  File "app.py", line 60, in main
    googleClient.users().dataSources().get(userId='me',dataSourceId=dataSourceId).execute()
  File "/usr/lib/python3/dist-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/googleapiclient/http.py", line 832, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources/raw%3Acom.google.body.fat.percentage%3A558973839847%3Afitbit%3Aaria%3Aio.pkp.fbit-gfit?alt=json returned "DataSourceId not found: raw:com.google.body.fat.percentage:558973839847:fitbit:aria:io.pkp.fbit-gfit">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "app.py", line 113, in <module>
    main()
  File "app.py", line 65, in main
    googleClient.users().dataSources().create(userId='me',body=helper.GetDataSource(dataType)).execute()
  File "/usr/lib/python3/dist-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/googleapiclient/http.py", line 832, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources?alt=json returned "Bad Request">
2016-08-18 / 13:59:09 ♒♒♒  ☹  => 1  

[rickysarraf]

WHen accessing the link in a browser, I get the following error:

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Login Required"
 }
}

With the steps followed, I was under the impression that the auth aspect was complete

[pkpio]

@rickysarraf , It seems like the exception is caused by body_fat sync. Could you please try against the latest commit? If you still face the issue, try setting sync_body_fat=0 in config.ini file.

Please post me a log if the problem persists. I just tried a full sync from my end without any errors.

[rickysarraf]

@praveendath92 That was my first suspect.

rrs@learner:/var/tmp/lxc/fitbit-googlefit (master)$ cat config.ini 
[params]

# Starting date of sync (inclusive)
start_date=2016-08-16

# Ending date of sync (exclusive)
end_date=2016-08-18

# Which data types to sync?
sync_steps=1
sync_distance=1
sync_heartrate=1
#sync_weight=1
#sync_calories=1
#sync_body_fat=1
sync_activities=1

# Note: End date is not respected by activities sync. 
# All activities logged after the start_date will be synced.
2016-08-18 / 14:27:25 ♒♒♒  ☺  

To cut system libs out, I also did pip installations. BTW, you may want to explicitly define "/usr/bin/env python3" because many distros still default to python2.

But with latest pip3 install libs too did not help.

rrs@learner:/var/tmp/lxc/fitbit-googlefit (master)$ ./app.py 
Traceback (most recent call last):
  File "./app.py", line 60, in main
    googleClient.users().dataSources().get(userId='me',dataSourceId=dataSourceId).execute()
  File "/home/rrs/.local/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/rrs/.local/lib/python3.5/site-packages/googleapiclient/http.py", line 838, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources/raw%3Acom.google.body.fat.percentage%3A558973839847%3Afitbit%3Aaria%3Aio.pkp.fbit-gfit?alt=json returned "DataSourceId not found: raw:com.google.body.fat.percentage:558973839847:fitbit:aria:io.pkp.fbit-gfit">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./app.py", line 113, in <module>
    main()
  File "./app.py", line 65, in main
    googleClient.users().dataSources().create(userId='me',body=helper.GetDataSource(dataType)).execute()
  File "/home/rrs/.local/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/rrs/.local/lib/python3.5/site-packages/googleapiclient/http.py", line 838, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources?alt=json returned "Bad Request">
2016-08-18 / 14:28:46 ♒♒♒  ☹  => 1  

[rickysarraf]

THere's indentation issues with your commit.

diff --git a/remote.py b/remote.py
index 2efc1ee..ce77ae6 100644
--- a/remote.py
+++ b/remote.py
@@ -84,13 +84,13 @@ def WriteSessionToGoogleFit(googleClient,session_data):

 def CreateGoogleFitDataSource(googleClient,dataType):
        dataSourceId = helper.GetDataSourceId(dataType)
-               try:
-                       googleClient.users().dataSources().get(userId='me',dataSourceId=dataSourceId).execute()
-               except HttpError as error:
-                       if not 'DataSourceId not found' in str(error):
-                               raise error
-                       # Data source doesn't already exist so, create it!
-                       googleClient.users().dataSources().create(userId='me',body=helper.GetDataSource(dataType)).execute()
+       try:
+               googleClient.users().dataSources().get(userId='me',dataSourceId=dataSourceId).execute()
+       except HttpError as error:
+               if not 'DataSourceId not found' in str(error):
+                       raise error
+               # Data source doesn't already exist so, create it!
+               googleClient.users().dataSources().create(userId='me',body=helper.GetDataSource(dataType)).execute()

But that did not help either.

rrs@learner:/var/tmp/lxc/fitbit-googlefit (master)$ python3 app.py 
Traceback (most recent call last):
  File "/media/SSHD/var/tmp/lxc/fitbit-googlefit/remote.py", line 88, in CreateGoogleFitDataSource
    googleClient.users().dataSources().get(userId='me',dataSourceId=dataSourceId).execute()
  File "/home/rrs/.local/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/rrs/.local/lib/python3.5/site-packages/googleapiclient/http.py", line 838, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources/raw%3Acom.google.body.fat.percentage%3A558973839847%3Afitbit%3Aaria%3Aio.pkp.fbit-gfit?alt=json returned "DataSourceId not found: raw:com.google.body.fat.percentage:558973839847:fitbit:aria:io.pkp.fbit-gfit">

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "app.py", line 107, in <module>
    main()
  File "app.py", line 58, in main
    remote.CreateGoogleFitDataSource(googleClient, dataType)
  File "/media/SSHD/var/tmp/lxc/fitbit-googlefit/remote.py", line 93, in CreateGoogleFitDataSource
    googleClient.users().dataSources().create(userId='me',body=helper.GetDataSource(dataType)).execute()
  File "/home/rrs/.local/lib/python3.5/site-packages/oauth2client/util.py", line 137, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/rrs/.local/lib/python3.5/site-packages/googleapiclient/http.py", line 838, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/fitness/v1/users/me/dataSources?alt=json returned "Bad Request">
2016-08-18 / 14:31:15 ♒♒♒  ☹  => 1  

[pkpio]

I think dependencies are not an issue. To sync data with Google Fit, the app should first create data source for each type of data. The log says there's a bad request when creating data source for body_fat. Strange thing is that it doesn't happen to me!

[rickysarraf]

Do you know why the URL, when opened, says "Login Required" ?

I don't think the issue may be with your code (given that you confirmed that it is working for you). But rather, with the workflow you've documented. Are the 2 json files enough for the auth or anything else is also required ?

[rickysarraf]

This UR: https://www.googleapis.com/fitness/v1/users/me/dataSources?alt=json

[pkpio]

Just the json files are enough. "Login Required" message is normal for Google API endpoints when opened in browser. The authentication is done through access_token set in the request headers.

[pkpio]

I got it. Let me send you a quick fix.

[pkpio]

In this line - https://github.com/praveendath92/fitbit-googlefit/blob/master/app.py#L57

Could you try removing body_fat from the list and try again?

[pkpio]

The app creates data source for all types whether or not sync for that type is enabled.

[pkpio]

I could reproduce the error with a different account. I will work on fix. Until then, this fix solves the error.

[rickysarraf]

That worked. But I got a different error now.

rrs@learner:/var/tmp/lxc/fitbit-googlefit (master)$ python3 app.py 
------------------------------   2016-08-16  -------------------------
Traceback (most recent call last):
  File "app.py", line 107, in <module>
    main()
  File "app.py", line 77, in main
    remote.SyncFitbitToGoogleFit(fitbitClient,googleClient,'steps',date_stamp,tzinfo)
  File "/media/SSHD/var/tmp/lxc/fitbit-googlefit/remote.py", line 112, in SyncFitbitToGoogleFit
    return SyncFitbitIntradayToGoogleFit(fitbitClient, googleClient, dataType, date_stamp, tzinfo)
  File "/media/SSHD/var/tmp/lxc/fitbit-googlefit/remote.py", line 143, in SyncFitbitIntradayToGoogleFit
    intraday_data = interday_raw[resp_id]['dataset']
KeyError: 'activities-steps-intraday'
2016-08-18 / 14:46:48 ♒♒♒  ☹  => 1  

[pkpio]

Would you mind if we discuss this as a new issue?