Vivofit support #99

Closed
psukys opened this Issue Jun 24, 2014 · 22 comments

Comments

Projects
None yet
5 participants
@psukys

psukys commented Jun 24, 2014

It seems to be partly working with vivofit. The workout first workout file is downloaded the right way, although I'm not sure about the second (1989 yr.) file
Here's a paste of the log, maybe I should just add an exception for this file?
http://pastebin.com/KhfSjHWG

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jun 24, 2014

Owner

Oh, a new device. Is it this one http://sites.garmin.com/vivo/? This bug might already be fixed on the split branch, https://github.com/Tigge/Garmin-Forerunner-610-Extractor/tree/split, could you try it with this branch? It requires the https://github.com/Tigge/openant library.

Owner

Tigge commented Jun 24, 2014

Oh, a new device. Is it this one http://sites.garmin.com/vivo/? This bug might already be fixed on the split branch, https://github.com/Tigge/Garmin-Forerunner-610-Extractor/tree/split, could you try it with this branch? It requires the https://github.com/Tigge/openant library.

@psukys

This comment has been minimized.

Show comment
Hide comment
@psukys

psukys Jun 29, 2014

Yes, it is vivofit :) I tried this branch and it retrieved current data without problems. I'll be testing it out throughout the week and post my findings :)

psukys commented Jun 29, 2014

Yes, it is vivofit :) I tried this branch and it retrieved current data without problems. I'll be testing it out throughout the week and post my findings :)

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jun 29, 2014

Owner

Excellent. Keep me posted!

Owner

Tigge commented Jun 29, 2014

Excellent. Keep me posted!

@psukys

This comment has been minimized.

Show comment
Hide comment
@psukys

psukys Jun 30, 2014

It seems that antfs-cli downloads only empty file of current time, even though I had an activity on (using the windowish Garmin connector after, it synced today's and activity's file.
http://pastebin.com/r6x9q9kT

psukys commented Jun 30, 2014

It seems that antfs-cli downloads only empty file of current time, even though I had an activity on (using the windowish Garmin connector after, it synced today's and activity's file.
http://pastebin.com/r6x9q9kT

@psukys

This comment has been minimized.

Show comment
Hide comment
@psukys

psukys Jul 1, 2014

I'm not sure, but might there be a problem that uploading and downloading files are mixed up?

psukys commented Jul 1, 2014

I'm not sure, but might there be a problem that uploading and downloading files are mixed up?

@mart-e

This comment has been minimized.

Show comment
Hide comment
@mart-e

mart-e Jul 5, 2014

Hello,

I have tried with the split branch and same as for the master branch, I just get a 274 bytes file

mart-e commented Jul 5, 2014

Hello,

I have tried with the split branch and same as for the master branch, I just get a 274 bytes file

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jul 5, 2014

Owner

I wonder if the same bug as getting weight data is present here. If all data are in the same file but that file is updated with new data it will not be downloaded again. What sort of data are one expected to get from the vivofit? It might also be that it is not in a category recognized by antfs-cli. Can someone post a log file?

Owner

Tigge commented Jul 5, 2014

I wonder if the same bug as getting weight data is present here. If all data are in the same file but that file is updated with new data it will not be downloaded again. What sort of data are one expected to get from the vivofit? It might also be that it is not in a category recognized by antfs-cli. Can someone post a log file?

@mart-e

This comment has been minimized.

Show comment
Hide comment
@mart-e

mart-e Jul 5, 2014

Ok after doing some debugging of the client, it seems that the files are present but it's just not the correct ones that are uploaded.

If I execute directory.print_list(), I get the following result

Index   Type    FIT Type    FIT Number  Size    Date    FIT Flags   Flags
1   1   14  0   64512   1989-12-31 01:00:00     0   -w----
2   128     2   0   200     1989-12-31 01:00:00     0   -w----
3   128     11  0   200     1989-12-31 01:00:00     0   -w----
4   128     3   0   200     1989-12-31 01:00:00     0   -w----
5   128     1   0   274     2014-07-05 22:01:00     0   r-----
6   128     32  36  865     2014-06-29 17:34:00     0   r-----
7   128     32  37  346     2014-06-29 22:01:00     0   r-----
8   128     32  38  342     2014-06-29 22:02:00     0   r-----
9   128     32  39  350     2014-06-29 22:03:00     0   r-----
10  128     32  40  377     2014-06-29 22:06:00     0   r-----
...
65  128     32  100     306     2014-07-04 10:12:00     0   r-----

So it seems that the vivofit uses the type 32 to record steps and activities.
Adding 32 in the Identifier makes the software download all these files.
I can now upload it on the manual import page on the garmin website but I am not sure it got all my steps (displays 3000 for yesterday while I got at least three time that). I guess it's because I get several files for the same day and it remembers only the first one.

I will make a pull request to add the 32 type. @Tigge do you prefer I do it on the master of this repository or on the openant ?

update: ok I don't get it, it seems that the following days, I did not get any files following the first synchronisation

update 2: so it seems the vivofit cleans the memory after sync with my phone but I miss the "update your directory now" command

mart-e commented Jul 5, 2014

Ok after doing some debugging of the client, it seems that the files are present but it's just not the correct ones that are uploaded.

If I execute directory.print_list(), I get the following result

Index   Type    FIT Type    FIT Number  Size    Date    FIT Flags   Flags
1   1   14  0   64512   1989-12-31 01:00:00     0   -w----
2   128     2   0   200     1989-12-31 01:00:00     0   -w----
3   128     11  0   200     1989-12-31 01:00:00     0   -w----
4   128     3   0   200     1989-12-31 01:00:00     0   -w----
5   128     1   0   274     2014-07-05 22:01:00     0   r-----
6   128     32  36  865     2014-06-29 17:34:00     0   r-----
7   128     32  37  346     2014-06-29 22:01:00     0   r-----
8   128     32  38  342     2014-06-29 22:02:00     0   r-----
9   128     32  39  350     2014-06-29 22:03:00     0   r-----
10  128     32  40  377     2014-06-29 22:06:00     0   r-----
...
65  128     32  100     306     2014-07-04 10:12:00     0   r-----

So it seems that the vivofit uses the type 32 to record steps and activities.
Adding 32 in the Identifier makes the software download all these files.
I can now upload it on the manual import page on the garmin website but I am not sure it got all my steps (displays 3000 for yesterday while I got at least three time that). I guess it's because I get several files for the same day and it remembers only the first one.

I will make a pull request to add the 32 type. @Tigge do you prefer I do it on the master of this repository or on the openant ?

update: ok I don't get it, it seems that the following days, I did not get any files following the first synchronisation

update 2: so it seems the vivofit cleans the memory after sync with my phone but I miss the "update your directory now" command

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jul 13, 2014

Owner

Pull requests on openant is very much prefered. Thanks for that. Both the openant and antfs-cli pull request are now merged.

So it only counts steps since last download? A bit weird. Right now antfs-cli will not redownload your file if it has changed. A quick fix to redownload if the archived flag is not set. This will work if you are always connecting the device to the same computer. A real fix will take a bit longer.

Owner

Tigge commented Jul 13, 2014

Pull requests on openant is very much prefered. Thanks for that. Both the openant and antfs-cli pull request are now merged.

So it only counts steps since last download? A bit weird. Right now antfs-cli will not redownload your file if it has changed. A quick fix to redownload if the archived flag is not set. This will work if you are always connecting the device to the same computer. A real fix will take a bit longer.

@mart-e

This comment has been minimized.

Show comment
Hide comment
@mart-e

mart-e Jul 23, 2014

Thanks @Tigge for the merges.
After further investigations, I figured out that I need one "empty sync" (meaning I put the vivofit in sync mode but my laptop is not) to force the device to generate .fit file. At the next sync (with my laptop this time), I get a fresh .fit file at the time of the empty sync.
Weird but working...

mart-e commented Jul 23, 2014

Thanks @Tigge for the merges.
After further investigations, I figured out that I need one "empty sync" (meaning I put the vivofit in sync mode but my laptop is not) to force the device to generate .fit file. At the next sync (with my laptop this time), I get a fresh .fit file at the time of the empty sync.
Weird but working...

@minybot

This comment has been minimized.

Show comment
Hide comment
@minybot

minybot Jul 25, 2014

I success to download some fit files from vivofit. But those fit files are corrupted.

-- 1117. #55: unknown (6 entries) --------------------------
unknown [uint32]: 774853680
unknown [uint32]: 0
unknown [uint32]: 586000
unknown [uint16]: 279
unknown [uint16]: 788
unknown [enum]: 0

-- 1118. #55: unknown (6 entries) --------------------------
unknown [uint32]: 774853680
unknown [uint32]: 3540
unknown [uint32]: 784000
unknown [uint16]: 505
unknown [uint16]: 788
unknown [enum]: 6

-- 1119. #125: unknown (3 entries) -------------------------
unknown [uint32]: 774853680
unknown [byte]:
unknown [byte]: ���
���

Traceback (most recent call last):
File "./sample_program.py", line 141, in
a.parse(hook_func=print_hook_func)
File "/home/ukcco/research/sport/python-fitparse/fitparse/activity.py", line 9, in parse
raise FitParseError("File parsed is not an activity file.")
fitparse.exceptions.FitParseError: File parsed is not an activity file.

minybot commented Jul 25, 2014

I success to download some fit files from vivofit. But those fit files are corrupted.

-- 1117. #55: unknown (6 entries) --------------------------
unknown [uint32]: 774853680
unknown [uint32]: 0
unknown [uint32]: 586000
unknown [uint16]: 279
unknown [uint16]: 788
unknown [enum]: 0

-- 1118. #55: unknown (6 entries) --------------------------
unknown [uint32]: 774853680
unknown [uint32]: 3540
unknown [uint32]: 784000
unknown [uint16]: 505
unknown [uint16]: 788
unknown [enum]: 6

-- 1119. #125: unknown (3 entries) -------------------------
unknown [uint32]: 774853680
unknown [byte]:
unknown [byte]: ���
���

Traceback (most recent call last):
File "./sample_program.py", line 141, in
a.parse(hook_func=print_hook_func)
File "/home/ukcco/research/sport/python-fitparse/fitparse/activity.py", line 9, in parse
raise FitParseError("File parsed is not an activity file.")
fitparse.exceptions.FitParseError: File parsed is not an activity file.

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jul 25, 2014

Owner

@mart-e that is a bit weird. Did anyone try my suggestion with the archived flag? It requires some python knowledge but in antfs-cli.py:

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype], remote_files[filetype])

should be something like

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype] or not fil.is_archived(), remote_files[filetype])
Owner

Tigge commented Jul 25, 2014

@mart-e that is a bit weird. Did anyone try my suggestion with the archived flag? It requires some python knowledge but in antfs-cli.py:

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype], remote_files[filetype])

should be something like

   downloading[filetype] = filter(lambda fil: self.get_filename(fil) not in local_files[filetype] or not fil.is_archived(), remote_files[filetype])
@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jul 25, 2014

Owner

@minybot the files downloaded from Vivofit are probably not FIT Activity files but rather other types of FIT files that contains information about heartbeat rate and such.

Owner

Tigge commented Jul 25, 2014

@minybot the files downloaded from Vivofit are probably not FIT Activity files but rather other types of FIT files that contains information about heartbeat rate and such.

@minybot

This comment has been minimized.

Show comment
Hide comment
@minybot

minybot Jul 25, 2014

Is any way or document to extract the information from vivofit Fit files?
Btw, I can download all files from my vivofit successfully.

minybot commented Jul 25, 2014

Is any way or document to extract the information from vivofit Fit files?
Btw, I can download all files from my vivofit successfully.

@mart-e

This comment has been minimized.

Show comment
Hide comment
@mart-e

mart-e Jul 25, 2014

I will try the archive flag this weekend.
@minybot I guess you are referring to the fitparse library. Try with the ng branch on it. The fitdump script give you some output but I haven't figured out the meaning yet. But as said @Tigge, the .fit files are a bit different (steps every x minutes and heartbeat mixed)

mart-e commented Jul 25, 2014

I will try the archive flag this weekend.
@minybot I guess you are referring to the fitparse library. Try with the ng branch on it. The fitdump script give you some output but I haven't figured out the meaning yet. But as said @Tigge, the .fit files are a bit different (steps every x minutes and heartbeat mixed)

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jul 25, 2014

Owner

If you is comfortable with the licence agreement, there is a FIT SDK
available here http://www.thisisant.com/resources/fit with all the
information you'll ever need. I'm guessing the interesting data from the
Vivofit is in the monitoring_a/b files (id 15 and 32) or perhaps daily
totals (id 28)

On 25 July 2014 11:16, Martin Trigaux notifications@github.com wrote:

I will try the archive flag this weekend.
@minybot https://github.com/minybot I guess you are referring to the
fitparse https://github.com/dtcooper/python-fitparse library. Try with
the ng branch on it. The fitdump
https://github.com/dtcooper/python-fitparse/blob/ng/scripts/fitdump
script give you some output but I haven't figured out the meaning yet.


Reply to this email directly or view it on GitHub
#99 (comment)
.

Owner

Tigge commented Jul 25, 2014

If you is comfortable with the licence agreement, there is a FIT SDK
available here http://www.thisisant.com/resources/fit with all the
information you'll ever need. I'm guessing the interesting data from the
Vivofit is in the monitoring_a/b files (id 15 and 32) or perhaps daily
totals (id 28)

On 25 July 2014 11:16, Martin Trigaux notifications@github.com wrote:

I will try the archive flag this weekend.
@minybot https://github.com/minybot I guess you are referring to the
fitparse https://github.com/dtcooper/python-fitparse library. Try with
the ng branch on it. The fitdump
https://github.com/dtcooper/python-fitparse/blob/ng/scripts/fitdump
script give you some output but I haven't figured out the meaning yet.


Reply to this email directly or view it on GitHub
#99 (comment)
.

@minybot

This comment has been minimized.

Show comment
Hide comment
@minybot

minybot Jul 25, 2014

From fitdump

  1. file_id

    • garmin_product: 1837
    • manufacturer: garmin
    • number: 27
    • serial_number: 3883044730
    • time_created: 2014-07-20 16:00:00
    • type: 32
    • unknown_6: 0
  2. device_info

    • battery_voltage: 5.94921875 [V]
    • hardware_version: 1
    • manufacturer: garmin
    • product: 1837
    • serial_number: 3883044730
    • software_version: 2.8
    • timestamp: 2014-07-25 16:27:00
  3. monitoring_info

    • local_timestamp: 2014-07-21 02:00:00
    • timestamp: 2014-07-20 16:00:00
    • unknown_1: (6, 1)
    • unknown_3: (7962, 11943)
    • unknown_4: (326, 1028)
    • unknown_5: 2175
    • unknown_7: (7171, 7171)
  4. monitoring

    • unknown_26: 39809
    • unknown_27: 69
  5. monitoring

    • unknown_26: 39824

    * unknown_27: 70

  6. monitoring

    • unknown_24: (104,)
    • unknown_26: 52828
  7. monitoring

    • unknown_24: (8,)
    • unknown_26: 52888
  8. monitoring

    • unknown_24: (72,)
    • unknown_26: 52948
  9. monitoring

    • unknown_24: (8,)
    • unknown_26: 54808
  10. monitoring

    • unknown_24: (72,)
    • unknown_26: 54928
  11. monitoring

    • unknown_24: (8,)
    • unknown_26: 56008
  12. monitoring

    • unknown_24: (72,)
    • unknown_26: 56128
  13. monitoring

    • unknown_24: (8,)
    • unknown_26: 56548
  14. monitoring

    • unknown_24: (104,)

    * unknown_26: 56608

  15. monitoring

    • cycles: 0 [cycles]
    • unknown_19: 6
    • unknown_24: (0,)

    * unknown_26: 57508

I think this is easy to extract now,
unknown_26: time
unknown_27: heart beat number

unknown_24: I am not sure

minybot commented Jul 25, 2014

From fitdump

  1. file_id

    • garmin_product: 1837
    • manufacturer: garmin
    • number: 27
    • serial_number: 3883044730
    • time_created: 2014-07-20 16:00:00
    • type: 32
    • unknown_6: 0
  2. device_info

    • battery_voltage: 5.94921875 [V]
    • hardware_version: 1
    • manufacturer: garmin
    • product: 1837
    • serial_number: 3883044730
    • software_version: 2.8
    • timestamp: 2014-07-25 16:27:00
  3. monitoring_info

    • local_timestamp: 2014-07-21 02:00:00
    • timestamp: 2014-07-20 16:00:00
    • unknown_1: (6, 1)
    • unknown_3: (7962, 11943)
    • unknown_4: (326, 1028)
    • unknown_5: 2175
    • unknown_7: (7171, 7171)
  4. monitoring

    • unknown_26: 39809
    • unknown_27: 69
  5. monitoring

    • unknown_26: 39824

    * unknown_27: 70

  6. monitoring

    • unknown_24: (104,)
    • unknown_26: 52828
  7. monitoring

    • unknown_24: (8,)
    • unknown_26: 52888
  8. monitoring

    • unknown_24: (72,)
    • unknown_26: 52948
  9. monitoring

    • unknown_24: (8,)
    • unknown_26: 54808
  10. monitoring

    • unknown_24: (72,)
    • unknown_26: 54928
  11. monitoring

    • unknown_24: (8,)
    • unknown_26: 56008
  12. monitoring

    • unknown_24: (72,)
    • unknown_26: 56128
  13. monitoring

    • unknown_24: (8,)
    • unknown_26: 56548
  14. monitoring

    • unknown_24: (104,)

    * unknown_26: 56608

  15. monitoring

    • cycles: 0 [cycles]
    • unknown_19: 6
    • unknown_24: (0,)

    * unknown_26: 57508

I think this is easy to extract now,
unknown_26: time
unknown_27: heart beat number

unknown_24: I am not sure

@minybot

This comment has been minimized.

Show comment
Hide comment
@minybot

minybot Jul 25, 2014

  1. monitoring

    • unknown_26: 4867
    • unknown_27: 76
  2. event

    • data: 1
    • data16: 1
    • event: 41
    • event_type: marker
    • timestamp: 2014-07-21 00:30:00
  3. monitoring

    • unknown_24: (72,)
    • unknown_26: 4872
  4. monitoring

    • active_time: 262.0 [s]
    • activity_type: generic
    • cycles: 0 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 132
    • unknown_29: 510
  5. monitoring

    • active_time: 437.0 [s]
    • activity_type: walking
    • cycles: 1332 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 206
    • unknown_29: 510
  6. unknown_125

    • unknown_1: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
    • unknown_2: (127, 0, 0, 0, 381, 0, 2, 199, 42, 0, 0, 0, 0, 0, 0, 30149)

    * unknown_253: 774837000

    end of file

minybot commented Jul 25, 2014

  1. monitoring

    • unknown_26: 4867
    • unknown_27: 76
  2. event

    • data: 1
    • data16: 1
    • event: 41
    • event_type: marker
    • timestamp: 2014-07-21 00:30:00
  3. monitoring

    • unknown_24: (72,)
    • unknown_26: 4872
  4. monitoring

    • active_time: 262.0 [s]
    • activity_type: generic
    • cycles: 0 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 132
    • unknown_29: 510
  5. monitoring

    • active_time: 437.0 [s]
    • activity_type: walking
    • cycles: 1332 [cycles]
    • timestamp: 2014-07-21 00:30:00
    • unknown_19: 206
    • unknown_29: 510
  6. unknown_125

    • unknown_1: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
    • unknown_2: (127, 0, 0, 0, 381, 0, 2, 199, 42, 0, 0, 0, 0, 0, 0, 30149)

    * unknown_253: 774837000

    end of file

mgr01 added a commit to mgr01/antfs-cli that referenced this issue Jan 3, 2015

Always redownload non-"archived" files
Should fix issues #45 and #99, but I don't have specific hardware to test it.
@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Jan 3, 2015

Owner

Hopefully this will work better with #120. Closing. Re-open if it still fails.

Owner

Tigge commented Jan 3, 2015

Hopefully this will work better with #120. Closing. Re-open if it still fails.

@Tigge Tigge closed this Jan 3, 2015

@balu-

This comment has been minimized.

Show comment
Hide comment
@balu-

balu- Mar 29, 2015

Contributor

seems like #120 (or 32aa009 in particular)
make antfs-cli redownload every single .fit file on the watch on every snyc?

Not familiar with python code, i only observed the behavior of the script, and by reverting the commit i got the old behavoir where only new files are transmitted.
Is there a reason to transfer all the files everytime?

Contributor

balu- commented Mar 29, 2015

seems like #120 (or 32aa009 in particular)
make antfs-cli redownload every single .fit file on the watch on every snyc?

Not familiar with python code, i only observed the behavior of the script, and by reverting the commit i got the old behavoir where only new files are transmitted.
Is there a reason to transfer all the files everytime?

@Tigge

This comment has been minimized.

Show comment
Hide comment
@Tigge

Tigge Mar 31, 2015

Owner

There should be no change in behavior with this commit. If it doesn't work anymore there is probably a bug in there somewhere. I'm not able to reproduce this on my system.

Owner

Tigge commented Mar 31, 2015

There should be no change in behavior with this commit. If it doesn't work anymore there is probably a bug in there somewhere. I'm not able to reproduce this on my system.

@balu-

This comment has been minimized.

Show comment
Hide comment
@balu-

balu- Apr 5, 2015

Contributor

my bad
redownload is triggerd by this change (as the commit message says 😳 ) da76e05

Contributor

balu- commented Apr 5, 2015

my bad
redownload is triggerd by this change (as the commit message says 😳 ) da76e05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment