Skip to content

OpenBarbell Data Storage Format

JonnyLinja edited this page Nov 26, 2018 · 21 revisions

SECTION 1: JSON DATA FORMAT

This section describes the data format for the JSON that is stored in our MongoDB and also reflects closely with the state of the Sets Reducer in the Redux Store of the app.

Note that because of changes across versions of the app, certain parts of the format have changed. Certain fields are only for newer versions, other fields were deprecated from older versions.

FIELD NAME DESCRIPTION
_id (MongoDB specific) The unique identifier for the set, used by the database. This does not show up in the Redux store of the app
setID The unique identifier for the set
userID (MongoDB specific) The unique identifier for the user. This does not show up in the Redux store of the app
workoutID The unique identifier for the workout (collection of sets)
exercise The name of the exercise, null if never entered, possibly empty string
setNumber The position of the set in the workout itself, not really used for anything. This is NOT the set number displayed in the app, that is calculated on the fly
weight The weight of the exercise. This can be null, an actual float/double, or a STRING (empty string included) depending on the version of the app. The latest version has it as a string.
metric 'kgs' or 'lbs' as a string. Note that when users toggle the metric, it does NOT change the weight. This is used simply as a label, not as a conversion tool.
rpe The RPE of the exercise. This can be null, an actual float/double, or a STRING (empty string included) depending on the version of the app. The latest version has it as a string.
tags (newer app releases) An array of strings. Each string is a tag that the user inputs to describe the set more. The only special tag is 'Bug' which is elevated, red, and used for us to track down issues.
startTime (deprecated) The start time of the set, calculated based on the first rep of the set. Used in older versions of the app. It is a string that is formatted based on the local phone/hardware. Theoretically nullable. Deprecated. See Special Note: Start Time Oddities
endTime (deprecated) The end time of the set, calculated based on the last rep of the set. Used in older versions of the app. It is a string that is formatted based on the local phone/hardware. Theoretically nullable. Deprecated. See Special Note: Start Time Oddities
initialStartTime (newer app releases) The start time of the set, created when a user first adds set info (like exercise name) to the set when it has no reps. This is done so we always have a timestamp even if the set has no reps. It is a string that is formatted based on the local phone/hardware and should theoretically never be null. Note that this is superseded by the rep's start time for start time calculation. See Special Note: Start Time Oddities
removed (to be deprecated) A boolean that indicates that the set is removed. While it's still being sent, this is no longer used by the app and is not a valid way to see if a set is removed. See Special Note: Removed Oddities
videoFileURL (newer app releases) The local URL of the video located on the user's device. We do not have access to the video. This can be null or a string.
videoType (newer app releases) Null or 'commentary' or 'lift'. Lift describes a video recorded for the working set. Commentary describes a video recorded after the set was already done.
reps An array of reps. See Section 2: Rep Data Format. Should not be null unless it's a bug. See Special Note: Other Format Oddities

Special Note: Start Time Oddities

  • Start time and end time was previously calculated as a single date via startTime and endTime. However, this proved inaccurate for calculating rest times if a rep was removed (it wouldn't change) and didn't account for sets with no reps (a feature we decided to allow later on). True start and end time is therefore calculated based on this priority order.
  • If there are reps, use the first valid not-removed rep's time property as the start time and the last valid not-removed rep's time property as the end time.
  • If there are none of the above, then use initialStartTime as the start time and nothing as the end time.
  • If that doesn't exist, then use startTime and endTime as the start and end time respectively.
  • Furthermore, all time formats use the basic JavaScript Date. It is therefore formatted based on whatever the local device specifies it should be and is NOT consistent across every single set. The app parses the string back into a Date as needed.

Special Note: Removed Oddities

  • Older versions of the app actively calculated and used the removed property of a set. However, as what determined that a rep was removed changed over time, the removed property got dropped in favor of calculating removed from the data itself. Removed sets are now determined by the following logic.
  • exercise must be null or ''
  • rpe must be null or ''
  • weight must be null or ''
  • videoFileType must be null or not exist
  • videoFileURL must be null or not exist
  • tags must be empty or not exist
  • reps must be empty or all removed/invalid

Special Note: Other Format Oddities

There may be times where a set follows none of these rules. These are bugs, some because a few of our users had test versions of the app, others because it was legitimately a bug in the app recording data that was fixed in later versions. For example, there are certain sets with just videos and nothing else, which are clearly bugs.

SECTION 2: REP DATA FORMAT

FIELD NAME DESCRIPTION
isValid Boolean that describes whether the rep is valid. The app checks for invalid by seeing if certain flags are in the correct positions. See Section 3: OpenBarbell Raw Data Format. Should never be null.
removed Boolean that describes whether the rep is removed. Users often remove reps that result from attaching the OpenBarbell roller hook to the Barbell. Should never be null.
hardware (newer app releases) 'android' or 'ios'. Important to know as there’s a chance there are issues / bugs specific to the hardware. Should never be null.
appVersion (newer app releases) Looks like '3.2.3'. Note that we do not have any version 2s, we went from v1 to v3 in order to keep the app version number coinciding with the hardware version number. Should never be null.
deviceName (newer app releases) Looks like 'OB 3215'. Describes which OpenBarbell was used to collect that data. Interesting as it's possible (unlikely) people use multiple OpenBarbells even for the same set. Should never be null. Note that all V1s are from 1000-1999, all V2s are from 2000-2999, and all V3s are from 3000-3999.
deviceIdentifier (newer app releases) A string describing the hardware's MAC address. Should never be null.
time (newer app releases) It is a string that is formatted based on the local phone/hardware and used to calculate the set's start and end time. Special Note: Start Time Oddities
data An array of OpenBarbell data. Should not be null unless it's a bug. See Section 3: OpenBarbell Raw Data Format

SECTION 3: OPENBARBELL RAW DATA FORMAT

This is the data that the OpenBarbell device sends directly to the app. It's basically a bunch of numbers sent AFTER a rep is complete; the app does not receive them mid rep. The OpenBarbell device only records the data for the concentric portion of the lift.

Note that each OpenBarbell version (v1, v2, v3) has different formats to its numbers. The numbers are all stored as Strings (if this is not the case, please inform the team). The app uses the very first flag to determine which OpenBarbell version is being used. For those working on the open source app, use app/utility/transforms/RepDataMap.js to convert the array data into metrics like velocity.

See this description on the wiki. Note that while the formats described in the wiki are for V1, the way the data works is still accurate for V3s.

OBV1 Data Format

  1. -1234 (Start Message)
  2. Rep Number
  3. Avg Velocity (m/s)
  4. Range of Motion (mm)
  5. Peak Velocity (m/s)
  6. Peak Velocity Location (%)
  7. -9999 (Start Bulk Data Settings Transmission)
  8. Number of Ticks Counted
  9. Precision of Compressed Values
  10. -9876 (Start Bulk Data)
  11. BULK DATA (UNKNOWN LENGTH)
  12. -6789 (End Bulk Data)
  13. Battery Charge

OBV2 Data Format

  1. -2345 (Start Message)
  2. Rep Number
  3. Avg Velocity (m/s)
  4. Range of Motion (mm)
  5. Peak Velocity (m/s)
  6. Peak Velocity Location (%)
  7. -2121 (Start OBV2 Extra Data)
  8. Duration of Rep (microseconds)
  9. Time between Reps (minutes)
  10. Timestamp of Rep Completion (microseconds)
  11. Amount of time "waiting" in rep (microseconds)
  12. Slowest Instantaneous Velocity Allowable (microseconds)
  13. Amount of Time the Backlight is allowed to stay on (microseconds)
  14. Minimum allowable Rep Length (micrometers)
  15. Is data compression enabled for Bulk Data (bool)
  16. Is filtration enabled for bulk data (bool)
  17. Code Version
  18. Unit Number
  19. -9999 (Start Bulk Data Settings Transmission)
  20. Number of Ticks Counted
  21. Precision of Compressed Values
  22. -9876 (Start Bulk Data)
  23. BULK DATA (UNKNOWN LENGTH)
  24. -6789 (End Bulk Data)
  25. Battery Charge

OBV3 Data Format

  1. -3456 (Start Message) (can also be -3456.0)
  2. Rep Number
  3. Avg Velocity (m/s)
  4. Range of Motion (mm)
  5. Peak Velocity (m/s)
  6. Peak Velocity Location (%)
  7. Peak Acceleration (this is unused in the app as we weren't able to validate it in time)
  8. Duration of Rep (microseconds)
  9. Time between Reps (minutes - this appears to be not working for V3s)
  10. Timestamp of Rep Completion (microseconds)
  11. Timestamp of time "waiting" in rep (microseconds)
  12. Slowest Instantaneous Velocity Allowable (microseconds)
  13. Amount of Time the Backlight is allowed to stay on (microseconds)
  14. Minimum allowable Rep Length (micrometers)
  15. Is data compression enabled for Bulk Data (bool)
  16. Is filtration enabled for bulk data (bool)
  17. Code Version
  18. Unit Number
  19. LED Color (RGB)
  20. Brightness (%)
  21. Low Power Brightness (%)
  22. -9999 (Flag Bulk Data Settings)
  23. Number of Ticks Counted
  24. Precision of Compressed Values
  25. -9876 (Start Bulk Data)
  26. BULK DATA (UNKNOWN LENGTH)
  27. -6789 (End Bulk Data)
  28. Battery Charge

Special Note: Invalid Reps

In newer versions of the app, a start flag is received in the middle of receiving rep data from the device, all previous data in the queue is dumped as an invalid rep. Older versions of the app would trash that data.

Special Note: Peak Velocity Location

This is the point of your lift where your peak velocity was recorded. So if it's 70% then 70% of the way through your lift was your peak velocity.

Special Note: Infinity

In the OpenBarbell app, Infinity (which is a result of a hardware bug) will be stored as Infinity. However, data coming from RepOne Alpha versions will replace Infinity with -1. -1 therefore can sometimes mean Infinity.

SECTION 4: UNDERSTANDING BULK DATA

See this description on the wiki. Note that while the formats described in the wiki are for V1, the way bulk data works is still accurate for V3s.

Special Note: Bulk Data Mode

The hardware has the ability to turn on higher precision for the bulk data. The app currently does not enable this as it would severely impact how long the OpenBarbell device took to calculate and send that data to the phone. The option might be made available in the near future.

Special Note: Bulk Data Truncation Bug

Older versions of the app truncated all numbers to two decimal points. This worked well for say velocities, but it also invalidated the recorded bulk data as bulk data needed more precision. Newer versions of the app resolved this.

Special Note: Other Bulk Data Bugs

There are bugs that we have not resolved where the velocities are off. This can be seen in the bulk data where there is sudden acceleration or deceleration beyond what's normal. It is likely an issue with the hardware or firmware.