Archive GPX processing fails on activities without GPX data

[gjreda]

Hi!

Thanks for creating and sharing this tool. I have a suspicion it's going to save me from writing a bunch of code on my own.

I ran into an issue today when trying to process the GPX data from my Strava archives. I have some activities that do not have GPX data. For example, an indoor ride on my trainer or a weight lifting session. Here is one example. When GPX processing tries to download the GPX file for this activity, it understandably cannot find one and throws the below stack trace.

Oddly, the ride does have has_location_data = 1 in the database, though it lacks a start_latlng and end_latlng (see data below).

Might excluding activities that do not have a start and end latlng here fix the issue?

I'd be happy to open a PR if you have a fix in mind.

Thank you again for sharing this project!

                  id = 461483444
           upload_id = 512650720
                name = Flywheel
          start_date = 2016-01-02T16:33:25Z
         moving_time = 2692
        elapsed_time = 2692
            distance = 0.0
total_elevation_gain = 0.0
             gear_id =
                type = Ride
             commute = 0
   has_location_data = 1
                json = {"resource_state": 2, "athlete": {"id": 4403225, "resource_state": 1}, "name": "Flywheel", "distance": 0.0, "moving_time": 2692, "elapsed_time": 2692, "total_elevation_gain": 0, "type": "Ride", "sport_type": "Ride", "workout_type": null, "id": 461483444, "start_date": "2016-01-02T16:33:25Z", "start_date_local": "2016-01-02T10:33:25Z", "timezone": "(GMT-06:00) America/Chicago", "utc_offset": -21600.0, "location_city": null, "location_state": null, "location_country": "United States", "achievement_count": 0, "kudos_count": 0, "comment_count": 0, "athlete_count": 1, "photo_count": 0, "map": {"id": "a461483444", "summary_polyline": "", "resource_state": 2}, "trainer": true, "commute": false, "manual": false, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [], "end_latlng": [], "average_speed": 0.0, "max_speed": 0.0, "has_heartrate": true, "average_heartrate": 168.0, "max_heartrate": 182.0, "heartrate_opt_out": false, "display_hide_heartrate_option": true, "upload_id": 512650720, "upload_id_str": "512650720", "external_id": "2016-01-02-1033_Spin_Class_WF.tcx", "from_accepted_tag": false, "pr_count": 0, "total_photo_count": 0, "has_kudoed": false}

Here's the full traceback, in case it's helpful.

(venv) greg@Gregs-MacBook-Air running % strava-offline gpx --dir-activities-backup data/strava_archive_20221122 --database data/strava.sqlite --strava4-session 1ggcfckqdog722tg57u55s6nuq0t22au --config co
nfig.yaml
Traceback (most recent call last):
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 124, in get_gpx
    return self._get_gpx("original", activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 117, in _get_gpx
    raise NotGpx(f"expected gpx attachment, got:\n{r.headers}")
strava_offline.strava.NotGpx: expected gpx attachment, got:
{'Content-Type': 'application/octet-stream', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Date': 'Tue, 06 Dec 2022 05:03:51 GMT', 'Via': '1.1 linkerd, 1.1 linkerd, 1.1 ad82d8a3c91257adecf18541576c7e72.cloudfront.net (CloudFront)', 'ETag': 'W/"f74f19e94a7e22a35f9d1cf12f85aadc"', 'Pragma': 'no-cache', 'Server': 'nginx/1.21.3', 'Status': '200 OK', 'Expires': 'Sat, 01 Jan 2000 00:00:00 GMT', 'X-Request-Id': 'b9c1ec14-68cb-42b4-99c3-0c5c8cd21031', 'Cache-Control': 'no-cache, no-store', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-Frame-Options': 'DENY', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'Content-Disposition': 'attachment; filename="Flywheel.tcx"; filename*=UTF-8\'\'Flywheel.tcx', 'X-Content-Type-Options': 'nosniff', 'Content-Transfer-Encoding': 'binary', 'X-Permitted-Cross-Domain-Policies': 'none', 'X-Cache': 'Miss from cloudfront', 'X-Amz-Cf-Pop': 'SFO53-C1', 'X-Amz-Cf-Id': 'oJVUCZfFHkWfzHF8L6utIPCUhTMX2gLpqACYrq0LyvRA7hyCV4Cxmw=='}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/greg/Dropbox/running/venv/bin/strava-offline", line 8, in <module>
    sys.exit(cli())
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/config.py", line 36, in wrapper
    return f(config_class(**config_kwargs), **f_kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/cli.py", line 42, in cli_gpx
    gpx.sync(config=config, strava=strava)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 75, in sync
    download_activities(db=db, strava=strava, dir_activities=config.dir_activities)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 59, in download_activities
    download_gpx(strava=strava, activity_id=activity_id, path=dir_activities)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 42, in download_gpx
    gpx = strava.get_gpx(activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 126, in get_gpx
    return self._get_gpx("gpx", activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 117, in _get_gpx
    raise NotGpx(f"expected gpx attachment, got:\n{r.headers}")
strava_offline.strava.NotGpx: expected gpx attachment, got:
{'Content-Type': 'text/html; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Date': 'Tue, 06 Dec 2022 05:03:52 GMT', 'Via': '1.1 linkerd, 1.1 linkerd, 1.1 ad82d8a3c91257adecf18541576c7e72.cloudfront.net (CloudFront)', 'ETag': 'W/"6a683fcb8869821749f7cb424d9bd470"', 'Pragma': 'no-cache', 'Server': 'nginx/1.21.3', 'Status': '200 OK', 'Expires': 'Sat, 01 Jan 2000 00:00:00 GMT', 'X-Request-Id': 'd353cde2-af0b-4179-8957-4e0b13d70fb8', 'Cache-Control': 'no-cache, no-store', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-Frame-Options': 'DENY', 'Content-Encoding': 'gzip', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'X-Permitted-Cross-Domain-Policies': 'none', 'X-Cache': 'Miss from cloudfront', 'X-Amz-Cf-Pop': 'SFO53-C1', 'X-Amz-Cf-Id': 'W8uwhNS6kMaY0fYQcGzVCjZnPTDK-tfK6KGUjpcpKppD7i_0vfKOqQ=='}
(venv) greg@Gregs-MacBook-Air running % strava-offline gpx --dir-activities-backup data/strava_archive_20221122 --config config.yaml --verbose
Traceback (most recent call last):
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 124, in get_gpx
    return self._get_gpx("original", activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 117, in _get_gpx
    raise NotGpx(f"expected gpx attachment, got:\n{r.headers}")
strava_offline.strava.NotGpx: expected gpx attachment, got:
{'Content-Type': 'application/octet-stream', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Date': 'Tue, 06 Dec 2022 05:05:45 GMT', 'Via': '1.1 linkerd, 1.1 linkerd, 1.1 b014854bd0108b7ed0058504b69ccb5a.cloudfront.net (CloudFront)', 'ETag': 'W/"f74f19e94a7e22a35f9d1cf12f85aadc"', 'Pragma': 'no-cache', 'Server': 'nginx/1.21.3', 'Status': '200 OK', 'Expires': 'Sat, 01 Jan 2000 00:00:00 GMT', 'X-Request-Id': 'd356e70a-cc99-437e-973d-cba5982f2351', 'Cache-Control': 'no-cache, no-store', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-Frame-Options': 'DENY', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'Content-Disposition': 'attachment; filename="Flywheel.tcx"; filename*=UTF-8\'\'Flywheel.tcx', 'X-Content-Type-Options': 'nosniff', 'Content-Transfer-Encoding': 'binary', 'X-Permitted-Cross-Domain-Policies': 'none', 'X-Cache': 'Miss from cloudfront', 'X-Amz-Cf-Pop': 'SFO53-C1', 'X-Amz-Cf-Id': 'IZdK7eu8C797NOyHRGJ1KuweMGfd53e0NjIKYqbh4kCMONL65U-aJQ=='}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/greg/Dropbox/running/venv/bin/strava-offline", line 8, in <module>
    sys.exit(cli())
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/config.py", line 36, in wrapper
    return f(config_class(**config_kwargs), **f_kwargs)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/cli.py", line 42, in cli_gpx
    gpx.sync(config=config, strava=strava)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 75, in sync
    download_activities(db=db, strava=strava, dir_activities=config.dir_activities)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 59, in download_activities
    download_gpx(strava=strava, activity_id=activity_id, path=dir_activities)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/gpx.py", line 42, in download_gpx
    gpx = strava.get_gpx(activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 126, in get_gpx
    return self._get_gpx("gpx", activity_id)
  File "/Users/greg/Dropbox/running/venv/lib/python3.9/site-packages/strava_offline/strava.py", line 117, in _get_gpx
    raise NotGpx(f"expected gpx attachment, got:\n{r.headers}")
strava_offline.strava.NotGpx: expected gpx attachment, got:
{'Content-Type': 'text/html; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Date': 'Tue, 06 Dec 2022 05:05:46 GMT', 'Via': '1.1 linkerd, 1.1 linkerd, 1.1 b014854bd0108b7ed0058504b69ccb5a.cloudfront.net (CloudFront)', 'ETag': 'W/"d9eb2cd9cb238fb428e1b5f1b599ce10"', 'Pragma': 'no-cache', 'Server': 'nginx/1.21.3', 'Status': '200 OK', 'Expires': 'Sat, 01 Jan 2000 00:00:00 GMT', 'X-Request-Id': 'dada34b2-6eda-4d0b-bd37-07e5447f6339', 'Cache-Control': 'no-cache, no-store', 'Referrer-Policy': 'strict-origin-when-cross-origin', 'X-Frame-Options': 'DENY', 'Content-Encoding': 'gzip', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'X-Permitted-Cross-Domain-Policies': 'none', 'X-Cache': 'Miss from cloudfront', 'X-Amz-Cf-Pop': 'SFO53-C1', 'X-Amz-Cf-Id': 'yarXWXK9_6m_Afsa2TC7HiaIGqjLNrI-Zxt3q4V9MJrDH2cKy5aVDQ=='}

[liskin]

Are you by any chance running an older version of strava-offline? A couple days ago I released https://github.com/liskin/strava-offline/releases/tag/1.0.3 which has this fix: 8c1dc65, so has_location_data shouldn't be 1 for that activity.

[gjreda]

I'm still on 1.0.2, so I'll give the new version a try later today and follow-up here.

I'm assuming I'll need to re-run the sqlite function to recreate the db data. Is that correct?

[liskin]

I'm assuming I'll need to re-run the sqlite function to recreate the db data. Is that correct?

No, any command that opens the database runs migrations if necessary, so just invoking strava-offline gpx should work.

[gjreda]

Upgrading to the new version resolved the issue. Thanks for your help!