Prevent DoesNotExist failure on activity API (fix #2227)

[noirbizarre]

This PR prevent the DoesNotExist exception from breaking the Activity API.

Under the hood, the problem occurs when some data is manually deleted without a proper purge. It would need a data migration to cleanup the existing data.

NB: tested but not included in this PR: to reduce the size of the activity collection it's possible to use TTL index and let mongo do its job in the background (possibly configurable with a ACTIVITY_RETENTION_DAYS settings). See for details;

• http://docs.mongoengine.org/guide/defining-documents.html#time-to-live-indexes
• https://docs.mongodb.com/manual/core/index-ttl/
• https://docs.mongodb.com/manual/tutorial/expire-data/

[abulte]

Have you done a quick performance assessment? From what I understand, we're switching from a QuerySet to a list and this can be less efficient.

[abulte]

❤️

[noirbizarre]

I remembered 😉

[noirbizarre]

Yep, done on my environment, might need some proper testing on preprod.
Reading the flask-mongoengine paginate code, it was already casted as a list so it shouldn't reduce perfs.

[abulte]

OK. For reference, it's not glorious as it is:

$ hey 'https://next.data.gouv.fr/api/1/activity?page=1&page_size=20'

Summary:
  Total:	80.0114 secs
  Slowest:	18.5939 secs
  Fastest:	2.9409 secs
  Average:	10.4998 secs
  Requests/sec:	2.4996


Response time histogram:
  2.941 [1]	|■■■■■■■■
  4.506 [4]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  6.072 [5]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  7.637 [0]	|
  9.202 [5]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  10.767 [5]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  12.333 [0]	|
  13.898 [5]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  15.463 [3]	|■■■■■■■■■■■■■■■■■■■■■■■■
  17.029 [2]	|■■■■■■■■■■■■■■■■
  18.594 [5]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■


Latency distribution:
  10% in 3.0890 secs
  25% in 5.6635 secs
  50% in 10.6053 secs
  75% in 15.2525 secs
  90% in 17.9233 secs
  95% in 18.5939 secs
  0% in 0.0000 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.4212 secs, 2.9409 secs, 18.5939 secs
  DNS-lookup:	0.0726 secs, 0.0709 secs, 0.0741 secs
  req write:	0.0001 secs, 0.0000 secs, 0.0006 secs
  resp wait:	10.0764 secs, 2.5265 secs, 18.1567 secs
  resp read:	0.0019 secs, 0.0004 secs, 0.0131 secs

Status code distribution:
  [200]	35 responses

Error distribution:
  [165]	Get https://next.data.gouv.fr/api/1/activity?page=1&page_size=20: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

[noirbizarre]

Yes, activity has not evolved since the first release when there was less than 1000 users.
The algorythm (dynamic fetching of activity items) can't stand as much data.
It was meant as a working base for a fanout model: one activity feed by user with static data as soon as activity happen.
The fanout model (which is well documented by orgs like Twitter, Facebook...) has many advantages beside performances:

• resilient against activity subject deletion (no foreign key, everything is stored into the user timeline)
• can handle read/unread status by user
• can easily handle private activity
• can be totaly extracted into its own service

but (because there is one), it requires a lot of storage.