Skip to content

Log API Requests #274

@nelsonic

Description

@nelsonic

With the creation of the API #273 and our goal of making it easy for anyone to request their @dwyl App data via JSON/WebSockets, we need to have the means of logging requests in place quite soon ... 💭 ⏳

Story

As a developer building an App with an API,
I want to have logging of all API requests
So that we can detect anomalous usage patterns and guarantee service levels.

Note: by having API request logging we will automatically have UI request logging because we are using Content Negotiation to render JSON for API and HTML for UI requests to the same route.
This issue is specific to API Logging because it's the highest "risk" for abuse and potential data breach. If someone accidentally leaks their AUTH_API_KEY e.g. by committing some code to GitHub, they could leak all their data. We need to avoid that by having logging to prevent new devices from accessing data. i.e. we need to log the device ID + IP address of all API requests so that we can check if new devices/IPs are being used and force a re-auth.

Todo

  • Create a logs Schema/Table (_for MVP we will store the logs in the same DB, we can split them out into a separate service later or use a 3rd Party Service like Timber.io, Logstash, etc.
  • email - encrypted email of the person the login attempt was made for. This allows us to keep track of how many attempts were made for a given account in a set time frame. If not set, leave null
  • person_id - if the request is authenticated, log the person_id otherwise just the email so that we can later analyse the failed login attempts. (e.g: typos)
  • apikey_id - the AUTH_API_KEY used for the request.
  • ip_address - so we can rate limit by person_id and ip_address and so that we can inform people when their account has been accessed from an unrecognised IP.
    see: https://github.com/dwyl/hits/blob/cd9a8e15c6e598281b7bc6b037963dab6ac515f8/lib/hits_web/controllers/hit_controller.ex#L32-L33
  • user_agent_id - same as in Hits, we will have a user_agents schema where we store the full User Agent string then we reference it in the log rather than duplicating it.
    See: https://github.com/dwyl/hits/blob/cd9a8e15c6e598281b7bc6b037963dab6ac515f8/lib/hits.ex#L48-L58

This will eventually be replaced by "Client Hints" for better privacy (according to Google) 🙄
see: https://www.zdnet.com/article/google-to-phase-out-user-agent-strings-in-chrome )
Probably not for the next year. So for now, User Agent Strings. 👍

  • timstamps (default in all Ecto/Phoenix schemas)

This is related to: Logging "Login" Attempts: dwyl/auth#67 (P2)
and Logging in General: dwyl/learn-devops#60

Metadata

Metadata

Assignees

No one assigned

    Labels

    APIMVPdiscussShare your constructive thoughts on how to make progress with this issueenhancementNew feature or enhancement of existing functionalitypriority-2Second highest priority, should be worked on as soon as the Priority-1 issues are finishedtechnicalA technical issue that requires understanding of the code, infrastructure or dependencies

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions