Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V0.3.7 api refactor #528

Merged
merged 77 commits into from
Nov 20, 2018
Merged

V0.3.7 api refactor #528

merged 77 commits into from
Nov 20, 2018

Conversation

fredkingham
Copy link
Contributor

@fredkingham fredkingham commented Oct 10, 2018

So this is the first (and the worst) of the api refactor pull requests.

Old state was

  • An API that provided all database access.
  • A loader file that would get the data from the api and pass it to updater files, e.g. update_lab_tests
  • There was a single batch load cron job that would run every hour.

New State

  • There are many services, each with their own cron jobs, backend and service.
  • The base service provides utilities to help the services.
  • Each service defines a /backend/ directory and a live.API and a dev.API that is used depending on a setting.
  • Each service has a service.py that requests data from the api and saves.
  • There will be one batch loads for each service.

@coveralls
Copy link

coveralls commented Oct 10, 2018

Coverage Status

Coverage increased (+2.0%) to 76.611% when pulling 9f3bfbb on v0.3.7-api-refactor into 4f610d2 on v0.3.7.

fabfile.py Outdated Show resolved Hide resolved
from intrahospital_api import logger

api = get_api()
from intrahospital_api import get_api
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have all the from intrahospital_api import x statements on one line ?

batch.start()
def initial_load(remaining=False):
"""
Runs an initial load.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a great docstring comment - it assumes I know what running an initial load is, when I would do it etc...

demographics_row = DemographicsRow(db_row)

demographics_row["hospital_number"] == "1"
demographics_row["name"] == "Wilma Flintstone"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely think it's weird to access @propertys this way...

@@ -196,7 +200,6 @@
database=None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, Django tends to do these as shouty ALL_CAPS keys....


if time_ago.seconds < MAX_ALLOWABLE_BATCH_RUN_TIME:
logger.info(
"batch still running after {} seconds, skipping".format(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we "skip" at the point of this log message?

def get_batch_start_time(service_name):
"""
If no previous successful batch run has started.
Use the started timestamp of the first Initial Patient Load
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First ? I thought there was never > 1 ?


if current_running.count() > 1:
# we should never have multiple batches running at the same time
raise BatchLoadError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You prefer to do your own checks and raise your own errors not have e.g. a Meta unique_together constraint ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So its not unique together on state as we should have lots of batch runs that are successful. However you're right, we don't want to blow up here, we want to blow up when duplicates are added. I'll put an error in the save method

If no previous successful batch run has started.
Use the started timestamp of the first Initial Patient Load

If a previous batch load has run and there was an initial patient
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a diagram that explains this ?
I'm only 75% sure I follow the text :)

… adds in a unit test for the refresh patient function
…ab test we are in at this point as otherwise we get the meta class
@davidmiller davidmiller merged commit 1af8e5d into v0.3.7 Nov 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants