Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor to support multiple cloud providers #119

Merged
merged 5 commits into from
Mar 16, 2018

Conversation

infinitewarp
Copy link
Member

@infinitewarp infinitewarp commented Mar 13, 2018

#105

refactor many things to make more cloud-agnostic

@codecov
Copy link

codecov bot commented Mar 13, 2018

Codecov Report

Merging #119 into master will increase coverage by 0.08%.
The diff coverage is 99.73%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #119      +/-   ##
==========================================
+ Coverage   99.33%   99.41%   +0.08%     
==========================================
  Files          22       27       +5     
  Lines         905     1034     +129     
  Branches       48       53       +5     
==========================================
+ Hits          899     1028     +129     
  Misses          6        6
Impacted Files Coverage Δ
cloudigrade/util/tests/helper.py 100% <ø> (ø) ⬆️
cloudigrade/util/tests/test_helper.py 100% <ø> (ø) ⬆️
cloudigrade/util/aws.py 95.14% <ø> (ø) ⬆️
cloudigrade/account/tests/test_serializers.py 100% <100%> (ø) ⬆️
cloudigrade/account/serializers.py 100% <100%> (ø) ⬆️
cloudigrade/account/reports/helper.py 100% <100%> (ø)
cloudigrade/account/tests/helper.py 100% <100%> (ø) ⬆️
cloudigrade/account/reports/__init__.py 100% <100%> (ø)
cloudigrade/account/tests/test_reports.py 100% <100%> (ø) ⬆️
cloudigrade/account/tests/test_reports_aws.py 100% <100%> (ø)
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 627f71a...c031d1f. Read the comment docs.

@infinitewarp infinitewarp force-pushed the 105-refactor-models branch 4 times, most recently from e6d900f to c8dd30f Compare March 15, 2018 19:47
@infinitewarp infinitewarp changed the title WIP refactor to support multiple cloud providers refactor to support multiple cloud providers Mar 15, 2018

for event in events:
product_key = cloud_helper.get_event_product_identifier(event)
instance_key = cloud_helper.get_event_instance_identifier(event)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the Django ORM populate the instance object when pulling the initial events or will this make another DB call for each event?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Django's ORM is generally lazy/deferred in its execution, and it only executes queries when it thinks it needs to. In this case, the act of calling the iterator should have executed the query to get the events, and by including the select_related function in the chain, it should have pre-fetched the related data all at once.

See also:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related

Returns a QuerySet that will “follow” foreign-key relationships, selecting additional related-object data when it executes its query. This is a performance booster which results in a single more complex query but means later use of foreign-key relationships won’t require database queries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent!


class Meta:
model = Account
fields = ('id', 'url', 'account_id', 'account_arn')
read_only_fields = ('account_id', )
fields = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this guarantee the order the fields will be serialized in? If so: I like alphabetical, but having the dates together can be beneficial.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, and I actually believe DRF always automatically alphabetizes the fields when outputting as JSON. I think I just listed them here alphabetically for consistency sake.

"""
if cloud_provider not in CLOUD_PROVIDERS:
raise InvalidCloudProviderError(
_('Unsupported cloud provider "{0}".').format(cloud_provider)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we not doing the f'{cloud_provider}' string format method anymore?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mentioned this in a previous PR. I believe f-strings often don't work for gettext because of the order of execution. Which reminds me I need to file an issue to go fix/clean up all our incorrect f-strings…

Take this for example:

astronaut = 'Dave'
message = _(f"I'm sorry, {astronaut}.")

The value passed into the _ function would have already been substituted by the Python interpreter, meaning the translation function gets "I'm sorry, Dave." which should not be found in the translation mapping.

class AwsReportHelper(ReportHelper):
"""Report helper for AWS."""

def assert_account_exists(self): # noqa: D102, see parent class
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is any harm in inserting an aws specific docstring for these methods, better than littering it with noqa tags.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, doing this felt iffy, and I was wondering how you guys would feel about it. On the other hand, copy-pasting large swaths of doc strings doesn't feel any better. I wish there was a better supported way to stay DRY with them. 😕

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not, but in case we ever use anything to auto generate docs from the methods it would still be nice that there would be something written down.

adberglund
adberglund previously approved these changes Mar 16, 2018
adberglund
adberglund previously approved these changes Mar 16, 2018
@infinitewarp infinitewarp merged commit 10d1bf8 into master Mar 16, 2018
@infinitewarp infinitewarp deleted the 105-refactor-models branch March 16, 2018 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants