New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Harvest config backend #1716
Harvest config backend #1716
Conversation
udata/harvest/actions.py
Outdated
@@ -63,7 +63,8 @@ def create_source(name, url, backend, | |||
description=None, | |||
frequency=DEFAULT_HARVEST_FREQUENCY, | |||
owner=None, | |||
organization=None): | |||
organization=None, | |||
config=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe move )
to a new line and add a trailing comma?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@@ -121,9 +121,19 @@ def backends_ids(): | |||
source_page_fields = api.model('HarvestSourcePage', | |||
fields.pager(source_fields)) | |||
|
|||
|
|||
filter_fields = api.model('HarvestFilter', { | |||
'label': fields.String(description='A localized human-readable label'), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Localized or localizable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Localized as its the documentation for an API consumer and when he gets the field, the value is localized according to the lang
parameter.
int: 'integer', | ||
bool: 'boolean', | ||
UUID: 'uuid', | ||
datetime: 'date-time', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is date-time
a known convention? I would have written datetime
, but not important.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, those are Swagger/OpenAPI/json-schema standard types
udata/harvest/backends/base.py
Outdated
@@ -35,6 +58,10 @@ class BaseBackend(object): | |||
display_name = None | |||
verify_ssl = True | |||
|
|||
# Define some allowed filters on the backend | |||
# This a Sequence[HarvestFilter] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
double space
udata/harvest/forms.py
Outdated
def get_backend(self, form): | ||
return [ | ||
b for b in list_backends() if b.name == form.backend.data | ||
][0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe next((a for a in [1, 2]))
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
udata/harvest/forms.py
Outdated
|
||
def get_specs(self, backend, key): | ||
candidates = [f for f in backend.filters if f.key == key] | ||
return candidates[0] if len(candidates) == 1 else None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe next((a for a in [1, 2]), None)
?
udata/harvest/forms.py
Outdated
if self.data: | ||
backend = self.get_backend(form) | ||
# Validate filters | ||
for f in (self.data.get('filters', None) or []): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need for , None
I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or better self.data.get('filters', [])
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sadly self.data.get('filters', [])
doesn't cover the case where filters is None
or I have to write self.data.get('filters', []) or []
.
I'll go for self.data.get('filters') or []
udata/harvest/tests/test_api.py
Outdated
@@ -115,6 +119,100 @@ def test_create_source_with_org_not_member(self, api): | |||
|
|||
assert403(response) | |||
|
|||
def test_create_source_with_config(self,api): | |||
'''It should create anew source with configuration''' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/anew/a new/
udata/harvest/forms.py
Outdated
|
||
def get_specs(self, backend, key): | ||
candidates = [f for f in backend.filters if f.key == key] | ||
return candidates[0] if len(candidates) == 1 else None | ||
return next(candidates, None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you need a tuple comprehension for this to work.
udata/harvest/forms.py
Outdated
b for b in list_backends() if b.name == form.backend.data | ||
][0] | ||
]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you need a tuple comprehension for this to work.
This PR:
HarvestSource.config
object is properly writable (with validation)Requires #1715 (included)