Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse args for features / datasources of interest #115

Closed
wants to merge 3 commits into from

Conversation

arlolra
Copy link
Member

@arlolra arlolra commented Jan 12, 2016

A start on #101

Some questions inline.


params = {}
if request.method == "POST":
data = request.get_json() # Assume POSTs are JSON.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just let POST data be x-www-form-urlencoded and use the same code to parse?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People like JSON? Not sure.

We could add a check for request.headers.get("Content-Type") == "application/json" and accept all three?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a lot of complication. The cache can't have a hierarchy to it. It's only one-dimensional. Since JSON object keys must be strings, I'm not sure if we get any benefit from JSON here. We'd be implementing JSON-soap.

@arlolra arlolra closed this Jan 12, 2016
@arlolra arlolra deleted the gh101 branch January 12, 2016 21:52
@arlolra arlolra restored the gh101 branch January 12, 2016 21:56
@arlolra arlolra reopened this Jan 12, 2016
@arlolra
Copy link
Member Author

arlolra commented Jan 12, 2016

Whoops, didn't mean to close that.

# Is this right? What's a precache?
precache = ("precache" in request.args) and (len(params) == 0)

# FIXME: Do something with `params`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scores_processor.score(content, model, [rev_id], cache=params, precache=precache)

@halfak
Copy link
Member

halfak commented Jan 22, 2016

Sorry for the delay here. I'm working on a write-up of the different ways we can have users provide dependencies for feature extraction for feedback. Should have an update this weekend.

@arlolra
Copy link
Member Author

arlolra commented Jan 22, 2016

Sure, no rush. Enjoy your weekend :)

@halfak
Copy link
Member

halfak commented Mar 9, 2016

I got a write-up together of what I see as the two good options we have. See #101 (comment)

@arlolra
Copy link
Member Author

arlolra commented Mar 14, 2016

Thanks, taking a look.

# Parse values for features / datasources of interest.
try:
for k, v in request.values.items():
if k == "caches":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the discussion, you said cache but argument to score() is caches.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh yes. That's right. I was imagining that we'd only support cached value injection on the /<context>/<model>/<revid>/ route where caches would be a simple {<revid>: { ... our flat JSON ... }}.

@arlolra
Copy link
Member Author

arlolra commented Mar 16, 2016

Updated for v1 / v2.

@@ -61,3 +63,18 @@ def format_output(context, scores, model_info, warning=None, notice=None):
for rev_id in scores[model]:
output['scores'][context][model]['scores'][rev_id] = scores[model][rev_id]
return jsonify(output)


# Parse values for features / datasources of interest.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should put documents in docstrings after function definition

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP256

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but there are number of other PEP8 warnings in the library. Should they be fixed? Should CI not pass until they are?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix all of PEP8 issues in another PR, don't worry. It'll pass without this but I'm saying you're providing documentation, do it in the most proper way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Q. Don't worry about pep 8 issues on the rest of the file structure (unless you want to). We can get those in a follow up PR.

@arlolra
Copy link
Member Author

arlolra commented Mar 16, 2016

This latest build failure seems unrelated.

@arlolra
Copy link
Member Author

arlolra commented Mar 16, 2016

Or not, nevermind, let me fix that.

@arlolra
Copy link
Member Author

arlolra commented Mar 16, 2016

Should be good to go now.

@@ -70,7 +70,7 @@ def extract_roots(self, model, rev_ids, caches=None):
root_ds = [d for d in dependencies.dig(features)
if isinstance(d, Datasource)]
error_root_vals = self.extractor.extract(rev_ids, root_ds,
cache=caches)
caches=caches)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't match the behavior of revscoring

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK fair point. I'm just going to fix this in revscoring now. Will have a PR shortly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wikimedia/revscoring#251

You'll also need to make a release and bump the requirements.txt here.

@halfak
Copy link
Member

halfak commented Mar 26, 2016

This was merged, but I had to rebase on the way so it seems to have not been caught. Thanks for your work, @arlolra. We'll be deploying this soon. I'll let you know.

@arlolra
Copy link
Member Author

arlolra commented Mar 26, 2016

- caches=caches
+ caches={rev_id: cache},

I see. Thanks @halfak!

@arlolra arlolra closed this Mar 26, 2016
@arlolra arlolra deleted the gh101 branch March 26, 2016 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants