Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

E-mail handling #12

Closed
nbumbarger opened this issue Nov 21, 2016 · 8 comments
Closed

E-mail handling #12

nbumbarger opened this issue Nov 21, 2016 · 8 comments

Comments

@nbumbarger
Copy link
Collaborator

The wireframes suggest that the user can expect an e-mail when processing is complete. How will this be implemented on the server, and how should the form be designed to accommodate this feature? For example:

  • Should the API return a unique task ID (potentially just the URL it already returns), which can be PUT at a particular API alongside the user's e-mail?
  • Should the user's e-mail be sent as another query string?
  • Should user's e-mails be stored in the database with their tokens, removing the need for e-mail submission? I had assumed this would be the case before seeing this comment: WIP Feature/layout #8 (comment)
@nbumbarger nbumbarger mentioned this issue Nov 21, 2016
Closed
@jflasher
Copy link

The trick here is that none of the adapters (where the data is brought in to be ingested) touch the database and it'd be nice to keep it that way. I think the easiest way to handle this within openaq-fetch is if you just ask them at submit time what their email address is if they want to be emailed (either modal or just one extra form input field) and just stick that on every measurement. It'll be somewhat needlessly ugly, but they'll never see the final submitted CSV. This way, the adapter will have the email at submit time. I had thought we'd do a separate meta file for each measurements CSV that'd have this info, but not sure we need that if this is the only thing that'd go in it.

@nbumbarger
Copy link
Collaborator Author

Alright, that seems like a fine workaround.

@nbumbarger
Copy link
Collaborator Author

I thought of a potential implementation problem, which may not actually be a problem, but the S3 bucket is expecting a file, correct? Right now, the page parses the input as JSON then sends the original file to the pre-signed URL- it doesn't need to actually modify the CSV data. Would it be possible to modify the CSV data inline then send that data to the bucket, rather than the file object?

@nbumbarger
Copy link
Collaborator Author

@jflasher I haven't been able to find much about uploading non-file data to S3, although it seems like it would have to be possible. Do you think that constructing a file object, eg. this post, might do the trick? Or is there a better-sanctioned way to go about it?

@jflasher
Copy link

When testing before, I was just uploading text where the file is currently being uploaded so I think you can upload whatever string you want there. You should have S3 credentials to download and check whatever was uploaded.

On November 22, 2016 at 21:09:39, Nick Bumbarger (notifications@github.com(mailto:notifications@github.com)) wrote:

@jflasher(https://github.com/jflasher) I haven't been able to find much about uploading non-file data to S3, although it seems like it would have to be possible. Do you think that constructing a file object, eg. this post(http://stackoverflow.com/questions/8390855/how-to-instantiate-a-file-object-in-javascript), might do the trick? Or is there a better-sanctioned way to go about it?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub(#12 (comment)), or mute the thread(https://github.com/notifications/unsubscribe-auth/AAz0JkjdH3un1C1jjBa7uzF8BkBuAP3qks5rAwy7gaJpZM4K4Yj6).

@nbumbarger
Copy link
Collaborator Author

Hmm, I got a somewhat contradictory error when I pasted the exact test you used in this issue (using the pre-signed URL from the API) into the uploader page. It's

Upload successful!  Server responded with: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>ASIAJY4C5RDHCG7DEXCA</AWSAccessKeyId><StringToSign>PUT


1479942915
x-amz-security-token:FQoDYXdzEK3//////////wEaDOVEuDVMzjgiVO7XtSKqA6xl5ddRoTWAhxKh5cYIzduZ78IEoK3+AJ/Pr1GPR4EX8HDCdp3XLR0e6ej0vXObc+L2IerrP8dk7DLEQtgnQURencmBXVWh6wTgxKeB0XIBVaSh+kdRCcqccmYJ7s2w/Iz/E4U7T4HcwSW8W9JyE4D7wb1tnRMUwv4tsqsKMZtevY0R40u/Tp54ho2R2khC5ZZDzSGYTNp4NBoqJP5rW2ajRMSdvwWGyg7AzzMczxuN5cu/tkGAY6phyfjX8FfaoOK2rE7zmGXDP62LpeCBMYy/6/ls5bHgIsR2CDj3Vu07PHN3ytVOIIOqwN+SOqgnDj1RbQaCz3bvfxAZj9FLNGFA2MBV3qXx7NFZtEr1DHCQ0dxJE2gZW5gg4mOW98QSQTXZwVntO3+47hsvp86wJ92aE/QvC6fTUpKugw4lesgQ7E0KE2yTykfTbAku3+91FMCACJbvipPXWDAEICk24DSiZ9ZjgwOyoZRq0aDX9CWfWmq3/ZFR4J6sd5rzGqC+kHLtlVC4p9LcpBtY2zLxKR9tlwiXtlM4Q6XdyozwH8ycjVKGeZLX22+0vSiP7dfBBQ==
/openaq-uploads-staging/passing-with-optional-columns.csv</StringToSign><SignatureProvided>0cpDMztYkww8TOwucCNxzeqJOEI=</SignatureProvided>

Any ideas? I'm out tomorrow but I will be back at work on Friday. Happy Thanksgiving!

@jflasher
Copy link

jflasher commented Nov 24, 2016 via email

@nbumbarger nbumbarger mentioned this issue Nov 28, 2016
Closed
@nbumbarger
Copy link
Collaborator Author

#16 adds email as an additional column in the transmitted CSV data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jflasher @nbumbarger