Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: storage quota notifications #1156

Merged
merged 13 commits into from
Apr 13, 2022

Conversation

GaryHomewood
Copy link
Contributor

@GaryHomewood GaryHomewood commented Mar 23, 2022

This PR resolves the first 3 tasks of #990

  1. DB function to retrieve all users with storage used with a percentage from/to range of their allocated storage quota
  2. DB function to save details of sent email
  3. Node function in the cron package to call the DB functions to get the users that need emailing and save details of email sent.

Task 4 about the Mailchimp email service integration is in a separate ticket #1153 and PR.

@olizilla
Copy link
Contributor

what is 990? If it's an issue number pls just link to it in the PR description

@flea89 flea89 self-assigned this Mar 28, 2022
@@ -47,7 +47,7 @@ services:

cluster0:
container_name: cluster0
image: ipfs/ipfs-cluster:latest
Copy link
Contributor

@flea89 flea89 Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the latest version introduces some issue or is buggy?
Do we have that context?
If yes I wonder if we could add a comment and maybe a follow-up ticket to move this back to latest if/when possible?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ipfs-cluster just went 1.0 and made a couple of breaking changes which impact us.

packages/db/utils.js Outdated Show resolved Hide resolved
packages/db/index.d.ts Outdated Show resolved Hide resolved
packages/db/index.js Outdated Show resolved Hide resolved
@flea89
Copy link
Contributor

flea89 commented Mar 28, 2022

@GaryHomewood great work.
I think this needs merging/rebasing main to get the changes in from the other quota tikcket, isn't it?

I knew I would have likely struggled to get to the end of the review and tomorrow I'm not working.
It's not a super thorough, but hopefull you will already find something useful in there 🤞.
FYI I had a bit more time looking at the DB, so that's where most of my comments are.
I only scanned through the rest quickly.

As stated in one of my comments my main concern is in the way we get the users on the fly for a given quota range and the performance implications that that would have.
I'm sure PL folks we'll have some insights into it as well, but I wonder if we should deploy this as is ( once ready with the eail service) monitor the performances, and pivot if we realize is a problem?

END
$$;

CREATE TABLE IF NOT EXISTS email_notification_history
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My instinct is to call this table email_history, rather than email_notification_history, just because it keeps it as a more generic thing. I.e. not all emails are necessarily notifications (they could be "please click this link to verify your email address", or "click this link to log in", or "here's the receipt for your account charges this month").

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also makes sense. Made more generic.

Comment on lines 35 to 42
const { data } = await dbClient._client
.from('user_tag')
.upsert({
user_id: userId,
tag: options.tag || '',
value: options.value || '',
reason: options.reason || 'test reason'
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to be inline with the way the user_tags table is intended to work, this should really always do an insert, and set deleted_at on the previously-active row.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we could also move this function into packages/db/index.js and then it would achieve this for us at the same time: #1195

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have moved to the db package, now that its needed elsewhere.

} from './utils.js'
import { ConstraintError, DBError } from './errors.js'

import { emailType } from './constants.js'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use UPPERCASE here to hightlight is a constant?

Also pretty sure this could be written in one line

export { emailType } from './constants.js'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, have uppercased.

* @param {string} [tag.reason]
* @returns {Promise<boolean>}
*/
async createUserTag (userId, tag = {}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be done as part of #1195

  • It's not transactional, we might end up having deleted a tag without adding the new one.
  • The function is currently doing 2 api calls, not great for performance.
  • This is currently not tested

packages/db/postgres/functions.sql Show resolved Hide resolved
packages/db/postgres/functions.sql Outdated Show resolved Hide resolved
packages/db/postgres/tables.sql Show resolved Hide resolved
})
await emailService.sendEmails({
users,
email: EMAIL_TYPE.Used90PercentStorage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please double check all the email types in this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, have corrected and added to the test.

packages/cron/test/storage.spec.js Show resolved Hide resolved
@GaryHomewood GaryHomewood marked this pull request as ready for review April 6, 2022 09:48
@adamalton
Copy link
Contributor

adamalton commented Apr 6, 2022

One other thought on this is that I'm wondering whether we should skip the call to this.db.logEmailSent() in email.js for now, because it will currently log that it's sent an email even though it hasn't.

So as as we do #1178 before we actually set this going as a cron job then I guess it's fine. But it might be good to comment those lines out for safety?

Edit: that would break the tests, so possibly isn't worth it. Maybe we should just make sure we do #1178 before we add the cron?!

@GaryHomewood GaryHomewood changed the title feat: 990 storage quota notifications feat: storage quota notifications Apr 8, 2022
@flea89 flea89 merged commit 2889572 into feat/storage-quota-emails Apr 13, 2022
@flea89 flea89 deleted the feat/990-storage-quota-notifications branch April 13, 2022 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants