Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge release/2.023.37 #4687

Merged
merged 77 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
367782b
return plans and add-ons separately in subscriptionStore
LMNTL Sep 25, 2023
014e568
account for subscription add-ons in usage calculations, merge price a…
LMNTL Sep 25, 2023
49e1dbe
show free tier override based on owner join date instead of user join…
LMNTL Sep 25, 2023
487b6ea
Deleted projects not taken into account in storage calculation
noliveleger Sep 26, 2023
fad634f
Remove join queries to retrieve KC counters
noliveleger Sep 26, 2023
d752780
Split `_get_per_asset_usage()` into multiple smaller methods
noliveleger Sep 26, 2023
2ef043b
Fix unit tests
noliveleger Sep 26, 2023
be48d59
Add unit test for service usage with projects in trash bin
noliveleger Sep 26, 2023
c4cd7ad
Merge pull request #4661 from kobotoolbox/fix-user-storage-with-delet…
LMNTL Sep 26, 2023
00da7df
Coerce user list to a list to avoid joins
noliveleger Sep 26, 2023
be4fd6c
Merge pull request #4665 from kobotoolbox/remove-joins-daily-counters
jnm Sep 26, 2023
a0395d1
use detail view in service usage endpoint instead of POSTing
LMNTL Sep 26, 2023
a34a7db
add test for detail view
LMNTL Sep 26, 2023
8dddf6b
add warning to usage page that data may be cached
LMNTL Sep 26, 2023
9622dfb
allow promotion codes in stripe checkout
LMNTL Sep 26, 2023
2b0b061
refactor active_subscription_billing_details
LMNTL Sep 26, 2023
13e259d
GET from /service_usage/ORGANIZATION_ID/ instead of POST
LMNTL Sep 26, 2023
adcdf27
Merge branch 'release/2.023.37' into cache-service-usage
LMNTL Sep 26, 2023
c7ca817
Add check that socialaccounts don't use kobo as ID
JacquelineMorrissette Sep 21, 2023
f408e60
rename subscriptionResponse to planResponse
LMNTL Sep 27, 2023
2fcd101
default to user's join date if they're not in an organization
LMNTL Sep 27, 2023
4a4ea5f
Merge pull request #4653 from kobotoolbox/add-social-account-check
noliveleger Sep 27, 2023
fd8cfae
remove stray print statement
LMNTL Sep 27, 2023
4976146
remove banner from usage page warning about stale data
LMNTL Sep 27, 2023
9473816
don't save() if not needed
LMNTL Sep 27, 2023
5496a78
add formatRelativeTime to js utils
LMNTL Sep 27, 2023
13f25c4
add fetchWithHeaders utility function
LMNTL Sep 27, 2023
5ffe981
use fetchWithHeaders in getOrganization
LMNTL Sep 27, 2023
960e388
get request cache date in useUsage
LMNTL Sep 27, 2023
d3b099c
display 'last updated' time on Usage page
LMNTL Sep 27, 2023
51e8e6b
pull empty free tier display and threshold settings into constants
LMNTL Oct 2, 2023
ad66496
add tests for FREE_TIER_CUTOFF_DATE settings
LMNTL Oct 2, 2023
c95384c
fix failing test for FREE_TIER_CUTOFF_DATE
LMNTL Oct 2, 2023
de5c46c
create ENDPOINT_CACHE_DURATION setting from env var
LMNTL Oct 2, 2023
d1ba611
use env.int() instead of env.str()
LMNTL Oct 2, 2023
2714b0a
refactor /service_usage/{org_id}/ to /organizations/{org_id}/service_…
LMNTL Oct 2, 2023
56ce4ec
fix failing test, include cache warning in endpoint docs
LMNTL Oct 2, 2023
4a23b63
break up getAccountLimits
LMNTL Oct 2, 2023
01b9aad
use upper snake case for DEFAULT_LIMITS
LMNTL Oct 2, 2023
cde8e0f
remove get_serializer_context overload
LMNTL Oct 2, 2023
54ae214
remove get_serializer_context and dummy out organization check
LMNTL Oct 2, 2023
c639e7a
Merge pull request #4666 from kobotoolbox/cache-service-usage
noliveleger Oct 2, 2023
f6182b2
clean up types
LMNTL Oct 2, 2023
e23a0d9
Merge remote-tracking branch 'origin/release/2.023.37' into storage-a…
LMNTL Oct 2, 2023
c825783
Allow GET to /token/ to create a new API token
jnm Oct 3, 2023
8cb1ea9
Merge pull request #4672 from kobotoolbox/create-api-token-via-get-re…
noliveleger Oct 3, 2023
810cfcb
check organization usage but only for logged-in user
LMNTL Oct 3, 2023
15ddb18
Fix DDA reloading bug, empty submit crash bug.
Oct 3, 2023
89806fd
remove unused import
LMNTL Oct 3, 2023
e81a9e7
Merge pull request #4673 from kobotoolbox/service-usage-one-user-per-org
noliveleger Oct 3, 2023
038763f
Merge pull request #4674 from kobotoolbox/release/2.023.37-dda-UI-bug
JacquelineMorrissette Oct 4, 2023
0c0dfed
Do not allow to uncheck default when there is no one else
noliveleger Oct 7, 2023
bf809ab
Cannot create a per-asset disclaimer without global one
noliveleger Oct 10, 2023
391545e
Merge pull request #4677 from kobotoolbox/fix-enketo-disclaimer
jnm Oct 10, 2023
723f9b3
better comments
LMNTL Oct 10, 2023
ef72a2d
bump ES target for typescript
LMNTL Sep 28, 2023
0697ff7
update return type now that Object.entries is recognized
LMNTL Oct 10, 2023
b211bd6
refactor getFreeTierLimits
LMNTL Oct 10, 2023
d9b06af
Use custom manager for KobocatAttachment model
noliveleger Oct 11, 2023
8432f81
Remove undeclared variables
noliveleger Oct 11, 2023
4a244ee
Add unit tests for preview with form disclaimer
noliveleger Oct 11, 2023
1109451
Merge pull request #4679 from kobotoolbox/fix-preview-with-enketo-footer
LMNTL Oct 11, 2023
ca0732e
use first() when getting org owner join date
LMNTL Oct 12, 2023
5ed6a45
Merge pull request #4660 from kobotoolbox/storage-addons-frontend
LMNTL Oct 12, 2023
0488bfe
Make fullname required by default
noliveleger Oct 13, 2023
bf1ca5d
Fix unit tests
noliveleger Oct 13, 2023
6c40a42
Merge pull request #4682 from kobotoolbox/require-fullname-field
JacquelineMorrissette Oct 13, 2023
fc75e4a
Use "deleted_at" instead of "replaced_at"
noliveleger Oct 16, 2023
45babca
Merge branch 'release/2.023.37' into do-not-use-soft-deleted-attachments
noliveleger Oct 16, 2023
429feeb
Cast array type when using PostgreSQL array_position
noliveleger Oct 17, 2023
2b590f7
Apply requested changes for PR#4686
noliveleger Oct 17, 2023
eb1e7f3
Merge pull request #4686 from kobotoolbox/fix-array-position-postgres
bufke Oct 17, 2023
fc96ad6
Merge remote-tracking branch 'origin/release/2.023.37' into beta
LMNTL Oct 17, 2023
5e0a721
remove fetchWithHeaders and use fetchGet with options instead
LMNTL Oct 17, 2023
254d7b6
Merge pull request #4681 from kobotoolbox/do-not-use-soft-deleted-att…
LMNTL Oct 17, 2023
5146d29
Merge branch 'release/2.023.37' into merge-release/2.023.37
LMNTL Oct 17, 2023
30f5536
remove redundant conditional types
LMNTL Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
178 changes: 143 additions & 35 deletions jsapp/js/account/stripe.api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,17 @@ export interface Organization {
slug: string;
}

enum Limits {
'unlimited' = 'unlimited',
}

type LimitAmount = number | Limits.unlimited;

export interface AccountLimit {
submission_limit: 'unlimited' | number;
nlp_seconds_limit: 'unlimited' | number;
nlp_character_limit: 'unlimited' | number;
storage_bytes_limit: 'unlimited' | number;
submission_limit: LimitAmount | number;
nlp_seconds_limit: LimitAmount | number;
nlp_character_limit: LimitAmount | number;
storage_bytes_limit: LimitAmount | number;
LMNTL marked this conversation as resolved.
Show resolved Hide resolved
}

export interface Product extends BaseProduct {
Expand All @@ -73,6 +79,13 @@ export interface Portal {
url: string;
}

const DEFAULT_LIMITS: AccountLimit = Object.freeze({
submission_limit: Limits.unlimited,
nlp_seconds_limit: Limits.unlimited,
nlp_character_limit: Limits.unlimited,
storage_bytes_limit: Limits.unlimited,
});

export async function getProducts() {
return fetchGet<PaginatedResponse<Product>>(endpoints.PRODUCTS_URL);
}
Expand All @@ -87,29 +100,38 @@ export async function getOrganization() {
return fetchGet<PaginatedResponse<Organization>>(endpoints.ORGANIZATION_URL);
}

/**
* Start a checkout session for the given price and organization. Response contains the checkout URL.
*/
export async function postCheckout(priceId: string, organizationId: string) {
return fetchPost<Checkout>(
`${endpoints.CHECKOUT_URL}?price_id=${priceId}&organization_id=${organizationId}`,
{}
);
}

/**
* Get the URL of the Stripe customer portal for an organization.
*/
export async function postCustomerPortal(organizationId: string) {
return fetchPost<Portal>(
`${endpoints.PORTAL_URL}?organization_id=${organizationId}`,
{}
);
}

/**
* Get the subscription interval (`'month'` or `'year'`) for the logged-in user.
* Returns `'month'` for users on the free plan.
*/
export async function getSubscriptionInterval() {
await when(() => envStore.isReady);
if (envStore.data.stripe_public_key) {
if (!subscriptionStore.isPending && !subscriptionStore.isInitialised) {
subscriptionStore.fetchSubscriptionInfo();
}
await when(() => subscriptionStore.isInitialised);
const subscriptionList: SubscriptionInfo[] =
subscriptionStore.subscriptionResponse;
const subscriptionList: SubscriptionInfo[] = subscriptionStore.planResponse;
const activeSubscription = subscriptionList.find((sub) =>
ACTIVE_STRIPE_STATUSES.includes(sub.status)
);
Expand All @@ -120,19 +142,104 @@ export async function getSubscriptionInterval() {
return 'month';
}

export async function getAccountLimits() {
/**
* Extract the limits from Stripe product/price metadata and convert their values from string to number (if necessary.)
* Will only return limits that exceed the ones in `limitsToCompare`, or all limits if `limitsToCompare` is not present.
*/
function getLimitsForMetadata(
metadata: {[key: string]: string},
limitsToCompare: false | AccountLimit = false
) {
const limits: Partial<AccountLimit> = {};
for (const [key, value] of Object.entries(metadata)) {
// if we need to compare limits, make sure we're not overwriting a higher limit from somewhere else
if (limitsToCompare) {
if (!(key in limitsToCompare)) {
continue;
}
if (
key in limitsToCompare &&
value !== Limits.unlimited &&
value <= limitsToCompare[key as keyof AccountLimit]
) {
continue;
}
}
// only use metadata needed for limit calculations
if (key in DEFAULT_LIMITS) {
limits[key as keyof AccountLimit] =
value === Limits.unlimited ? Limits.unlimited : parseInt(value);
}
}
return limits;
}

/**
* Get limits for the custom free tier (from `FREE_TIER_THRESHOLDS`), and merges them with the user's limits.
* The `/environment/` endpoint handles checking whether the logged-in user registered before `FREE_TIER_CUTOFF_DATE`.
*/
const getFreeTierLimits = async (limits: AccountLimit) => {
await when(() => envStore.isReady);
const thresholds = envStore.data.free_tier_thresholds;
const newLimits: AccountLimit = {...limits};
if (thresholds.storage) {
newLimits['storage_bytes_limit'] = thresholds.storage;
}
if (thresholds.data) {
newLimits['submission_limit'] = thresholds.data;
}
if (thresholds.translation_chars) {
newLimits['nlp_character_limit'] = thresholds.translation_chars;
}
if (thresholds.transcription_minutes) {
newLimits['nlp_seconds_limit'] = thresholds.transcription_minutes * 60;
}
return newLimits;
};

/**
* Get limits for any recurring add-ons the user has, merged with the rest of their limits.
*/
const getRecurringAddOnLimits = (limits: AccountLimit) => {
let newLimits = {...limits};
let activeAddOns = [...subscriptionStore.addOnsResponse];
let metadata = {};
// only check active add-ons
activeAddOns = activeAddOns.filter((subscription) =>
ACTIVE_STRIPE_STATUSES.includes(subscription.status)
);
if (activeAddOns.length) {
activeAddOns.forEach((addOn) => {
metadata = {
...addOn.items[0].price.product.metadata,
...addOn.items[0].price.metadata,
};
newLimits = {...newLimits, ...getLimitsForMetadata(metadata, newLimits)};
});
}
return newLimits;
};

/**
* Get all metadata keys for the logged-in user's plan, or from the free tier if they have no plan.
*/
const getStripeMetadataAndFreeTierStatus = async () => {
await when(() => subscriptionStore.isInitialised);
const subscriptions = [...subscriptionStore.subscriptionResponse];
const activeSubscriptions = subscriptions.filter((subscription) =>
const plans = [...subscriptionStore.planResponse];
// only use metadata for active subscriptions
const activeSubscriptions = plans.filter((subscription) =>
ACTIVE_STRIPE_STATUSES.includes(subscription.status)
);
let metadata;
let hasFreeTier = false;
if (activeSubscriptions.length) {
// get metadata from the user's subscription
metadata = activeSubscriptions[0].items[0].price.product.metadata;
// get metadata from the user's subscription (prioritize price metadata over product metadata)
metadata = {
...activeSubscriptions[0].items[0].price.product.metadata,
...activeSubscriptions[0].items[0].price.metadata,
};
} else {
// the user has no subscription, so get limits from the free monthly price
// the user has no subscription, so get limits from the free monthly product
hasFreeTier = true;
try {
const products = await getProducts();
Expand All @@ -141,40 +248,41 @@ export async function getAccountLimits() {
(price: BasePrice) =>
price.unit_amount === 0 && price.recurring?.interval === 'month'
)
);
)[0];
metadata = {
...freeProduct[0].metadata,
...freeProduct[0].prices[0].metadata,
...freeProduct.metadata,
...freeProduct.prices[0].metadata,
};
} catch (error) {
// couldn't find the free monthly product, continue in case we have limits to display from the free tier override
metadata = {};
}
}
return {metadata, hasFreeTier};
};

/**
* Get the complete account limits for the logged-in user.
* Checks (in descending order of priority):
* - the user's recurring add-ons
* - the `FREE_TIER_THRESHOLDS` override
* - the user's subscription limits
*/
export async function getAccountLimits() {
const {metadata, hasFreeTier} = await getStripeMetadataAndFreeTierStatus();

// initialize to unlimited
const limits: AccountLimit = {
submission_limit: 'unlimited',
nlp_seconds_limit: 'unlimited',
nlp_character_limit: 'unlimited',
storage_bytes_limit: 'unlimited',
};
let limits: AccountLimit = {...DEFAULT_LIMITS};

// get the limits from the metadata
for (const [key, value] of Object.entries(metadata)) {
if (Object.keys(limits).includes(key)) {
limits[key as keyof AccountLimit] =
value === 'unlimited' ? value : parseInt(value);
}
}
// apply any limits from the metadata
limits = {...limits, ...getLimitsForMetadata(metadata)};

// if the user is on the free tier, overwrite their limits with whatever free tier limits exist
if (hasFreeTier) {
await when(() => envStore.isReady);
const thresholds = envStore.data.free_tier_thresholds;
thresholds.storage && (limits['storage_bytes_limit'] = thresholds.storage);
thresholds.data && (limits['submission_limit'] = thresholds.data);
thresholds.translation_chars && (limits['nlp_character_limit'] = thresholds.translation_chars);
thresholds.transcription_minutes && (limits['nlp_seconds_limit'] = thresholds.transcription_minutes * 60);
// if the user is on the free tier, overwrite their limits with whatever free tier limits exist
limits = await getFreeTierLimits(limits);

// if the user has active recurring add-ons, use those as the final say on their limits
limits = getRecurringAddOnLimits(limits);
}

return limits;
Expand Down
6 changes: 3 additions & 3 deletions jsapp/js/account/stripe.utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@ export async function hasActiveSubscription() {
}

await when(() => subscriptionStore.isInitialised);
const subscriptions = subscriptionStore.subscriptionResponse;
if (!subscriptions.length) {
const plans = subscriptionStore.planResponse;
if (!plans.length) {
return false;
}

return (
subscriptions.filter(
plans.filter(
(sub) =>
ACTIVE_STRIPE_STATUSES.includes(sub.status) &&
sub.items?.[0].price.unit_amount > 0
Expand Down
16 changes: 12 additions & 4 deletions jsapp/js/account/subscriptionStore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,9 @@ export async function fetchProducts() {
}

class SubscriptionStore {
public subscriptionResponse: SubscriptionInfo[] = [];
public planResponse: SubscriptionInfo[] = [];
public addOnsResponse: SubscriptionInfo[] = [];
public subscribedProduct: BaseProduct | null = null;
public productsResponse: Product[] | null = null;
public isPending = false;
public isInitialised = false;

Expand Down Expand Up @@ -121,8 +121,16 @@ class SubscriptionStore {
private onFetchSubscriptionInfoDone(
response: PaginatedResponse<SubscriptionInfo>
) {
this.subscriptionResponse = response.results;
this.subscribedProduct = response.results[0]?.plan?.product || null;
// get any plan subscriptions for the user
this.planResponse = response.results.filter(
(sub) => sub.items[0]?.price.product.metadata?.product_type == 'plan'
);
// get any recurring add-on subscriptions for the user
this.addOnsResponse = response.results.filter(
(sub) => sub.items[0]?.price.product.metadata?.product_type == 'addon'
);
this.subscribedProduct =
this.planResponse[0]?.items[0]?.price.product || null;
this.isPending = false;
this.isInitialised = true;
}
Expand Down
12 changes: 8 additions & 4 deletions jsapp/js/account/usage.api.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import {fetchGet, fetchPost} from 'jsapp/js/api';
import {fetchGet} from 'jsapp/js/api';
import {getOrganization} from 'js/account/stripe.api';
import {createContext} from 'react';

interface AssetUsage {
asset: string;
Expand Down Expand Up @@ -33,12 +32,17 @@ export interface UsageResponse {
}

const USAGE_URL = '/api/v2/service_usage/';
const ORGANIZATION_USAGE_URL =
'/api/v2/organizations/##ORGANIZATION_ID##/service_usage/';

export async function getUsage(organization_id: string | null = null) {
if (organization_id) {
return fetchPost<UsageResponse>(USAGE_URL, {organization_id});
return fetchGet<UsageResponse>(
ORGANIZATION_USAGE_URL.replace('##ORGANIZATION_ID##', organization_id),
{includeHeaders: true}
);
}
return fetchGet<UsageResponse>(USAGE_URL);
return fetchGet<UsageResponse>(USAGE_URL, {includeHeaders: true});
}

export async function getUsageForOrganization() {
Expand Down
12 changes: 11 additions & 1 deletion jsapp/js/account/usage.component.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,17 @@ export default function Usage() {

return (
<div className={styles.root}>
<h2>{t('Your account total use')}</h2>
<header className={styles.header}>
<h2>{t('Your account total use')}</h2>
{typeof usage.lastUpdated === 'string' && (
<p className={styles.updated}>
{t('Last update: ##LAST_UPDATE_TIME##').replace(
'##LAST_UPDATE_TIME##',
usage.lastUpdated
)}
</p>
)}
</header>
<UsageContext.Provider value={usage}>
<LimitNotifications usagePage />
</UsageContext.Provider>
Expand Down
13 changes: 13 additions & 0 deletions jsapp/js/account/usage.module.scss
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,19 @@ h2 {
}
}

.header {
display: flex;
width: 100%;
justify-content: space-between;
align-items: center;
}

.updated {
border-radius: 0.25em;
background-color: colors.$kobo-cloud;
padding: 0.5em;
}

.row {
display: flex;
justify-content: center;
Expand Down