Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterize sample data generator #1486

Merged
merged 10 commits into from Aug 8, 2023
Merged

Parameterize sample data generator #1486

merged 10 commits into from Aug 8, 2023

Conversation

lukesonnet
Copy link
Contributor

Features and Changes

  • Updates sample data generator to be easily customizable for use by the demo datasource project
  • Remove old data generator
  • Closes GB-299

Testing

Ran script, values and number of events in line with before change. Not pivotal to be identical!

@linear
Copy link

linear bot commented Jul 20, 2023

GB-299 Update data generation script

Update the data generator to be suitable for this project

@github-actions
Copy link

github-actions bot commented Jul 20, 2023

Your preview environment pr-1486-bttf has been deployed.

Preview environment endpoints are available at:

@lukesonnet lukesonnet marked this pull request as ready for review July 25, 2023 21:50
@lukesonnet lukesonnet requested a review from jdorn July 27, 2023 16:43
s.setDate(s.getDate() + start);
e.setDate(e.getDate() + end);
s.setDate(s.getDate() + startPct * dateLength);
e.setDate(e.getDate() + endPct * dateLength);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to consider using date-fns to clean up a lot of the date manipulation in this part of the codebase. Existing example below

const sevenDaysAgo = formatDate(subDays(new Date(), 7), dateFormat);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use addDays to clean this up in a few places but I didn't want to refactor everything else for this standalone script because much of it relies on in-place manipulations of the date. ty.


const NUM_USERS = 10000;
const OUTPUT_DIR = "/tmp/csv";
import { int } from "aws-sdk/clients/datapipeline";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely a typo. I'm guessing you meant to use number instead of int as a type somewhere and your IDE autocompleted to this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i can't shake the urge to int. fixed.

@lukesonnet lukesonnet requested a review from jdorn August 2, 2023 17:52
@lukesonnet lukesonnet merged commit 0201edd into main Aug 8, 2023
3 checks passed
@lukesonnet lukesonnet deleted the lsonnet/sim-data branch August 8, 2023 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants