Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Data Organisation Section to Docs #1553

Merged
merged 5 commits into from
May 9, 2024

Conversation

DrNickClarke
Copy link
Contributor

Docs only

Copy link
Contributor

@athakur91 athakur91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good Nick, have added a few comments in line

@ms041223
Copy link
Contributor

ms041223 commented May 8, 2024

For the second and third points in the "Data Update Process" section, its missing capital letters.

  • Delivery format (the format that the data source uses) vs storage format of the data. The trade-offs are
  • storing data in a format close to delivery format makes it easier to reconcile with the source
  • the delivery format is often not the best format for downstream processes
  • Frequency of data updates
  • Storage growth characteristics over time
  • System performance over time

@ms041223
Copy link
Contributor

ms041223 commented May 8, 2024

It is really good and will be super useful. Hopefully, this answers many of the questions we have been getting.

docs/mkdocs/docs/tutorials/data_organisation.md Outdated Show resolved Hide resolved
docs/mkdocs/docs/tutorials/data_organisation.md Outdated Show resolved Hide resolved
docs/mkdocs/docs/tutorials/data_organisation.md Outdated Show resolved Hide resolved
@DrNickClarke DrNickClarke merged commit 1c5a419 into master May 9, 2024
115 checks passed
@DrNickClarke DrNickClarke deleted the nc/docs_add_storage_layout_guide branch May 9, 2024 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants