Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(models): Adding 'created', 'lastModified' timestamp to Dataset, Container, Dashboard, Chart #6527

Merged

Conversation

jjoyce0510
Copy link
Collaborator

Summary

Adding a 'createdAt' timestamp to Datasets, Containers, DataJobs, DataFlows. This is useful for reporting the native platform created time for data assets when ingesting into DataHub.

We are not using an AuditStamp here because the Actor field inside is required. We do not always know that the actor will be resolvable to an URN for created actions, so we are choosing to omit for now.

Status

Ready for review.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions
Copy link

github-actions bot commented Nov 23, 2022

Unit Test Results (metadata ingestion)

       8 files         8 suites   1h 1m 14s ⏱️
   765 tests    762 ✔️ 2 💤 1
1 532 runs  1 526 ✔️ 5 💤 1

For more details on these failures, see this check.

Results for commit 55e1f25.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Nov 23, 2022

Unit Test Results (build & test)

621 tests  ±0   617 ✔️ ±0   15m 56s ⏱️ +17s
157 suites ±0       4 💤 ±0 
157 files   ±0       0 ±0 

Results for commit 55e1f25. ± Comparison against base commit 6eb63c2.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

why no modifiedAt field?

@shirshanka
Copy link
Contributor

How would we record the actor who created the resource? Maybe create a variant of AuditStamp which loosens the restriction on actor being required but keeps the other goodies (like impersonator) and use it for the createdAt field?

@jjoyce0510
Copy link
Collaborator Author

Actor I didn't want to require. If we can actually get a correct actor (one which lines up with other urns) we'd need to add those fields later on.

As for lastModified, it's a good question. We typically used Operation.pdl for this but I think it makes sense to reserve that for more rich details around the operations, and have a separate lastModified stamp here.

Let me do this - I'll create a new TimeStamp.pdl that will basically allow the Actor field to be not provided, so the UI knows when we do not have one.

@jjoyce0510 jjoyce0510 changed the title feat(models): Adding 'createdAt' timestamp to Dataset, Container, Dashboard, Chart feat(models): Adding 'created', 'lastModified' timestamp to Dataset, Container, Dashboard, Chart Nov 23, 2022
@jjoyce0510 jjoyce0510 merged commit d1b1224 into datahub-project:master Nov 29, 2022
cccs-Dustin pushed a commit to CybercentreCanada/datahub that referenced this pull request Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants