Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UT new logic #4

Merged
merged 15 commits into from
May 11, 2023
Merged

Conversation

mrpalide
Copy link
Contributor

@mrpalide mrpalide commented May 4, 2023

Fixes: #2

Changes:

  • remove old monthly model
  • remove old data once a day. Delete old data as default more than 7 days, but this value could change by --store-data-cutoff flag
  • add new flag --store-data-path for path of save daily data, default value is /var/lib/skywire-ut/daily-data
  • save last day data to json file, whole of db, as YYYY-MM-DD-uptime-data.json
  • decided to return old data as endpoint or store/upload in storage

Screenshots:
image
image
image

How to test:

  • run uptime-tracker by README.md manual
  • run visor and use local UT
  • wait 5 minutes, check endpoint and db
  • stop UT, change create_at value of record to yesterday by update daily_uptime_histories set created_at = current_date - 1
  • add api.dailyRoutine(logger) to line 153 of api.go file, build and run UT again
  • check endpoint, db and store data path for daily data json file there

@@ -153,6 +158,10 @@ func (api *API) RunBackgroundTasks(ctx context.Context, logger logrus.FieldLogge
return
case <-cacheTicker.C:
api.updateInternalCaches(logger)
if time.Now().Format("2006-01-02") > date {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, I would like the routine to look like this:

oldestEntry := dbClient.Limit(1).Order("created_at asc")Find(&entry)
from := oldestEntry.createdAt.Format("2006-01-02")
to := time.Now().Add(0, 0 , -(storeCutoffDate))


if (from < to) {
for from; from < to; from.Add(0, 0, 1) {
  dbClient.Limit(1).Where("date == from").FInd(&exportDate)
  exportUptimeDayToStoreIfNotExist(exportDate)
  dbClient.Delete(&exportDae)

}
}

This is just some leetcode to I am sure it needs to be adapted to work properly. But this way we have a few advantages:

  1. We always run the routine even if the uptime tracker gets restarted frequently, because we dont rely on the date variable that stored in memory
  2. We always back up data even if its older than the store-cutoff-date and dont delete any data without backing up

@mrpalide mrpalide changed the base branch from master to develop May 10, 2023 13:49
@mrpalide mrpalide changed the title [WIP] UT new logic UT new logic May 10, 2023
Copy link
Member

@jdknives jdknives left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Will do a round of integration testing and then merge if it works. 👍

@jdknives jdknives merged commit b2b3fca into develop May 11, 2023
1 check passed
@jdknives jdknives deleted the fix/remve-old-logic-add-new-endpoint-files branch May 12, 2023 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants