Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Persistence agent for ODAHU CRD #268

Closed
vlad-tokarev opened this issue Jun 18, 2020 · 0 comments
Closed

Persistence agent for ODAHU CRD #268

vlad-tokarev opened this issue Jun 18, 2020 · 0 comments
Assignees
@vlad-tokarev
Labels
1.3
Projects
odahu-kanban

Comments

@vlad-tokarev
Copy link
Member

After implementation of base persistence layer
#175

Now, we need create service named "Odahu Persistence Agent". Is separate process running using k8s tools (go-client, etc) and monitor odahu k8s CRD and persist them into the main storage
@vlad-tokarev vlad-tokarev self-assigned this Jun 18, 2020
@vlad-tokarev vlad-tokarev added this to To do in odahu-kanban Jul 8, 2020
@vlad-tokarev vlad-tokarev moved this from To do to In progress in odahu-kanban Jul 8, 2020
@vlad-tokarev vlad-tokarev moved this from In progress to To do in odahu-kanban Jul 14, 2020
@vlad-tokarev vlad-tokarev moved this from To do to In progress in odahu-kanban Jul 17, 2020
vlad-tokarev added a commit that referenced this issue Jul 22, 2020
@vlad-tokarev
Persistence for training (#268)
6fc4fb4
vlad-tokarev added a commit that referenced this issue Jul 26, 2020
@vlad-tokarev
Manager of background runnables(#268)
9a78f4e
vlad-tokarev added a commit that referenced this issue Jul 26, 2020
@vlad-tokarev
Prepare server (#268)
283375e
vlad-tokarev added a commit that referenced this issue Jul 27, 2020
@vlad-tokarev
Made concurrency (#268)
3bdb4bf
vlad-tokarev added a commit that referenced this issue Jul 27, 2020
@vlad-tokarev
Training persistence WIP (#268)
b878de3
vlad-tokarev added a commit that referenced this issue Jul 28, 2020
@vlad-tokarev
Training persistence WIP (#268)
58b8941
vlad-tokarev added a commit that referenced this issue Jul 28, 2020
@vlad-tokarev
Training persistence templated algorithm of reconcilation DB and kube…
ebed17e 
…rnetes (#268)
vlad-tokarev added a commit that referenced this issue Jul 28, 2020
@vlad-tokarev
WIP generics storage sync (#268)
5c55773
vlad-tokarev added a commit that referenced this issue Jul 29, 2020
@vlad-tokarev
generics storage sync (#268)
8cd4d8a
vlad-tokarev added a commit that referenced this issue Jul 29, 2020
@vlad-tokarev
generics storage sync WIP (#268)
538943a
vlad-tokarev added a commit that referenced this issue Jul 29, 2020
@vlad-tokarev
generics storage sync WIP (#268)
da9cb84
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Persistence for training (#268)
5a602da
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Manager of background runnables(#268)
061df7e
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Prepare server (#268)
9259e4d
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Made concurrency (#268)
f5009e0
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Training persistence WIP (#268)
d1cf8ef
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Training persistence WIP (#268)
f4cf3ec
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Training persistence templated algorithm of reconcilation DB and kube…
91872f3 
…rnetes (#268)
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
WIP generics storage sync (#268)
e453bb4
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
generics storage sync (#268)
349cd2d
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
generics storage sync WIP (#268)
ef41daa
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
generics storage sync WIP (#268)
b403215
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Ability to disable workers and a bit of refactoring (#268)
ad8e675
vlad-tokarev added a commit that referenced this issue Jul 30, 2020
@vlad-tokarev
Fix linter (#268)
2dfdf7b
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Feat #268 Unit tests for training_service.go
2fc9c37
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Feat #268 packaging service
5ca8050 
* Implement packaging service layer
* Add context to packaging repository
* Webserver uses packaging_service.go not packaging repository
* Controller uses packaging_service.go not packaging repository
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Feat #268 deployment service
ef8b3a5 
* Implement deployment service layer
* Add context to deployment repository
* Webserver uses deployment_service.go.go not deployment repository
* Controller uses deployment_service.go not deployment repository
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 remove redundant nil argument
30a23f9
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt fields as timestamptz into postgres…
1cf57b0 
… tables
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Upgrade testify package
d85318a
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Add fields CreatedAt, UpdatedAt for ModelTraining. Deprecate…
7551331 
… fields in status
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt login in repo & refactoring
81dc379 
* CreatedAt, UpdatedAt added to trainining repo
* Use squirrel builder for GetModelTraining method instead of manual sql query formatting
* UpdateModelTraining repo method now does not drop .Status.State
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 CreateModelTraining and UpdateModelTraining in training_serv…
ebbd44c 
…ice.go now drop status and update fields CreatedAt, UpdatedAt
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Packaging CreatedAt, UpdatedAt fields are removed from statu…
9edc378 
…s to only app level
vlad-tokarev added a commit that referenced this issue Sep 11, 2020
@vlad-tokarev
Fix #268 Deployment CreatedAt, UpdatedAt fields are removed from stat…
47ead14 
…us to only app level
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt fields as timestamptz into postgres…
25d48de 
… tables
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Upgrade testify package
d016513
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Add fields CreatedAt, UpdatedAt for ModelTraining. Deprecate…
9e6a623 
… fields in status
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt login in repo & refactoring
778dfa0 
* CreatedAt, UpdatedAt added to trainining repo
* Use squirrel builder for GetModelTraining method instead of manual sql query formatting
* UpdateModelTraining repo method now does not drop .Status.State
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 CreateModelTraining and UpdateModelTraining in training_serv…
6fd40ef 
…ice.go now drop status and update fields CreatedAt, UpdatedAt
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Packaging CreatedAt, UpdatedAt fields are removed from statu…
0824711 
…s to only app level
vlad-tokarev added a commit that referenced this issue Sep 12, 2020
@vlad-tokarev
Fix #268 Deployment CreatedAt, UpdatedAt fields are removed from stat…
40092f9 
…us to only app level
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
ODAHU System Controller Unit Tests and refactoring (#268)
ea565bf 
* Add tests
* Rename syncer -> adapter (adapters are used by generic_worker.go)
* Rename runners_manager -> controller
* Now: Controller syncs "DB spec" and" Runtime (kubernetes) Status" using generic workers with different adapters (training adapter, packaging adapter and so on)
* Move controller to different HELM deployment
* Add context.Context into all repositories methods
* Passing custom interface (utils.Querier) to interact with db to all repositories explicitly (for *sql.DB and sql.Tx interchangeability)
* Function to calculate struct hashes encapsulated to avoid direct call of external library
* Add service layer with business logic between WebServer <-> Repository / Controller <-> Repository
* If controller push new status from runtime to storage and reveal that spec in its update is stale then controller forget about this update (update is not actual cuz we have new Spec in storage)
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt fields as timestamptz into postgres…
a2eaa91 
… tables
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Upgrade testify package
1be5e26
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Add fields CreatedAt, UpdatedAt for ModelTraining. Deprecate…
26fa557 
… fields in status
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Add CreatedAt, UpdatedAt login in repo & refactoring
f620c60 
* CreatedAt, UpdatedAt added to trainining repo
* Use squirrel builder for GetModelTraining method instead of manual sql query formatting
* UpdateModelTraining repo method now does not drop .Status.State
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 CreateModelTraining and UpdateModelTraining in training_serv…
183b0ad 
…ice.go now drop status and update fields CreatedAt, UpdatedAt
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Packaging CreatedAt, UpdatedAt fields are removed from statu…
e39fc2a 
…s to only app level
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Deployment CreatedAt, UpdatedAt fields are removed from stat…
331ce17 
…us to only app level
vlad-tokarev added a commit that referenced this issue Sep 14, 2020
@vlad-tokarev
Fix #268 Move CreatedAt, UpdatedAt in status
aa49722
vlad-tokarev added a commit that referenced this issue Sep 15, 2020
@vlad-tokarev
Fix #268 Move CreatedAt, UpdatedAt in status
8747b94
odahu-kanban automation moved this from In progress to Done Sep 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vlad-tokarev vlad-tokarev

Labels
1.3
Projects
odahu-kanban
  
Done
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vlad-tokarev