Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate to neuro-flow #25

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Migrate to neuro-flow #25

wants to merge 19 commits into from

Conversation

YevheniiSemendiak
Copy link
Contributor

Major changes:

  • replaced Makefile with neuro-flow project description
  • tests were removed since the project structure was changed a lot. Question: should I update them and include some sort of NF-based tests? cc @mariyadavydova
  • data is controlled by DVC and stored per user on a platform
  • added inference via Seldon, annotations via LabelStudio, experiment tracking via MLFlow, etc.

YevheniiSemendiak and others added 19 commits December 24, 2020 11:24
@YevheniiSemendiak
add deployment to seldon
c523f9a
@YevheniiSemendiak
Start migration to neuro-flow.
eb9281d
@YevheniiSemendiak
Share project root
1c1fcf1
@YevheniiSemendiak
Log metrics on each epoch
415a53a
Add data_utils/extend_dataset.py
4989b3d
@YevheniiSemendiak
Squashed commit of the following:
ad001fe 
commit 22ad603
Author: yevheniisemendiak <job-c54b5e16-3d6a-4e59-9f54-d879f37af72f@neuro-compute>
Date:   Mon Jan 25 15:34:16 2021 +0000

    add 1 img to dataset.

commit a4164d6
Author: yevheniisemendiak <job-5623c647-6fca-43a9-aad4-c21006f64bb3@neuro-compute>
Date:   Mon Jan 25 15:08:18 2021 +0000

    add 20 img to dataset.
@YevheniiSemendiak
Publish all changes regarding train/extend_data tasks.
1a2db34
@anayden @YevheniiSemendiak
Seldon runs (#16)
0d218c0 
Co-authored-by: Yevhenii Semendiak <semendyak@gmail.com>
@YevheniiSemendiak
Fix label studio (#17)
2d47ba0 
* rename secret

* add label studio config and wrapper

* Configure label-studio job run, cleanup of live.yaml.

* Add extend_dataset_by param into label-studio job.

* Fix label-studio job, extend_dataset util.

* annotate 1 images in job-1668e962-97fa-4755-8094-3d959330caeb

* Fix label-studio job (2)

* annotate 0 images in job-37430f48-78b6-4b10-ad41-419d9854f950

* Revert "annotate 0 images in job-37430f48-78b6-4b10-ad41-419d9854f950"

This reverts commit 27da2bd.

* Get confirmation for commiting into DVC and GIT from label-studio

* annotate 2 images in job-41e257bd-c2b0-45c5-9ddd-1120d164d36c

* One more trial to fix things.

* annotate 1 images in job-d231cb64-b8d2-4d35-bbb2-99bed64e2673

* annotate 2 images in job-bdc3ef77-1f06-43d4-b442-4b399dc59a5f

* Final minor changes.

Co-authored-by: yevheniisemendiak <yevheniisemendiak@neuro-compute>
@anayden
Add locust (#15)
2d3ad3a
Fix training scripts and add git flow (#18)
8aa3a8d
Fix MLflow uri for train
da8fd6a
Locust: defaults + print on exception
e1d2ff9
@YevheniiSemendiak
Fixed to run end-to-end (#19)
a19e126 
* minor fixes

* fix merging datasets

* fix merging datasets (2)

* use shared seldon docker image
@YevheniiSemendiak
Cleanup (#22)
85bbb4b 
* cleanup

* proceed with the cleanup, update readme, remove CI

* Enhance the readme and rm redundant test file.

* Use named disk for postgres
@YevheniiSemendiak
"bump deployment dependencies, update deployment job" (#24)
812aa31
Auto-commit: update dataset: add 10 images
d37b380
@YevheniiSemendiak
Revert "Auto-commit: update dataset: add 10 images"
cbf302c 
This reverts commit d37b380.
@YevheniiSemendiak @anayden
Use buckets in DVC (#26)
21912f4 
* extend_data

* migrate dvc to s3

* fix role name, update requirements, cleanup

* Fix project dir env variable

Co-authored-by: Alexey Naiden <alexey.nayden@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@YevheniiSemendiak @anayden