Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

frontend: data-pipelines build and test running CI/CD #1671

Merged

Conversation

ivakoleva
Copy link
Contributor

@ivakoleva ivakoleva commented Feb 23, 2023

We need to build and run unit tests of data-pipelines library and ui application that uses it. The component is dependent on shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites, the other one to rebuild and relink continuously, needed during development.
The proposed implementation triggers a data-pipelines rebuild (+test), upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script) on a clean setup, and verified the unit tests were successful. Also, did start and browse the application manually, to check its operational.

@ivakoleva ivakoleva marked this pull request as ready for review February 24, 2023 12:08
@antoniivanov
Copy link
Collaborator

I was once told (when our open source team was reviewing our commit history) that if we have bullet points in the commit message, a change is doing too many things and should have been split. Since then when I catch myself that I make bullet points or start saying "fixing this and that" I consider if I should better split it into 2 or more.

I thought that was good advice and I am passing it on.

@ivakoleva
Copy link
Contributor Author

I was once told (when our open source team was reviewing our commit history) that if we have bullet points in the commit message, a change is doing too many things and should have been split. Since then when I catch myself that I make bullet points or start saying "fixing this and that" I consider if I should better split it into 2 or more.

I thought that was good advice and I am passing it on.

I generally agree, it's a good advice. When you are open sourcing an existing component, a lot of stuff are known and also tested & verified already in advance. So, I also think it is normal to be confident with enumerating the changes, it rather depends on the context.

@antoniivanov
Copy link
Collaborator

I generally agree, it's a good advice. When you are open sourcing an existing component, a lot of stuff are known and also tested & verified already in advance. So, I also think it is normal to be confident with enumerating the changes, it rather depends on the context.

Yes I agree, for that specific case, we are fine.

I wanted to note this advice because the change reminded me.
And also want to note (especially for any new contributors) that while now we have some bigger PRs around open-sourcing the UI and that's not a practice we recommend.

@ivakoleva
Copy link
Contributor Author

I generally agree, it's a good advice. When you are open sourcing an existing component, a lot of stuff are known and also tested & verified already in advance. So, I also think it is normal to be confident with enumerating the changes, it rather depends on the context.

Yes I agree, for that specific case, we are fine.

I wanted to note this advice because the change reminded me. And also want to note (especially for any new contributors) that while now we have some bigger PRs around open-sourcing the UI and that's not a practice we recommend.

I actually think 11 files is a pretty standard amount of file changes. The PR description does not contain bullets, it contains logical steps that the install script performs, that would be an atomic change (since the script is an atomic item itself). So I don't think artificially splitting the script, in a series of PRs, would be practical. I considered splitting the import fixes, but decided not to due arguments listed in #1671 (comment)

@ivakoleva ivakoleva force-pushed the person/ikoleva/frontend-data-pipelines-cicd-build-test branch from e5150e8 to 3e2935e Compare March 2, 2023 08:41
@ivakoleva ivakoleva force-pushed the person/ikoleva/frontend-data-pipelines-cicd-build-test branch from 9254375 to e2e7986 Compare March 2, 2023 11:56
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A few functional import fixes were made, to import the correctly renamed
class.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
@ivakoleva ivakoleva force-pushed the person/ikoleva/frontend-data-pipelines-cicd-build-test branch from f87b350 to 616816f Compare March 2, 2023 14:25
ivakoleva and others added 7 commits March 2, 2023 16:41
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
We need to build and run unit tests of data-pipelines library and
ui application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites,
the other one to rebuild and relink continuously, needed during
development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also,
did start and browse the application manually, to check its operational.

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
@ivakoleva ivakoleva enabled auto-merge (squash) March 6, 2023 08:10
@ivakoleva ivakoleva merged commit 8d6e47f into main Mar 6, 2023
@ivakoleva ivakoleva deleted the person/ikoleva/frontend-data-pipelines-cicd-build-test branch March 6, 2023 08:38
yonitoo pushed a commit that referenced this pull request Mar 6, 2023
We need to build and run unit tests of data-pipelines library and ui
application that uses it. The component is dependent on
shared-components library.

Added Bash scripts to
(1) link if any shared-components rebuild was made and is about to be
published, or otherwise default to latest already published one
(2) build the data-pipelines library and link @vdk/data-pipelines 
(3) build the UI application
(4) lint the subprojects to verify format
(5) run the unit tests
One bash script is meant to be used to satisfy build prerequisites, the
other one to rebuild and relink continuously, needed during development.
The proposed implementation triggers a data-pipelines rebuild (+test),
upon any shared-components change, as well as data-pipelines changes.
A README.md link fix to the VEP.

Testing done: run the install script (that evaluates the build script)
on a clean setup, and verified the unit tests were successful. Also, did
start and browse the application manually, to check its operational.

---------

Signed-off-by: ivakoleva <iva.koleva@clearcode.bg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants