Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

patch: Add Continued MySQL Support #18921

Merged
merged 20 commits into from Aug 22, 2023
Merged

patch: Add Continued MySQL Support #18921

merged 20 commits into from Aug 22, 2023

Conversation

PeteLevineA
Copy link
Contributor

@PeteLevineA PeteLevineA commented Aug 3, 2023 •
edited

Linked Issue: #18668

Hey Team!
We added the following changes to the following packages/plugins to continue forward progress with working with the mysql database.

  • packages/backend-common
    • Expanded with MySQL Tests
  • packages/backend-tasks
    • Updated column types to conform to MySQL
    • Updated database tests with MySQL
    • Updated datetime column to work with MySQL
  • packages/create-app
    • Updated with a production MySQL App Config template
  • packages/e2e-test
    • Updated to allow for e2e testing with MySQL
  • plugins/app-backend
    • Updated some text columns to string
    • Updated interval to work with MySQL
  • plugins/bazaar-backend
    • Updated to run db tests against MySQL
  • plugins/catalog-backend-module-incremental-ingestion
    • Updated to run db tests against MySQL
  • plugins/catalog-backend
    • Updated text columns to longtext to work with MySQL like issue suggested
  • plugins/code-coverage-backend
    • Updated text column to string
  • plugins/linguist-backend
    • Updated text column to string
  • plugins/tech-insights-backend
    • Updated text columns to string
    • Updated db tests to include MySQL

We also added a new workflow adjacent to the Linux E2E github action that runs specifically against MySQL:8 docker image.

Hey, I just made a Pull Request!

✔️ Checklist

  • A changeset describing the change and affected packages. (more info)
  • Added or updated documentation
  • Tests for new functionality and regression tests for bug fixes
  • Screenshots attached (for UI changes)
  • All your commits have a Signed-off-by line in the message. (more info)

@PeteLevineA PeteLevineA requested a review from freben August 3, 2023 04:11
@github-actions github-actions bot added documentation Improvements or additions to documentation area:catalog Related to the Catalog Project Area labels Aug 3, 2023
@backstage-goalie
Copy link
Contributor

backstage-goalie bot commented Aug 3, 2023 •
edited

Changed Packages

Package Name Package Path Changeset Bump Current Version
@backstage/backend-common packages/backend-common patch v0.19.2
@backstage/backend-tasks packages/backend-tasks patch v0.5.5
example-backend packages/backend none v0.2.86
e2e-test packages/e2e-test none v0.2.6
@backstage/plugin-app-backend plugins/app-backend patch v0.3.48
@backstage/plugin-bazaar-backend plugins/bazaar-backend patch v0.2.11
@backstage/plugin-catalog-backend plugins/catalog-backend patch v1.12.0
@backstage/plugin-code-coverage-backend plugins/code-coverage-backend patch v0.2.14
@backstage/plugin-linguist-backend plugins/linguist-backend patch v0.4.0
@backstage/plugin-scaffolder-backend plugins/scaffolder-backend patch v1.16.0
@backstage/plugin-tech-insights-backend plugins/tech-insights-backend patch v0.5.14

@backstage-goalie
Copy link
Contributor

Thanks for the contribution!
All commits need to be DCO signed before they are reviewed. Please refer to the the DCO section in CONTRIBUTING.md or the DCO status for more info.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 3, 2023 •
edited

Uffizzi Ephemeral Environment Deploying

☁️ https://app.uffizzi.com/github.com/backstage/backstage/pull/18921

⚙️ Updating now by workflow run 5928206782.

What is Uffizzi? Learn more!

awanlin
awanlin requested changes Aug 3, 2023
View reviewed changes
Copy link
Collaborator

@awanlin awanlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @PeteLevineA, this work is pretty challenging so thanks for taking it on. I'm the owner of the Linguist plugin so my focus really is just on that part but I do think my comments probably apply more broadly as well.

Beyond my comments on the changes would it be possible to format the PR description into a list? Right now it all runs into each other making it hard to read 👍

plugins/linguist-backend/migrations/20221115_init.js Show resolved Hide resolved
.changeset/yellow-trains-protect.md Outdated Show resolved Hide resolved
@PeteLevineA PeteLevineA force-pushed the continued-mysql-support branch 2 times, most recently from 84967f9 to 3bf9d2b Compare August 7, 2023 16:34
awanlin
awanlin approved these changes Aug 9, 2023
View reviewed changes
Copy link
Collaborator

@awanlin awanlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me for the Linguist changes, thanks again @PeteLevineA 🚀

@PeteLevineA
Copy link
Contributor Author

@alde @freben I think we're in a good spot to get another review / merge in if possible. Any other suggestions?

freben
freben reviewed Aug 14, 2023
View reviewed changes
Copy link
Member

@freben freben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a quick review before EOD

plugins/app-backend/migrations/20211229105307_init.js Outdated Show resolved Hide resolved
plugins/app-backend/src/lib/assets/StaticAssetsStore.ts Outdated Show resolved Hide resolved
.github/workflows/test-db.yml Outdated Show resolved Hide resolved
packages/create-app/templates/default-app/app-config.production.mysql.yaml Outdated Show resolved Hide resolved
@PeteLevineA PeteLevineA force-pushed the continued-mysql-support branch 2 times, most recently from 3185838 to 7e14aef Compare August 15, 2023 14:33
@github-actions github-actions bot added the area:scaffolder Everything and all things related to the scaffolder project area label Aug 17, 2023
Gerg and others added 9 commits August 17, 2023 12:11
@Gerg @xandroc
@gururajsh @a-b @PeteLevineA
patch: Add Continued MySQL Support
cfc3ca6 
Expanded packages/backend-common with MySQL Tests
Updated packages/backend-tasks column types to conform to MySQL
Updated packages/backend-tasks database tests with MySQL
Updated packages/backend-tasks datetime column to work with MySQL
Updated packages/create-app with a production MySQL App Config template
Updated packages/e2e-test to allow for e2e testing with MySQL
Updated plugins/app-backend some text columns to string
Updated plugins/app-backend interval to work with MySQL
Updated plugins/bazaar-backend to run db tests against MySQL
Updated plugins/catalog-backend-module-incremental-ingestion to run
db tests against MySQL
Updated plugins/catalog-backend text columns to longtext to work
with MySQL like issue suggested
Updated plugins/code-coverage-backend text column to string
Updated plugins/linguist-backend text column to string
Updated plugins/tech-insights-backend text columns to string
Updated plugins/tech-insights-backend db tests to include MySQL

Added New E2E tests to run on pull requests to test against MySQL

Co-authored-by: Alex Rocha <alexr1@vmware.com>
Co-authored-by: David Alvarado <dalvarado@vmware.com>
Co-authored-by: Shwetha Gururaj <gururajsh@vmware.com>
Co-authored-by: Al <aberezovsky@vmware.com>
Co-authored-by: Gerg <gcobb@vmware.com>
Signed-off-by: Pete Levine A <lpete@vmware.com>
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
Updated changeset contribution for create-app
9572733 
Signed-off-by: Pete Levine A <lpete@vmware.com>
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
Remove packages/plugins not referenced in PR
f651154 
Signed-off-by: Pete Levine A <lpete@vmware.com>
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@xandroc @PeteLevineA
Simplifying changeset message to minimal relevant changes across upda…
e43a818 
…ted plugins (#1)

Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
remove duplicate packages from yarn lock
7143e55 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
fix linting and prettier issues
02cedae 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
fix: update migrations used in tests causing error
6e37918 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
chore: update changeset to reflect new fixes
8f61efa 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
remove unnecessary testing, fix columns, feedback
a2960b0 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>

fix issues with merge

Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
fix argument for e2e-test run
a0e6241 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
no longer patching create-app
274a321 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
64KB length for mysql support, dbms option for e2e
ba7bdec 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
Path length to maximum pk, rm e2e mysql for now
6ec403f 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
Swap pk for app-backend path, fix heartbeat task
9ae110c 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
fix pk for path for the stat assets, fix run e2e
2a6250b 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
replace pk with unique due to size constraints
ca061f5 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
remove pk for mysql db
bfa604d 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
Copy link
Contributor Author

@freben can you please give a re-review when you get a chance so we can make it in 1.18-next? Thanks! Also the failing e2e appears to be due to latest rebase of main branch. I can rebase again once that's fixed to get green across the board

freben
freben reviewed Aug 18, 2023
View reviewed changes
packages/e2e-test/src/commands/run.ts Outdated Show resolved Hide resolved
packages/e2e-test/src/commands/run.ts Outdated Show resolved Hide resolved
packages/e2e-test/src/commands/run.ts Outdated Show resolved Hide resolved
packages/e2e-test/src/commands/index.ts Outdated Show resolved Hide resolved
@PeteLevineA
remove unnecessary dbms string, fix ternary
90c19f8 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@PeteLevineA
update api report with updated e2e test run cmd
3f8558b 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
freben
freben reviewed Aug 21, 2023
View reviewed changes
Copy link
Member

@freben freben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright so very close! Let's try to get these two last things set and then you'll be good to go as far as I can see

plugins/app-backend/migrations/20211229105307_init.js Show resolved Hide resolved
packages/e2e-test/src/commands/run.ts Outdated Show resolved Hide resolved
@PeteLevineA
set unique index for mysql, set connection var
e57d718 
Signed-off-by: lpete@vmware.com <lpete@vmware.com>
@freben freben merged commit 20def97 into backstage:master Aug 22, 2023
36 checks passed
@github-actions
Copy link
Contributor

Thank you for contributing to Backstage! The changes in this pull request will be part of the 1.18.0 release, scheduled for Tue, 19 Sep 2023.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xandroc xandroc xandroc left review comments

@freben freben freben approved these changes

@awanlin awanlin awanlin approved these changes

@backstage-service backstage-service Awaiting requested review from backstage-service backstage-service is a code owner

@alde alde Awaiting requested review from alde alde is a code owner

Assignees
No one assigned
Labels
area:catalog Related to the Catalog Project Area area:scaffolder Everything and all things related to the scaffolder project area documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@PeteLevineA @freben @xandroc @awanlin @Gerg