Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Submodule protobuf #41

Merged
merged 26 commits into from Sep 20, 2022
Merged

Submodule protobuf #41

merged 26 commits into from Sep 20, 2022

Conversation

dmarx
Copy link
Contributor

@dmarx dmarx commented Sep 14, 2022

No description provided.

@dmarx dmarx marked this pull request as draft September 14, 2022 00:19
@dmarx dmarx marked this pull request as ready for review September 14, 2022 00:52
@dmarx
Copy link
Contributor Author

dmarx commented Sep 14, 2022

commit 38e5568 was triggered automatically by the workflow added in this PR. The workflow is triggered on push. It first updates submodules, copies new protofbuf artifacts into the sdk, commits, and pushes back to the branch the triggered the workflow.

main is a protected branch so it will not accept pushes from this workflow, but I haven't yet added anything to block the workflow from being triggered on push to main. I think a consequence of this is that PR merges into main will trigger this workflow and the workflow will then fail. I don't believe this will impact the merge to main at this time, but it might become an issue in the future and will need to be addressed (if not addressed by this PR).

I'm also not sure triggering this update on push is the best CI step for this event and am open to discussion if anyone has an opinion.

git add ./src/**/*.proto
git commit -m "Updated protobuf files"
- name: Push change
uses: ad-m/github-push-action@master

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also lean towards having actions create PRs instead of just pushing changes (unless they are pushing to a branch which is automated only). So using an action like https://github.com/marketplace/actions/create-pull-request.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless of course the plan is to not manually change the files under these paths, then I think the push could be ok as a result of a manual commit to update the submodule.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah that was the idea. main is a protected branch so this can only be triggered on other branches. So it works, but also just typing that out makes me feel worse about it. not a huge fan of part of our CI depending on future us knowing that the main branch needs to stay protected. having it create a PR instead of pushing is probably a good idea. something something human intervention. go ahead and tag this PR as "changes requested"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the goal is to ensure that any changes that occur upstream in api-interfaces will eventually make its way into this repo, then having this workflow run periodically and having it create an MR when the interfaces change seems like the ideal way to go.

update_interfaces.sh Outdated Show resolved Hide resolved
Copy link

@arsenetar arsenetar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we should either have the submodule update be the manual part (aka a user commits that change and then this action takes care of the rest) or have this automate everything and create a PR. I lean towards leaving the submodule updates as manual.

@dmarx
Copy link
Contributor Author

dmarx commented Sep 14, 2022

I lean towards leaving the submodule updates as manual.

could you elaborate on why you favor this option?

Copy link
Contributor

@chigozienri chigozienri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For whatever reason, in our other branches that have api-interfaces as a submodule, we've called the directory interfaces. Not saying you should do that here, but FYI.

cp $SRC/$KIND/${KIND}_pb2_grpc.py $TGT/$KIND/${KIND}_pb2_grpc.py
cp $SRC/$KIND/${KIND}_pb2.py $TGT/$KIND/${KIND}_pb2.py
#############################
# for backwards compatibility
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how I feel about copying these files out of the submodule. I know people already rely on this directory structure, but I wonder if it would be better for us to have a clean, versioned break. Open to arguments either way, though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to agree. Since people can now pull these generated files out of the api-interfaces, I think it makes sense to only have this project be concerned about the generated files meant for python consumption.

Like Chia said, we'll want to cut a major version so we don't break existing consumers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's been less than a month since the beta release so the timing for a major version is good too. especially considering the third party ts client is currently broken because of this dependency already. maybe we do this PR leaving the backwards compatibility stuff in it as a temporary fix to get people who are using the third party ts client back on their feet, and then follow that up with a new major version. this would give a little bit of breathing room to users depending on the current structure to upgrade without requiring that their stuff be broken and unavilable while they're in transition

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alternatively, choosing not to support backwards compatibility is a good forcing function for upgrading

@arsenetar
Copy link

I lean towards leaving the submodule updates as manual.

could you elaborate on why you favor this option?

Mainly since the api-interfaces may have changes that we do not want to bring in yet, either because a) they are not implemented yet and could cause confusion here or b) have some level of breaking changes that we want to be intentional on when to update.

name: Update protobuf stubs

on:
push:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably only want this running on main (once it is known to work), and possibly with a path filter to just when api-interfaces is changed.

@sonarcloud
Copy link

sonarcloud bot commented Sep 20, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 78 Code Smells

No Coverage information No Coverage information
2.0% 2.0% Duplication

Copy link

@arsenetar arsenetar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good starting point for an action.

@dmarx dmarx merged commit 10c6865 into main Sep 20, 2022
@dmarx dmarx deleted the submodule_protobuf branch September 20, 2022 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants