Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSDS with PR info not SonarQube #614

Merged
merged 17 commits into from
Jul 13, 2021
Merged

Conversation

hrcornejo
Copy link
Contributor

Generate a CSV file with bitbucket info (commits and PR over default branch per every repo) to use later in the generation of SSDS document.

} else {
nextPageStart = commits.nextPageStart
}
processCommits(token, repo, commits, file)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this right, this make one API request per page until all commits have been found, and for each commit, make another API request to find any pull requests.

This sounds like a very expensive operation - which will get more expensive over time. Did you calculate the impact on the Bitbucket Server?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is the process. And no, right now we don't have calculated the impact, we are making test to probe it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hector, found out that we reach a limit at ~8k commits. The reason is not yet clear, since there is no error being shown. @hrcornejo @jorge-romero we should consider only including those commits that belong to the current product version. This requires keeping track of the commit SHA of the previous releases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would other endpoints help us instead? E.g. going through the merged pull requests instead, and then plucking the ones we want to use?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelsauter seems like the endpoint offering the merged pull requests does not show the commit SHA.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. I think there is another endpoint that gives you "PR actions" or something like this which does show the merged commit. I would have to look it up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hrcornejo @michaelsauter we should definitely investigate this.

@metmajer
Copy link
Member

metmajer commented Apr 1, 2021

@hrcornejo the current implementation using a CSV file as a result is only a prototype. The final solution removes the CSV file again and puts the data into a table within the SSDS document's section "2.3 Code Reviews". Please put this PR into Draft mode, adjust the title to reflect the end goal, and continue to use this branch for delivering the final solution. Thanks!

@metmajer metmajer marked this pull request as draft April 1, 2021 13:45
@hrcornejo hrcornejo changed the title Feature/generate csv bitbucket SSDS with PR info not, SonarQube Apr 16, 2021
@hrcornejo hrcornejo changed the title SSDS with PR info not, SonarQube SSDS with PR info not SonarQube Apr 16, 2021
*/
@SuppressWarnings(['JavaIoPackageAccess'])
String generateSourceCodeReviewFile() {
def file = new File("${steps.env.WORKSPACE}/${CSV_FILE}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically use some namespace directories to differentiate different types of files. Please avoid writing this file directly into env.WORKSPACE. See MROPipelineUtil class members for examples.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created in new "review" folder

steps.env.WORKSPACE = tempFolder.getRoot().absolutePath
logger = new LoggerStub(log)
project = buildProject(logger)
bitbucketServiceMock = new BitbucketServiceMock().setUp("csv").startServer(RECORD_WIREMOCK, BB_URL_TO_RECORD)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the WireMock server utils in SpecHelper.groovy?

@metmajer metmajer added the enhancement New feature or request label Apr 30, 2021
@metmajer metmajer added this to To Do in OpenDevStack 4.0.0 via automation Apr 30, 2021
@metmajer
Copy link
Member

@hrcornejo @jorge-romero shall we "updraft" and proceed with the review?

@hrcornejo
Copy link
Contributor Author

Updated the templates in opendevstack/ods-document-generation-templates#36

@hrcornejo hrcornejo marked this pull request as ready for review July 8, 2021 11:42
@michaelsauter michaelsauter moved this from To Do to In Progress in OpenDevStack 4.0.0 Jul 12, 2021
@metmajer
Copy link
Member

@hrcornejo is this ready to go? If yes, please resolve conflicts and let's get this merged.

@hrcornejo
Copy link
Contributor Author

@metmajer solved, pending approval to merge

@hrcornejo hrcornejo requested a review from metmajer July 13, 2021 08:18
Copy link
Member

@metmajer metmajer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! I approve, but we need to ensure we integrate with Jira in the most efficient way. @michaelsauter has proposed an alternative REST API to be used. Let's follow up on this in the next iteration.

@metmajer
Copy link
Member

@hrcornejo please squash and merge into a single commit. Cheers!

@hrcornejo hrcornejo merged commit 72c2d02 into master Jul 13, 2021
OpenDevStack 4.0.0 automation moved this from In Progress to Done Jul 13, 2021
@hrcornejo hrcornejo deleted the feature/generate_csv_bitbucket branch July 13, 2021 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants