-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(couchdb-java-client): Migrating from ektorp to cloudant #1149
Conversation
7b03835
to
ff7dbd4
Compare
ff7dbd4
to
8be1574
Compare
test run cleanly on vagrant (Java 11 based, Ubuntu 20, Couchdb 311) |
8be1574
to
7f935fe
Compare
7f935fe
to
5b43433
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed the PR.
Kindly have a look at the review comments.
List<DocumentOperationResult> results = attachmentUsageRepository.executeBulk(sanitizedUsages); | ||
if (!results.isEmpty()) { | ||
List<Response> results = attachmentUsageRepository.executeBulk(sanitizedUsages); | ||
if (CommonUtils.isNotEmpty(sanitizedUsages) && results.isEmpty()) { | ||
throw new SW360Exception("Some of the usage documents could not be created: " + results); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
results
seems to be empty , maybe it would be good to change exception message as well since previously exception was thrown when not empty
same for line 201
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have validated the response/results before exception
@@ -185,20 +184,20 @@ public AttachmentUsage updateAttachmentUsage(AttachmentUsage attachmentUsage) { | |||
} | |||
|
|||
public void updateAttachmentUsages(List<AttachmentUsage> attachmentUsages) throws TException { | |||
List<DocumentOperationResult> results = attachmentUsageRepository.executeBulk(attachmentUsages); | |||
List<Response> results = attachmentUsageRepository.executeBulk(attachmentUsages); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the documentation, it specifies, that for each record/request there is a response and that response should be validated with statusCode and getError() to ensure no failed records before the next check in line 188-190.
Based on java docs from com.cloudant.client.api.database.class
* <p>
* Note that the value returned by {@code getStatusCode()} on each {@code Response} object is
* the overall status returned from {@code bulk_docs} and will therefore be the same for all
* {@code Response} objects.
* </p>
* <p>
* The returned list of {@code Response}s should be examined to ensure that all of the documents submitted
* in the original request were successfully added to the database.
* </p>
* <p>
* When a document (or document revision) is not correctly committed to the database because of
* an error, you must check the value of {@code getError()} to determine error type and course
* of action. See
* <a href="https://console.bluemix.net/docs/services/Cloudant/api/document.html#bulk-document-validation-and-conflict-errors" target="_blank">
* Bulk Document Validation and Conflict Errors</a> for more information.
* </p>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have added the validation
} | ||
|
||
@View(name = "projectVulnerabilityRating", map = PROJECT_VULNERABILITY_LINK) | ||
public List<ProjectVulnerabilityRating> getProjectVulnerabilityRating(String projectId) { | ||
List<ProjectVulnerabilityRating> queryResults = queryByPrefix("projectVulnerabilityRating", projectId); | ||
|
||
if (queryResults.size() > 1) { | ||
log.error("More than one projectVulnerabilityRating found for project with id " + projectId); | ||
// log.error("More than one projectVulnerabilityRating found for project with id " + projectId); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove/uncomment commented line of code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have added the logger
@@ -478,7 +482,7 @@ public RequestStatus updateLicenseFromAdditionsAndDeletions(License licenseAddit | |||
if (!PermissionUtils.isUserAtLeast(UserGroup.CLEARING_ADMIN, user)){ | |||
return null; | |||
} | |||
final List<DocumentOperationResult> documentOperationResults = licenseTypeRepository.executeBulk(licenseTypes); | |||
final List<Response> documentOperationResults = licenseTypeRepository.executeBulk(licenseTypes); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kindly check whether returned Response
needs to be validated for any error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have added the validation
backend/src/src-licenses/src/test/java/org/eclipse/sw360/licenses/LicenseHandlerTest.java
Outdated
Show resolved
Hide resolved
...datahandler/src/main/java/org/eclipse/sw360/datahandler/common/CustomThriftDeserializer.java
Outdated
Show resolved
Hide resolved
...ies/lib-datahandler/src/main/java/org/eclipse/sw360/datahandler/common/DatabaseSettings.java
Outdated
Show resolved
Hide resolved
...lib-datahandler/src/main/java/org/eclipse/sw360/datahandler/couchdb/AttachmentConnector.java
Show resolved
Hide resolved
...lib-datahandler/src/test/java/org/eclipse/sw360/datahandler/common/DatabaseSettingsTest.java
Outdated
Show resolved
Hide resolved
...ndler/src/test/java/org/eclipse/sw360/datahandler/couchdb/AttachmentStreamConnectorTest.java
Outdated
Show resolved
Hide resolved
36f301e
to
b4d3d90
Compare
@JaideepPalit , Thanks for the review. Have incorporated most of them. Kindly have a look and let me know. @ravi110336 Thanks for the test. Have fixed the mentioned issues however the observation regarding "Request" tab, I am not able to reproduce it. |
cf7b624
to
466a359
Compare
@ravi110336 , From discussion, have rectified the issue with moderation request. Kindly have a look |
Tested the moderation request and attachment download working fine. |
tested also with vagrant: looks good |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re-reviewed the PR.
Kindly have a look at some very minor comments.
Note-
The travis-ci logs gives lot of error messages during test, which are not coming in other branches.
https://travis-ci.org/github/eclipse/sw360/jobs/768972323#L2046
Apr 30, 2021 6:56:28 AM com.cloudant.client.org.lightcouch.CouchDbClient createDB
INFO: Created Database: 'sw360_test_db'
2021-04-30 06:56:28 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/AttachmentUsage : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/AttachmentUsage. Error: not_found. Reason: missing.
2021-04-30 06:56:28 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Attachment : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Attachment. Error: not_found. Reason: missing.
2021-04-30 06:56:28 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Source : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Source. Error: not_found. Reason: missing.
2021-04-30 06:56:28 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Vendor : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Vendor. Error: not_found. Reason: missing.
2021-04-30 06:56:28 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Release : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Release. Error: not_found. Reason: missing.
2021-04-30 06:56:29 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Component : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Component. Error: not_found. Reason: missing.
2021-04-30 06:56:29 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/Project : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/Project. Error: not_found. Reason: missing.
2021-04-30 06:56:29 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/User : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/User. Error: not_found. Reason: missing.
2021-04-30 06:56:29 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id _design/ProjectVulnerabilityRating : 404 Object Not Found at http://localhost:5984/sw360_test_db/_design/ProjectVulnerabilityRating. Error: not_found. Reason: missing.
2021-04-30 06:56:29 ERROR DatabaseConnectorCloudant:138 - Error fetching document of type DesignDocument with id
Seems like , using cloudant - design docs are not getting generated.
It should also be tested on a clean system with no DBs or Design docs
...er/src/main/java/org/eclipse/sw360/datahandler/cloudantclient/DatabaseConnectorCloudant.java
Outdated
Show resolved
Hide resolved
...main/java/org/eclipse/sw360/datahandler/cloudantclient/DatabaseRepositoryCloudantClient.java
Outdated
Show resolved
Hide resolved
...src-cvesearch/src/main/java/org/eclipse/sw360/cvesearch/datasink/VulnerabilityConnector.java
Outdated
Show resolved
Hide resolved
backend/src/src-components/src/test/java/org/eclipse/sw360/components/ComponentHandlerTest.java
Outdated
Show resolved
Hide resolved
backend/src/src-changelogs/src/main/java/org/eclipse/sw360/changelogs/ChangeLogsHandler.java
Outdated
Show resolved
Hide resolved
I have tested the branch after deleting all the DBs and design docs in my local system, all the DBs are getting created with the design docs successfully. |
Signed-off-by: Smruti Prakash Sahoo <smruti.sahoo@siemens.com>
466a359
to
ce72095
Compare
@JaideepPalit Thanks for review. Have incorporated changes.
(from discussion): This is due to added logger message, it's ok to keep it.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...360-portlet/src/main/java/org/eclipse/sw360/portal/portlets/components/ComponentPortlet.java
Outdated
Show resolved
Hide resolved
case 3: | ||
query = getConnector().createQuery(Component.class, "bycomponenttype"); | ||
break; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix indentation for switch - case
extra newline could also be removed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed new line. Tried with CTRL+SHIFT+F
, didn't help with switch case indentation.
backend/src-common/src/main/java/org/eclipse/sw360/datahandler/db/ComponentRepository.java
Outdated
Show resolved
Hide resolved
ce72095
to
0f5adba
Compare
Signed-off-by: Smruti Prakash Sahoo <smruti.sahoo@siemens.com>
0f5adba
to
0e22d55
Compare
(from discussion) Raised an issue #1199 Fixed it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code looks fine to me.
has been tested on our staging server, vagrant of course and again staging server.... |
Have added a new library(cloudant java client) instead of ektorp for DB operations.
Issue: Closes #1148
Suggest Reviewer
How To Test?
Need to
Checklist
Must:
Signed-off-by: Smruti Prakash Sahoo smruti.sahoo@siemens.com