Skip to content

[Bug]: Exception handling on MongoDB output doesnot works & hop server crashes #3626

@swwapnily4io

Description

@swwapnily4io

Apache Hop version?

2.7.0

Java version?

17

Operating system

Windows

What happened?

Adding exception handling on the mongodb output results in server crash.
I tried with 10 inputs, batch size as 1, standard DB configurations and i purposefully connected to read only DB to reproduce the scenario.
image
image

The hop GUI / Server Hangs while repeating the below mentioned logs,
2024/02/14 14:52:22 - ID_NumberToString 2.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:22 - CreatePartyCustomerBillingAccount - Pipeline has allocated 17 threads and 14 rowsets.
2024/02/14 14:52:22 - RelatedParty.0 - Starting to run...
2024/02/14 14:52:22 - ID_NumberToString 2.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - RatingType.0 - Starting to run...
2024/02/14 14:52:22 - MongoDB_customer.0 - Starting to run...
2024/02/14 14:52:22 - ReferredType.0 - Starting to run...
2024/02/14 14:52:22 - Account State.0 - Starting to run...
2024/02/14 14:52:22 - MongoDB_billingAccounts.0 - Starting to run...
2024/02/14 14:52:22 - FailedIndividualRecords.0 - Starting to run...
2024/02/14 14:52:22 - MongoDB_customer.0 - Signaling 'output done' to 0 output rowsets.
2024/02/14 14:52:22 - FailedAccountRecords.0 - Starting to run...
2024/02/14 14:52:22 - AddressLines.0 - Starting to run...
2024/02/14 14:52:22 - MongoDB_customer.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - MongoDB_individual.0 - Starting to run...
2024/02/14 14:52:22 - IsResponsible.0 - Starting to run...
2024/02/14 14:52:22 - FailedCustomerRecords.0 - Starting to run...
2024/02/14 14:52:22 - RatingType.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:22 - RatingType.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - Account State.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:22 - Account State.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - FailedCustomerRecords.0 - Signaling 'output done' to 0 output rowsets.
2024/02/14 14:52:22 - FailedCustomerRecords.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - IsResponsible.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:22 - IsResponsible.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - MongoDB_billingAccounts.0 - Signaling 'output done' to 0 output rowsets.
2024/02/14 14:52:22 - MongoDB_billingAccounts.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:22 - FailedAccountRecords.0 - Signaling 'output done' to 0 output rowsets.
2024/02/14 14:52:22 - FailedAccountRecords.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
2024/02/14 14:52:23 - MongoDB_individual.0 - The following incoming fields will not be inserted/upserted: 'groupType', '@referredType', 'accountName', 'dealerCode', '@type', 'externalID', 'referredTypeIndividual', 'language', 'href_holder', 'billCycleID', 'billingAddress1', 'R_P_type', 'billingType', 'responsible', 'state', 'accountRole', 'CustomerHref_holder', 'billingAccountHref_holder', 'creditCategory', 'taxAuthority', 'accountType', 'addressLineOne', 'href_customer', 'billingAccountType', 'CustomerType', 'referredTypeCustomer', 'discountClass', 'addressLineTwo', 'isRef', 'CustomerStatus', 'name', 'id_num', 'href_billingAccount', 'addressLineThree',
2024/02/14 14:52:23 - MongoDB_individual.0 - Committing a batch...
2024/02/14 14:52:23 - MongoDB_individual.0 - ERROR: An error occurred during write: com.mongodb.MongoSocketOpenException: Exception opening socket
2024/02/14 14:52:23 - MongoDB_individual.0 - Retrying write to mongodb in 10 seconds...
2024/02/14 14:52:23 - AccountTable.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - AccountTable.0 - Finished reading query, closing connection.
2024/02/14 14:52:23 - GenerateID.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - GenerateID.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - NFT_BSS - Commit on database connection [NFT_BSS]
2024/02/14 14:52:23 - ID_NumberToString.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - ID_NumberToString.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - manipulate_href_field.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - manipulate_href_field.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - RelatedParty.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - RelatedParty.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - ReferredType.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - ReferredType.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - AddressLines.0 - Signaling 'output done' to 1 output rowsets.
2024/02/14 14:52:23 - AddressLines.0 - Finished processing (I=0, O=0, R=10, W=10, U=0, E=0)
2024/02/14 14:52:23 - NFT_BSS - Connection to database closed!
2024/02/14 14:52:23 - AccountTable.0 - Finished processing (I=10, O=0, R=0, W=10, U=0, E=0)
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 1
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - FailedIndividualRecords.0 - Parent folder exists [file:///C:/Migration]!
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - FailedIndividualRecords.0 - Opening output stream using provider: None
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - FailedIndividualRecords.0 - Opening output stream in encoding: windows-1252
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - FailedIndividualRecords.0 - Opened new file with name [file:///C:/Migration/FailedIndividualRecords_20240214_145233.txt]
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch
2024/02/14 14:52:33 - MongoDB_individual.0 - Saving individual documents in current batch...
2024/02/14 14:52:33 - MongoDB_individual.0 - Current batch size: 0
2024/02/14 14:52:33 - MongoDB_individual.0 - Successfully saved 0 documents from the current batch

Issue Priority

Priority: 3

Issue Component

Component: Database, Component: Other

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions