Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

It does not create an array of objects when using a ReadStream of CSV #467

Closed
AllanOricil opened this issue Mar 9, 2023 · 1 comment
Closed

It does not create an array of objects when using a ReadStream of CSV #467

AllanOricil opened this issue Mar 9, 2023 · 1 comment

Comments

@AllanOricil
Copy link

AllanOricil commented Mar 9, 2023 •
edited
Loading

I get different outputs when converting the following csv to json

"Id","Name","SBQQ__AssetAmendmentBehavior__c","SBQQ__AssetConversion__c","SBQQ__BatchQuantity__c","SBQQ__BillingFrequency__c","SBQQ__BillingType__c","SBQQ__BlockPricingField__c","SBQQ__ChargeType__c","SBQQ__Component__c","SBQQ__CompoundDiscountRate__c","SBQQ__ConfigurationEvent__c","SBQQ__ConfigurationFieldSet__c","SBQQ__ConfigurationFields__c","SBQQ__ConfigurationFormTitle__c","SBQQ__ConfigurationType__c","SBQQ__ConfigurationValidator__c","SBQQ__ConfiguredCodePattern__c","SBQQ__ConfiguredDescriptionPattern__c","SBQQ__CostEditable__c","SBQQ__CostSchedule__c","SBQQ__CustomConfigurationPage__c","SBQQ__CustomConfigurationRequired__c","SBQQ__CustomerCommunityAvailability__c","SBQQ__DefaultPricingTable__c","SBQQ__DefaultQuantity__c","SBQQ__DescriptionLocked__c","SBQQ__DiscountCategory__c","SBQQ__DiscountSchedule__c","SBQQ__DynamicPricingConstraint__c","SBQQ__ExcludeFromMaintenance__c","SBQQ__ExcludeFromOpportunity__c","SBQQ__ExternallyConfigurable__c","SBQQ__GenerateContractedPrice__c","SBQQ__HasConfigurationAttributes__c","SBQQ__HasConsumptionSchedule__c","SBQQ__Hidden__c","SBQQ__HidePriceInSearchResults__c","SBQQ__IncludeInMaintenance__c","SBQQ__NewQuoteGroup__c","SBQQ__NonDiscountable__c","SBQQ__NonPartnerDiscountable__c","SBQQ__OptionLayout__c","SBQQ__OptionSelectionMethod__c","SBQQ__Optional__c","SBQQ__PriceEditable__c","SBQQ__PricingMethodEditable__c","SBQQ__PricingMethod__c","SBQQ__ProductPictureID__c","SBQQ__QuantityEditable__c","SBQQ__QuantityScale__c","SBQQ__ReconfigurationDisabled__c","SBQQ__RenewalProduct__c","SBQQ__SortOrder__c","SBQQ__Specifications__c","SBQQ__SubscriptionBase__c","SBQQ__SubscriptionCategory__c","SBQQ__SubscriptionPercent__c","SBQQ__SubscriptionPricing__c","SBQQ__SubscriptionTarget__c","SBQQ__SubscriptionTerm__c","SBQQ__SubscriptionType__c","SBQQ__TaxCode__c","SBQQ__Taxable__c","SBQQ__TermDiscountLevel__c","SBQQ__TermDiscountSchedule__c","SBQQ__UpgradeCredit__c","SBQQ__UpgradeRatio__c","SBQQ__UpgradeSource__c","SBQQ__UpgradeTarget__c","SBQQ__EnableLargeConfiguration__c","SBQQ__PricingGuidance__c"
"01tDm000000IW1QIAW","SolarBots","Default","One per quote line","","Monthly","Advance","Quantity","Recurring","false","","","","","","","","","","false","","","false","","","1.0","false","","","","false","false","false","","false","false","false","false","false","false","false","false","","Click","false","false","false","List","","true","","false","","","","List","","","Fixed Price","","1.0","Renewable","","false","","","","","","","false",""
"01tDm000000IW1RIAW","Major Solar Panel","Default","One per quote line","","","","Quantity","One-Time","false","","","","","","","","","","false","","","false","","","1.0","false","","","","false","false","false","","false","false","false","false","false","false","false","false","","Click","false","false","false","List","","true","","false","","","","List","","","","","","Renewable","","false","","","","","","","false",""
"01tDm000000IW1SIAW","Ursa Major Warranty and Maintenance Plan","Default","One per quote line","","Quarterly","Advance","Quantity","Recurring","false","","","","","","","","","","false","","","false","","","1.0","false","","","","false","false","false","","false","false","false","false","false","false","false","false","","Click","false","false","false","List","","true","","false","","","","List","","","Fixed Price","","12.0","Renewable","","false","","","","","","","false",""

when using the cli

csvtojson my-csv.csv > my-json.json

I get the following output, which is exactly what I'm expecting.

[
{"Id":"01tDm000000IW1QIAW","Name":"SolarBots","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"Monthly","SBQQ__BillingType__c":"Advance","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"Recurring","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"Fixed Price","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"1.0","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""},
{"Id":"01tDm000000IW1RIAW","Name":"Major Solar Panel","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"","SBQQ__BillingType__c":"","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"One-Time","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""},
{"Id":"01tDm000000IW1SIAW","Name":"Ursa Major Warranty and Maintenance Plan","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"Quarterly","SBQQ__BillingType__c":"Advance","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"Recurring","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"Fixed Price","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"12.0","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""}
]

But when using streams

  const readStream = conn.bulk.query(query).stream(); //this stream provides the csv
  const writeStream = fs.createWriteStream(outputPath);

  await new Promise((resolve, reject) => {
    const errors = [];
    function handler(err) {
      console.log(err);
      errors.push(err);
    }

    readStream.on('error', handler);
    writeStream.on('error', handler);

    readStream.on('end', () => {
      if (errors.length) return reject({ success: false, errors });
      resolve({ success: true });
    });

    if (outputFormat === 'json') {
      readStream
        .pipe(
          csv(),
        )
        .pipe(writeStream);
    } else {
      readStream.pipe(writeStream);
    }
  });

I get this output. As you can see, each entry is a separate object. Why can't it create an array of objects?

{"Id":"01tDm000000IW1QIAW","Name":"SolarBots","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"Monthly","SBQQ__BillingType__c":"Advance","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"Recurring","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"Fixed Price","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"1.0","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""}
{"Id":"01tDm000000IW1RIAW","Name":"Major Solar Panel","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"","SBQQ__BillingType__c":"","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"One-Time","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""}
{"Id":"01tDm000000IW1SIAW","Name":"Ursa Major Warranty and Maintenance Plan","SBQQ__AssetAmendmentBehavior__c":"Default","SBQQ__AssetConversion__c":"One per quote line","SBQQ__BatchQuantity__c":"","SBQQ__BillingFrequency__c":"Quarterly","SBQQ__BillingType__c":"Advance","SBQQ__BlockPricingField__c":"Quantity","SBQQ__ChargeType__c":"Recurring","SBQQ__Component__c":"false","SBQQ__CompoundDiscountRate__c":"","SBQQ__ConfigurationEvent__c":"","SBQQ__ConfigurationFieldSet__c":"","SBQQ__ConfigurationFields__c":"","SBQQ__ConfigurationFormTitle__c":"","SBQQ__ConfigurationType__c":"","SBQQ__ConfigurationValidator__c":"","SBQQ__ConfiguredCodePattern__c":"","SBQQ__ConfiguredDescriptionPattern__c":"","SBQQ__CostEditable__c":"false","SBQQ__CostSchedule__c":"","SBQQ__CustomConfigurationPage__c":"","SBQQ__CustomConfigurationRequired__c":"false","SBQQ__CustomerCommunityAvailability__c":"","SBQQ__DefaultPricingTable__c":"","SBQQ__DefaultQuantity__c":"1.0","SBQQ__DescriptionLocked__c":"false","SBQQ__DiscountCategory__c":"","SBQQ__DiscountSchedule__c":"","SBQQ__DynamicPricingConstraint__c":"","SBQQ__ExcludeFromMaintenance__c":"false","SBQQ__ExcludeFromOpportunity__c":"false","SBQQ__ExternallyConfigurable__c":"false","SBQQ__GenerateContractedPrice__c":"","SBQQ__HasConfigurationAttributes__c":"false","SBQQ__HasConsumptionSchedule__c":"false","SBQQ__Hidden__c":"false","SBQQ__HidePriceInSearchResults__c":"false","SBQQ__IncludeInMaintenance__c":"false","SBQQ__NewQuoteGroup__c":"false","SBQQ__NonDiscountable__c":"false","SBQQ__NonPartnerDiscountable__c":"false","SBQQ__OptionLayout__c":"","SBQQ__OptionSelectionMethod__c":"Click","SBQQ__Optional__c":"false","SBQQ__PriceEditable__c":"false","SBQQ__PricingMethodEditable__c":"false","SBQQ__PricingMethod__c":"List","SBQQ__ProductPictureID__c":"","SBQQ__QuantityEditable__c":"true","SBQQ__QuantityScale__c":"","SBQQ__ReconfigurationDisabled__c":"false","SBQQ__RenewalProduct__c":"","SBQQ__SortOrder__c":"","SBQQ__Specifications__c":"","SBQQ__SubscriptionBase__c":"List","SBQQ__SubscriptionCategory__c":"","SBQQ__SubscriptionPercent__c":"","SBQQ__SubscriptionPricing__c":"Fixed Price","SBQQ__SubscriptionTarget__c":"","SBQQ__SubscriptionTerm__c":"12.0","SBQQ__SubscriptionType__c":"Renewable","SBQQ__TaxCode__c":"","SBQQ__Taxable__c":"false","SBQQ__TermDiscountLevel__c":"","SBQQ__TermDiscountSchedule__c":"","SBQQ__UpgradeCredit__c":"","SBQQ__UpgradeRatio__c":"","SBQQ__UpgradeSource__c":"","SBQQ__UpgradeTarget__c":"","SBQQ__EnableLargeConfiguration__c":"false","SBQQ__PricingGuidance__c":""}
@AllanOricil AllanOricil mentioned this issue Mar 9, 2023
Closed
@AllanOricil
Copy link
Author

AllanOricil commented Mar 9, 2023 •
edited
Loading

If you find this issue, the solution is simple. Just add this configuration to the csv transformer:

import fs from 'fs-extra';
import csv from 'csvtojson';
import { pipeline } from 'stream/promises';

...


async function exportData2File(
  conn: jsforce.Connection,
  query: string,
  outputPath: string,
  outputFormat?: 'csv' | 'json' = 'csv',
): Promise<AsynchronousJobResult> {
  await new Promise(async (resolve, reject) => {
    try {
      await pipeline(
        [
          conn.bulk.query(query).stream(),
          outputFormat === 'json' && csv({ downstreamFormat: 'array' }),
          fs.createWriteStream(outputPath),
        ].filter(Boolean),
      );
      resolve({
        success: true,
        message: 'success while exporting data to file',
      });
    } catch (ex) {
      reject({
        success: false,
        message: 'error while exporting data to file',
        errors: ex,
      });
    }
  });
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AllanOricil