degiro exporters only writes 12 transactions #4

dascgit · 2023-10-02T16:13:14Z

When trying to run the degiro exporters it only exports 12 transactions and then stops:
[nodemon] 2.0.22 [nodemon] to restart at any time, enter rs[nodemon] watching path(s): src\**\* [nodemon] watching extensions: ts,js [nodemon] startingts-node ./src/index.ts degiro Processing file using DeGiro converter Read CSV file C:\Users\User\Downloads\Account.csv. Start processing.. ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 4% | ETA 225s | Duration: 18s | 12/273 Processing complete, writing to file.. Wrote data to 'ghostfolio-degiro.json'! [nodemon] clean exit - waiting for changes before restart

The text was updated successfully, but these errors were encountered:

dickwolff · 2023-10-02T16:14:43Z

I have seen this issue yesterday as well when running the tool myself. Will work on this as soon as I have time!

dickwolff · 2023-10-02T18:17:59Z

I have created a branch that processes my test file. Can you check if this solves your issue? See this.

dascgit · 2023-10-03T05:44:33Z

Yahoo request seems to not work anymore
`The following result did not validate witt
[
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'EQUITY' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Equity' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'OPTION' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Option' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'ETF' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'ETF' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'MUTUALFUND' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Fund' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'INDEX' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Index' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'CURRENCY' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Currency' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'CRYPTOCURRENCY' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/typeDisp/const',
keyword: 'const',
params: { allowedValue: 'Cryptocurrency' },
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: { missingProperty: 'name' },
message: "must have required property 'name'",
data: {
exchange: 'NAS',
shortname: 'Columbia Government Money Marke',
quoteType: 'MONEY_MARKET',
symbol: 'IDSXX',
index: 'quotes',
score: 20010,
typeDisp: 'MoneyMarket',
longname: 'Columbia Funds Series Trust II - Columbia Government Money Market Fund',
exchDisp: 'NASDAQ',
isYahooFinance: true
}
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: { missingProperty: 'permalink' },
message: "must have required property 'permalink'",
data: {
exchange: 'NAS',
shortname: 'Columbia Government Money Marke',
quoteType: 'MONEY_MARKET',
symbol: 'IDSXX',
index: 'quotes',
score: 20010,
typeDisp: 'MoneyMarket',
longname: 'Columbia Funds Series Trust II - Columbia Government Money Market Fund',
exchDisp: 'NASDAQ',
isYahooFinance: true
}
},
{
instancePath: '/quotes/1/isYahooFinance',
schemaPath: '#/definitions/SearchQuoteNonYahoo/properties/isYahooFinance/const',
keyword: 'const',
params: { allowedValue: false },
message: 'must be equal to constant',
data: true
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/quoteType/const',
keyword: 'const',
params: { allowedValue: 'FUTURE' },
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/typeDisp/enum',
keyword: 'enum',
params: { allowedValues: [ 'Future', 'Futures' ] },
message: 'must be equal to one of the allowed values',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/properties/quotes/items/anyOf',
keyword: 'anyOf',
params: {},
message: 'must match a schema in anyOf',
data: {
exchange: 'NAS',
shortname: 'Columbia Government Money Marke',
quoteType: 'MONEY_MARKET',
symbol: 'IDSXX',
index: 'quotes',
score: 20010,
typeDisp: 'MoneyMarket',
longname: 'Columbia Funds Series Trust II - Columbia Government Money Market Fund',
exchDisp: 'NASDAQ',
isYahooFinance: true
}
}
]

This may happen intermittently and you should catch errors appropriately.
However: 1) if this recently started happening on every request for a symbol
that used to work, Yahoo may have changed their API. 2) If this happens on
every request for a symbol you've never used before, but not for other
symbols, you've found an edge-case (OR, we may just be protecting you from
"bad" data sometimes stored for e.g. misspelt symbols on Yahoo's side).
Please see if anyone has reported this previously:

https://github.com/gadicc/node-yahoo-finance2/issues?q=is%3Aissue+Failed%20validation%3A%20%23%2Fdefinitions%2FSearcht

or open a new issue (and mention the symbol): yahoo-finance2 v2.8.0

https://github.com/gadicc/node-yahoo-finance2/issues/new?labels=bug%2C+validation&template=validation.md&title=Failedt

For information on how to turn off the above logging or skip these errors,
see https://github.com/gadicc/node-yahoo-finance2/tree/devel/docs/validation.md.

At the end of the doc, there's also a section on how to
Help Fix Validation Errors
in case you'd like to contribute to the project. Most of the time, these
fixes are very quick and easy; it's just hard for our small core team to keep up,
so help is always appreciated!

FailedYahooValidationError: Failed Yahoo Schema validation
at validate (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs\src\lib\v)
at Object. (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs)
at Generator.next ()
at fulfilled (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs\src\lib)
at processTicksAndRejections (node:internal/process/task_queues:95:5) {
result: {
explains: [],
count: 7,
quotes: [
[Object], [Object],
[Object], [Object],
[Object], [Object],
[Object]
],
news: [],
nav: [],
lists: [],
researchReports: [],
screenerFieldResults: [],
totalTime: 37,
timeTakenForQuotes: 427,
timeTakenForNews: 0,
timeTakenForAlgowatchlist: 400,
timeTakenForPredefinedScreener: 400,
timeTakenForCrunchbase: 400,
timeTakenForNav: 400,
timeTakenForResearchReports: 0,
timeTakenForScreenerField: 0,
timeTakenForCulturalAssets: 0
},
errors: [
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: [Object],
message: "must have required property 'name'",
data: [Object]
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: [Object],
message: "must have required property 'permalink'",
data: [Object]
},
{
instancePath: '/quotes/1/isYahooFinance',
schemaPath: '#/definitions/SearchQuoteNonYahoo/properties/isYahooFinance/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: true
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/typeDisp/enum',
keyword: 'enum',
params: [Object],
message: 'must be equal to one of the allowed values',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/properties/quotes/items/anyOf',
keyword: 'anyOf',
params: {},
message: 'must match a schema in anyOf',
data: [Object]
}
]
}
C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs\src\lib\validateAndCoerceT4
throw new errors_js_1.FailedYahooValidationError("Failed Yahoo Schema validation", {
^
FailedYahooValidationError: Failed Yahoo Schema validation
at validate (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs\src\lib\v)
at Object. (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs)
at Generator.next ()
at fulfilled (C:\temp\Export-To-Ghostfolio\node_modules\yahoo-finance2\dist\cjs\src\lib)
at processTicksAndRejections (node:internal/process/task_queues:95:5) {
result: {
explains: [],
count: 7,
quotes: [
[Object], [Object],
[Object], [Object],
[Object], [Object],
[Object]
],
news: [],
nav: [],
lists: [],
researchReports: [],
screenerFieldResults: [],
totalTime: 37,
timeTakenForQuotes: 427,
timeTakenForNews: 0,
timeTakenForAlgowatchlist: 400,
timeTakenForPredefinedScreener: 400,
timeTakenForCrunchbase: 400,
timeTakenForNav: 400,
timeTakenForResearchReports: 0,
timeTakenForScreenerField: 0,
timeTakenForCulturalAssets: 0
},
errors: [
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooEquity/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooOption/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooETF/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFund/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooIndex/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCurrency/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooCryptocurrency/properties/typeDisp/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: [Object],
message: "must have required property 'name'",
data: [Object]
},
{
instancePath: '/quotes/1',
schemaPath: '#/definitions/SearchQuoteNonYahoo/required',
keyword: 'required',
params: [Object],
message: "must have required property 'permalink'",
data: [Object]
},
{
instancePath: '/quotes/1/isYahooFinance',
schemaPath: '#/definitions/SearchQuoteNonYahoo/properties/isYahooFinance/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: true
},
{
instancePath: '/quotes/1/quoteType',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/quoteType/const',
keyword: 'const',
params: [Object],
message: 'must be equal to constant',
data: 'MONEY_MARKET'
},
{
instancePath: '/quotes/1/typeDisp',
schemaPath: '#/definitions/SearchQuoteYahooFuture/properties/typeDisp/enum',
keyword: 'enum',
params: [Object],
message: 'must be equal to one of the allowed values',
data: 'MoneyMarket'
},
{
instancePath: '/quotes/1',
schemaPath: '#/properties/quotes/items/anyOf',
keyword: 'anyOf',
params: {},
message: 'must match a schema in anyOf',
data: [Object]
}
]
}
[nodemon] app crashed - waiting for file changes before starting...`

dickwolff · 2023-10-03T05:47:57Z

I see, I had this before and disabled YF validation but it seems I only did this for the Quote Summary request. I'll add the validation skip to search as well then.

dickwolff · 2023-10-13T12:49:22Z

I have added the validation skip to the proposed fix branch here. Can you validate if the issue persists?

I also fixed the number in the progress bar, as this was not representative to the real number of processed lines (there were some "move to next line" actions that did not increment the bar).

dascgit · 2023-10-13T17:44:46Z

Had no errors during the convert task. But the JSON now only has "DIVIDEND" entries and no BUY or SELL entries. The exported CSV but has BUY and SELL entries.

dickwolff · 2023-10-13T17:57:58Z

Allright, will look into it later this weekend and check of my example export has the same issue. Will also try a recent export of mine in case the example export is oudated.

dascgit · 2023-10-13T18:04:40Z

Fyi my BUY and SELL entries do not have a @ symbol in the CSV. Since mine is in german its just BUY for example: "Kauf XX zu je XX.XX EUR (DEXXXXXX)"

dickwolff · 2023-10-13T18:06:49Z

That could be the root cause. The Dutch export does have that '@'.

Could you send me a file (or lines) of buy/sell in German so I can work with that?

dickwolff · 2023-10-13T18:26:43Z

Thank you, will look into later this weekend

dickwolff · 2023-10-15T13:18:14Z

@dascgit I have added some German specific checks and the example you sent me can be processed now.

dascgit · 2023-10-16T06:27:01Z

Now I get this output:
Processing file using DeGiro converter Read CSV file C:\Users\User\Downloads\Account.csv. Start processing.. █████████████████████░░░░░░░░░░░░░░░░░░░ 52% | ETA 80s | Duration: 53s | 147/278 C:\git\Export-To-Ghostfolio\src\converters\degiroConverter.ts:185 result.activities[result.activities.length - 1].symbol = security.symbol; ^ TypeError: Cannot read properties of null (reading 'symbol') at DeGiroConverter.<anonymous> (C:\git\Export-To-Ghostfolio\src\converters\degiroConverter.ts:185:81) at Generator.next (<anonymous>) at fulfilled (C:\git\Export-To-Ghostfolio\src\converters\degiroConverter.ts:28:58) at processTicksAndRejections (node:internal/process/task_queues:95:5) [nodemon] app crashed - waiting for file changes before starting...

dickwolff · 2023-10-22T17:18:36Z

Will look in to this probably later this week, haven’t had time due to other responsibilities.

I also saw Ghostfolio added a new api for search, might look into that and then remove my own implementation if Ghostfolio works as fast. The current list of issues started when I moved from Ghostfolio search to my own implementation of search because the old Ghostfolio api was extremely slow since 2.x release.

dickwolff · 2023-10-27T08:41:08Z

Closed by mistake (by merging to main).

Can you please send me the order line and preceding transaction fee line that causes issues? This is probably because there was no symbol found on the sell order. But I need to check to be sure.

dascgit · 2023-10-29T08:23:23Z

I have no idea which line or transaction it was to be honest. I have seen the CSV Export from degiro does strange things. For example they split one line into two lines and the second line misses a lot of informations.

`LINE150: 16-03-2023,14:23,16-02-2023,BASF SE,DE000BASF111,Währungswechsel (Ausbuchung),1.0154,EUR,-XX.XX,EUR,0.00,17d16616-7634-11ee-

LINE151: ,,,,,,,,,,,b962-0242ac120002`

Maybe the symbol that could not be found was "GB00BDVZYZ77" ?

mlbullett · 2023-10-29T10:42:19Z

Hi there, I've been running into a similar issue. Hopefully this anonymized data can help elucidate the issue:

getSecurity(): Found 1 matches by ISIN <ISIN_NUMBER_1>
getSecurity(): No initial match found, trying by symbol <SYMBOL_1>
getSecurity(): No match found for symbol <SYMBOL_1>, trying by name <NAME_1>.
        getSymbolsByQuery(): Quote has no symbol at Yahoo Finance undefined. Skipping..
        getSymbolsByQuery(): Quote has no symbol at Yahoo Finance undefined. Skipping..
getSecurity(): Found 1 matches by ISIN <ISIN_NUMBER_1>
getSecurity(): No initial match found, trying by symbol <SYMBOL_1>
getSecurity(): No match found for symbol <SYMBOL_1>, trying by name <NAME_1>.
███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 28% | ETA 91s | Duration: 47s | 110/384/<path>/Export-4
              result.activities[result.activities.length - 1].symbol = security.symbol;
                                                                                ^
TypeError: Cannot read properties of null (reading 'symbol')
    at DeGiroConverter.<anonymous> (/<path>/Export-To-Ghostfolio/src/converters/degiroConverter)
    at Generator.next (<anonymous>)
    at fulfilled (/<path>/Export-To-Ghostfolio/src/converters/degiroConverter.ts:28:58)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
[nodemon] app crashed - waiting for file changes before starting...

The first line in the csv where we encounter that security is a transaction fee labelled as "DEGIRO Transaktionsgebühren und/oder Fremdkosten". That is then followed by the actual purchase, labelled "Kauf AMOUNT zu je PRICE EUR (ISIN)", where we encounter the crash. Both possess the same ISIN.

Looks like we find the security just fine with the ISIN but because this security is listed on TDG under a different symbol than on NASD it struggles to match it to the expected currency in findSymbolMatch. I'm guessing our trouble may come from that unhandled edge case but it's just a shot in the dark.

dickwolff · 2023-12-01T09:53:10Z

I have no idea which line or transaction it was to be honest. I have seen the CSV Export from degiro does strange things. For example they split one line into two lines and the second line misses a lot of informations.

`LINE150: 16-03-2023,14:23,16-02-2023,BASF SE,DE000BASF111,Währungswechsel (Ausbuchung),1.0154,EUR,-XX.XX,EUR,0.00,17d16616-7634-11ee-

LINE151: ,,,,,,,,,,,b962-0242ac120002`

Maybe the symbol that could not be found was "GB00BDVZYZ77" ?

@dascgit I have added "Währungswechsel" lines to the skip list, so this should make sure you can continue processing.

dickwolff · 2023-12-01T09:55:12Z

Hi there, I've been running into a similar issue. Hopefully this anonymized data can help elucidate the issue:
getSecurity(): Found 1 matches by ISIN <ISIN_NUMBER_1>
getSecurity(): No initial match found, trying by symbol <SYMBOL_1>
getSecurity(): No match found for symbol <SYMBOL_1>, trying by name <NAME_1>.
        getSymbolsByQuery(): Quote has no symbol at Yahoo Finance undefined. Skipping..
        getSymbolsByQuery(): Quote has no symbol at Yahoo Finance undefined. Skipping..
getSecurity(): Found 1 matches by ISIN <ISIN_NUMBER_1>
getSecurity(): No initial match found, trying by symbol <SYMBOL_1>
getSecurity(): No match found for symbol <SYMBOL_1>, trying by name <NAME_1>.
███████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 28% | ETA 91s | Duration: 47s | 110/384/<path>/Export-4
              result.activities[result.activities.length - 1].symbol = security.symbol;
                                                                                ^
TypeError: Cannot read properties of null (reading 'symbol')
    at DeGiroConverter.<anonymous> (/<path>/Export-To-Ghostfolio/src/converters/degiroConverter)
    at Generator.next (<anonymous>)
    at fulfilled (/<path>/Export-To-Ghostfolio/src/converters/degiroConverter.ts:28:58)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
[nodemon] app crashed - waiting for file changes before starting...
The first line in the csv where we encounter that security is a transaction fee labelled as "DEGIRO Transaktionsgebühren und/oder Fremdkosten". That is then followed by the actual purchase, labelled "Kauf AMOUNT zu je PRICE EUR (ISIN)", where we encounter the crash. Both possess the same ISIN.

Looks like we find the security just fine with the ISIN but because this security is listed on TDG under a different symbol than on NASD it struggles to match it to the expected currency in findSymbolMatch. I'm guessing our trouble may come from that unhandled edge case but it's just a shot in the dark.

@mlbullett Can you give me the name of the stock/etf that gives you issues? That way I can check wether I get the same issue. I have also added a "prefered exchange" config setting recently. You can set this so a prefered exchange postfix (e.g. .AS for Euronext Amsterdam) and then we pick that one if it is present in the result list.

mlbullett · 2023-12-01T10:59:13Z

Hi @dickwolff, thanks for replying.
Sure, in this instance it was US88160R1014 but it also happened with other securities when I removed it like US0079031078.

dickwolff · 2023-12-01T11:02:09Z

I have added some checks in this branch for a similar issue someone else had. Could you check the version in that branch and see if it gives you the same issue, or if that change resolves it?

mlbullett · 2023-12-01T11:07:02Z

Cheers, that seems to have solved the issue although now I've run into another issue with US90184L1026 but that's unrelated to this.

dickwolff · 2023-12-02T10:04:54Z

It seems that Yahoo Finance does no longer recognize TWTR since it has been taken private by Elon.

mlbullett · 2023-12-02T10:14:15Z

Yeah I figured. Shame for record keeping but not the end of the world as the position is resolved. Thanks for the assist in the main issue!

dickwolff self-assigned this Oct 2, 2023

dickwolff mentioned this issue Oct 2, 2023

Feature/fix-degiro-processing #5

Merged

dickwolff mentioned this issue Oct 22, 2023

UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'length' of undefined at DeGiroConverter #6

Closed

dickwolff closed this as completed in #5 Oct 27, 2023

dickwolff reopened this Oct 27, 2023

dickwolff added the Bug Something isn't working label Oct 27, 2023

dickwolff mentioned this issue Dec 1, 2023

Fault handling on empty result and provide product name for search #12

Merged

dickwolff added the DEGIRO Related to DEGIRO label Dec 2, 2023

dickwolff closed this as completed in #12 Dec 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

degiro exporters only writes 12 transactions #4

degiro exporters only writes 12 transactions #4

dascgit commented Oct 2, 2023

dickwolff commented Oct 2, 2023

dickwolff commented Oct 2, 2023 •

edited

Loading

dascgit commented Oct 3, 2023

dickwolff commented Oct 3, 2023

dickwolff commented Oct 13, 2023 •

edited

Loading

dascgit commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dascgit commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dickwolff commented Oct 15, 2023 •

edited

Loading

dascgit commented Oct 16, 2023

dickwolff commented Oct 22, 2023

dickwolff commented Oct 27, 2023 •

edited

Loading

dascgit commented Oct 29, 2023 •

edited

Loading

mlbullett commented Oct 29, 2023

dickwolff commented Dec 1, 2023

dickwolff commented Dec 1, 2023

mlbullett commented Dec 1, 2023

dickwolff commented Dec 1, 2023 •

edited

Loading

mlbullett commented Dec 1, 2023

dickwolff commented Dec 2, 2023

mlbullett commented Dec 2, 2023

degiro exporters only writes 12 transactions #4

degiro exporters only writes 12 transactions #4

Comments

dascgit commented Oct 2, 2023

dickwolff commented Oct 2, 2023

dickwolff commented Oct 2, 2023 • edited Loading

dascgit commented Oct 3, 2023

dickwolff commented Oct 3, 2023

dickwolff commented Oct 13, 2023 • edited Loading

dascgit commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dascgit commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dickwolff commented Oct 13, 2023

dickwolff commented Oct 15, 2023 • edited Loading

dascgit commented Oct 16, 2023

dickwolff commented Oct 22, 2023

dickwolff commented Oct 27, 2023 • edited Loading

dascgit commented Oct 29, 2023 • edited Loading

mlbullett commented Oct 29, 2023

dickwolff commented Dec 1, 2023

dickwolff commented Dec 1, 2023

mlbullett commented Dec 1, 2023

dickwolff commented Dec 1, 2023 • edited Loading

mlbullett commented Dec 1, 2023

dickwolff commented Dec 2, 2023

mlbullett commented Dec 2, 2023

dickwolff commented Oct 2, 2023 •

edited

Loading

dickwolff commented Oct 13, 2023 •

edited

Loading

dickwolff commented Oct 15, 2023 •

edited

Loading

dickwolff commented Oct 27, 2023 •

edited

Loading

dascgit commented Oct 29, 2023 •

edited

Loading

dickwolff commented Dec 1, 2023 •

edited

Loading