Exception: volume is not double

[Fredrik81]

Step 1: Have you search for this issue before posting it?

Yes and talked about it on slack.

Step 2: Describe your environment

• Operating system: Ubuntu 19.10
• Python Version: Python 3.7.5
• CCXT version: ccxt==1.22.61
• Branch: Develop
• Last Commit ID: 43add0b

Step 3: Describe the problem:

When running hyperopt using downloaded data an exception will be thown.
Exception: volume is not double

When doing some troubleshooting it only affects some of the downloaded data (not all pairs)
XRP/BTC

date      datetime64[ns, UTC]
open                  float64
high                  float64
low                   float64
close                 float64
volume                  int64
dtype: object

Steps to reproduce:

1. Not sure :-/
2. The main indicator that throws error is MFI (dataframe['mfi'] = ta.MFI(dataframe))

Observed Results:

Unable to run hyperopt

Relevant code exceptions or logs:

2020-02-20 22:47:28,025 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
File "/opt/tradebot/freqtrade/.env/lib/python3.7/site-packages/talib/__init__.py", line 20, in wrapper
  for arg in chain(args, kwargs.values())
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/freqtrade/freqtrade/main.py", line 36, in main
  return_code = args['func'](args)
File "/freqtrade/freqtrade/commands/optimize_commands.py", line 76, in start_hyperopt
  hyperopt.start()
File "/freqtrade/freqtrade/optimize/hyperopt.py", line 488, in start
  preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data)
File "/freqtrade/freqtrade/strategy/interface.py", line 448, in tickerdata_to_dataframe
  for pair, pair_data in tickerdata.items()}
File "/freqtrade/freqtrade/strategy/interface.py", line 448, in <dictcomp>
  for pair, pair_data in tickerdata.items()}
File "/freqtrade/freqtrade/strategy/interface.py", line 464, in advise_indicators
  return self.populate_indicators(dataframe, metadata)
File "/freqtrade/user_data/strategies/MyDev.py", line 205, in populate_indicators
  dataframe['mfi'] = ta.MFI(dataframe)
File "talib/_abstract.pxi", line 352, in talib._ta_lib.Function.__call__
File "talib/_abstract.pxi", line 383, in talib._ta_lib.Function.__call_function
File "/opt/tradebot/freqtrade/.env/lib/python3.7/site-packages/talib/__init__.py", line 24, in wrapper
  return func(*args, **kwargs)
File "talib/_func.pxi", line 8454, in talib._ta_lib.MFI
Exception: volume is not double

[Fredrik81]

tried to delete downloaded data and re-download. Same issue

[hroff-1902]

Thanks,

Similar was fixed an year ago: #1546

Either the fix was missing with later changes, or it was only applicable for non-stored dataframes, used during live/dry runs (so the same should be applied to downloaded data or during their load)

[Fredrik81]

Still get same error message :-(

• Last Commit ID: 430f53c

What was done:

• Updated to latest devel version
• Deleted all downloaded data
• re-downloaded data
• tested hyperopt

Error:

2020-02-22 16:15:33,158 - freqtrade - ERROR - Fatal exception!
Traceback (most recent call last):
  File "/opt/tradebot/freqtrade/.env/lib/python3.7/site-packages/talib/__init__.py", line 20, in wrapper
    for arg in chain(args, kwargs.values())
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/freqtrade/freqtrade/main.py", line 36, in main
    return_code = args['func'](args)
  File "/freqtrade/freqtrade/commands/optimize_commands.py", line 76, in start_hyperopt
    hyperopt.start()
  File "/freqtrade/freqtrade/optimize/hyperopt.py", line 488, in start
    preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data)
  File "/freqtrade/freqtrade/strategy/interface.py", line 448, in tickerdata_to_dataframe
    for pair, pair_data in tickerdata.items()}
  File "/freqtrade/freqtrade/strategy/interface.py", line 448, in <dictcomp>
    for pair, pair_data in tickerdata.items()}
  File "/freqtrade/freqtrade/strategy/interface.py", line 462, in advise_indicators
    return self.populate_indicators(dataframe)  # type: ignore
  File "/freqtrade/user_data/strategies/MyDev.py", line 218, in populate_indicators
    dataframe['mfi'] = ta.MFI(dataframe)
  File "talib/_abstract.pxi", line 352, in talib._ta_lib.Function.__call__
  File "talib/_abstract.pxi", line 383, in talib._ta_lib.Function.__call_function
  File "/freqtrade/.env/lib/python3.7/site-packages/talib/__init__.py", line 24, in wrapper
    return func(*args, **kwargs)
  File "talib/_func.pxi", line 8454, in talib._ta_lib.MFI
Exception: volume is not double

[hroff-1902]

(an organizational point)

@xmatthias strange, @Fredrik81 shown me a screenshot, he does not have a 'Reopen issue' button near green "Comment"... Are users able to reopen issues at all? There should be a configuration option for the repo for this...

(Added: I found the description for workflow implemented in github: https://stackoverflow.com/questions/21333654/how-to-re-open-an-issue-in-github/21333938#21333938

• you can re-open your own issues *if you closed them yourself
• you cannot re-open your own issues if a repo collaborator closed them

Seems it's not customizable...)

[hroff-1902]

back to the issue itself:

I do not why, but this:

        pairdata = read_json(filename, orient='values', dtype=False)

resolves the issue (volumes are float64)...
Volumes are float in the json datatypes, but dtype: dict converts them to ints... Very strange behavior...

[hroff-1902]

the problem is that columns after read_json do not have names 'date', 'close', etc, so dtypes= does not work...