2022.8

Highlighted changes

• FreqAI: an interface enabling users to train, backtest, and deploy custom predictive models.
• Strategies can now send notifications via self.dp.send_msg().
• Slightly modified liquidation price handling, allowing liquidation prices to change as needed.
• Support for partial exits through adjust_trade_position(). Make sure to read the documentation to understand how calculations for this will work.
• Partial exits through FreqUI.
• /whitelist telegram commands now support optional arguments "sorted" and "nobase".
• Add --show-timerange to the list-data subcommand to show available timerange.
• Okx leverage tier caching (Speeds up startup).
• Improved backtesting results by applying exchange precision limits.

How to update

As always, you can update your bot using one of the following commands:

docker-compose

docker-compose pull
docker-compose up -d

Installation via setup script

# Deactivate venv and run 
./setup.sh --update

Plain native installation

git pull
pip install -U -r requirements.txt

Expand full changelog

2403a03fc Version bump 2022.8
8b0cfe123 Reduce image sizes in freqai doc (#7304)
a44a235b5 isort imports in tests/freqai
6634229cc appease the flake8 gods
fcb5d1cb5 remove debugging flag
dd628eb52 add tests for outlier detection and removal functions
1e41c773a fix outlier protection
22b42e91f add new parameter to freqai doc
b9f35cadb add /stopentry alias for /stopbuy
59a723aec Add /health to rest client
71f7d6878 Fixed mypy error
c3e74e6e8 Improve doc wording
2b70c3d0c support price callback for partial exits in bt
9204f0131 Don't lock pairs on partial exit
86c5ac44e Add outlier percentage check
2ef4534fe Fix ccxt / longrun tests
efe4fd3e2 Add libgomp1 to dockerfile
01126c43f Fix liquidation price tier calculation
753d1b2aa Update leverage tier terminology to be clear and aligned with ccxt
b2d664c63 Change MinPts calculation
53d46a038 align `max_entry_position_adjustment` behavior of backtesting to live
1fd223c81 Update --prepend help string
f2a356a80 Fix some imports
6636f17e0 Simplify usage of amount_to_contract precision
9e48e6a40 Update docs about precision limits in backtesting
205ab26e9 Remove TODO in test
70df03769 Improve test precision
32faad933 Fix backtest calculation problem with DCA
a6d78a861 initialize Since parameter properly
fe7108ae7 Convert amount to contracts before comparing for close
78b161e14 add contract_size to database
6036018f3 Extract contracts_to_amount and amount_to_contracts to standalone functions
1b0f37a93 Fix documentation typo
5f38a574c Add okx broker id
5ce1c6980 Improve DBSCAN epsilon identification (#7269)
96d8882f1 Plug mem leak, add training timer
f55d5ffd8 Don't fail when `--strategy-path` is not a valid directory.
015be770c ccxt now defaults to base volume for all markets
93d2f7fc8 types-requests bump pre-commit
ff9ed1aba Bump ccxt from 1.92.20 to 1.92.52
354d3c0cd Bump time-machine from 2.7.1 to 2.8.1
eeb177110 Bump jsonschema from 4.9.1 to 4.14.0
70848a258 Bump types-requests from 2.28.8 to 2.28.9
3958e53aa Bump mkdocs-material from 8.4.0 to 8.4.1
dfa7d1fc2 Bump fastapi from 0.79.0 to 0.79.1
2dc34779d Fix line length
f6d832c6d Add get_option to expose ft_has via method
87a311507 Add get_open_trade_count() to simplify getting open trade count.
085f81ec9 Fix header indent on stake-size-management
0ec38e0cf Fix typo in docs
1fb2e9558 Disable caching of leverage tiers in ccxt compat methods
5b3f03159 Use hyperopt safe amount precision method
4511634f3 improve test coverage
738e95b87 Add tests for leverage tiers caching
b6e8b9df3 Use cached leverage tiers
52ec0d104 Update binance Leverage tiers
7563050f1 Realign tests to precision backtesting
0da060083 Have backtesting respect tradable size
54ddc1a4c Add --tradingmode alias
aa3da092a Dont' use classProperty - that's not supported on 3.8
665cf4431 Add explicit test cov. for .range behavior
692c6bf1f Added and updated figs and fig descriptions
88d6a7fbf additional edits
9c6b745f0 Restructure and improve doc, add fiq
b9d48c327 use numbers in HyperoptState properly ...
1389c8f5b Update documentation with show-timerange option
733f71681 Update documentation
bc359675a Add --analyze-per-epoch - moving populate_analysis to the epoch process
09f890454 Extract analysis to separate method
08ef5ad2d Add HyperoptState enum and container class
1c6f96657 Hyperopt: simplify parameter "can_optimize" handling
16af10a5b Update notebook sample with simplified datadir configuration
b7553d20d Get min/max data in list-data command
7d84ef2e2 Remove unused imports
521381ebf undo example strategy newline
cfa5b3f12 add new line
277245c69 remove line
b420614d6 Reduce code duplication in datahandlers
975bf8fe8 Update Docstring to match actual return values
47b314353 Simplify and fix some tests
42eb50851 Attempt fix of #7184
76a3e97e0 Add migrations end message
70a77ba3d Check for "last" availability in PrecisionFilter
1fada53dd remove newline
85b43ec1a Remove double-check for "isolated margin"
fde469a25 Remove unnecessary check
075e9b852 Log Exchange responses for set_leverage
46e8d9a5e Reduce verbosity when whitelist is empty
18fab8643 Add dock segment about webserver mode for docker
0461a8934 Fix test failures
f70b0bab8 remove line
66910bfe6 Don't fail if mark candles are missing
9c38c27ee ignore sample itself distance for avg_mean_dist computation
72c34291e newline
398336822 gateio futures is showing base currency in volume!
c615e1bc6 Avoid loop error during ccxt tests
b9667f50c Fix random test failure
e7902bffa Remove checks for dataprovider existance - it's available in all modes.
e0883a4ea Improve doc wording
819bc7194 Update docs for freqai docker container
463cf6688 Fix bad image name
c8d7c2caa Update CI to build and push freqAI images
8d182768f stoploss should also use trimmed prices
0b0e7eaf9 Mutex TTL Cache accesses which can be accessed by multiple threads
24690c191 Don't convert open_rate to precision
3b44dc52e Minor corrections
ea6bc47d7 gateio default configs should specify unknown_fee_rate
5dde011b3 Add unknown_fee_rate to full sample config
a1e4fbf31 Run price_to_precision for dry-run orders
1ac81aa31 Show message if fee update failed due to missing
4c0fda400 fix input shape warning for LGBMClassifier, add sample_weights/eval_weights
fa89368c0 Add test for precision backpopulation
96d2f6181 Properly round timestamps to avoid constant log messages
b8c1cf010 Fix test leakage if config is available
15a1c59a9 Backtesting should cache precisionMode
a73e4f8e4 Truncate amount before comparing for closure
2fb7a3091 Improve backfill of precisions
a5b438e41 Run price_to_precision for dry-run orders
1dd56e35d Ensure comparisions align when closing a trade
e4b7bcaea Fix some tests
e81879742 Minor fix in amount_to_precision logic
c0bdb7181 Update docstring
f2b6ff910 Accept wrong pair in get_precision_amount
e6af9a690 Allow empty precisionMode on conversions
c3f159bd5 Add precision fields to database
22241c55d Add methods to get precision_amount from markets
15e85797c Simplify to_precision tests and imports
6c3233174 Move precision calculations to standalone functions
c7e171921 Fix interface import sorting
398b2946b Update test formatting
dba7d7fd6 Bump ccxt from 1.91.93 to 1.92.20
6fb5fbdd3 Bump orjson from 3.7.11 to 3.7.12
bc79027cf Bump mkdocs-material from 8.3.9 to 8.4.0
2581acd75 Bump filelock from 3.7.1 to 3.8.0
baa0af68b Bump plotly from 5.9.0 to 5.10.0
025ff27dd Bump numpy from 1.23.1 to 1.23.2
96c279f86 Bump nbconvert from 6.5.0 to 6.5.3
4b708caa6 Bump sqlalchemy from 1.4.39 to 1.4.40
3f6d42708 add a check for number of training features in tests
006b11e5d fix leftover bug in indicator population
8961b8d56 merge in inference timer and historic predictions handling improvements.
ad846cdb7 fix lock bug, update docstring
464d99808 Update doc table formatting
d442383a1 Fix ta-lib install script
a29402ddd Rename and move analysis_lock to data_kitchen
3a9ec76c9 Move "freqai.lock" to backend to simplify user interface
a5e96881f slightly update doc wording
e7513c96b install py-find-1st from conda forge
24f1dc4ec Update patched ta-lib install for conda
044cf8bb2 Allow new whitelist combination in "button" commands
c9c128f78 finalize logo, improve doc, improve algo overview, fix base tensorflowmodel for mypy
8d9284a52 Fix docs edit button
7a2b4dbb9 Fix docs edit button
58de20af0 make BaseClassifierModel. Add predict_proba to lightgbm
31be707cc clean up code, add docstrings
3e38c1b0b take dynamic sized tail off historic_predictions as return dataframe to strategy.
7d448fd4a allow fit_live_predictions access to current pair
1f192be43 avoid denormalizing labels twice
b1b76a2db debug classifier with predict proba
23cc21ce5 add predict_proba to base classifier, improve historic predictions handling
61acbf21d Fix broken telegram tests
7075b00e2 Remove odd dry run stoploss behavior
7c18ec405 Add missing key to "full" config sample
e09fbe9e5 Improve test resiliance
d36da9594 Fix bad import
82ac8cb41 Add freqai backtesting_load test
0b92c30ab Fix typo in test file
5aaab75d1 Add test for dynamic_pairlist_expand
1ac6ec144 Fix failing test...
b682fc446 Graciously fail if strategy has freqAI code, but freqAI is not enabled.
c190d57f1 Test populate_any_indicator interface
3918f4abb Simplify strategy interface by removing explicit self.freqai_info assignment
3b827ee60 Add "freqai.enabled" flag to disable freqAI via config flag
49989012a Bump catboost requirement to latest
f6545ebdb Disallow backtesting with --strategy-list for now.
e3a5b97b4 Update recalc_from_trades to use FtPrecise
9513c39a1 Fix migration rounding test
3bcb47d75 Remove usage of Decimal
902afc2f0 Use FtPrecise in interest calculation
da253f12f Bump CCXT to required version
0e61c2d05 Replace Decimal with FtPrecise in trade_model
2312b86a6 Update telegram-usage.md
ccc0ad6f6 fix - reload whitelist
923f73a51 nobase -> baseonly
fb4b73ce8 ensure dates are saved
b427c7ff1 Use diff. close time to avoid buggy behavior
d93bb8219 Add more Commits to failing test
aa1bf2adb Try fix windows testfailure
cc885e25a Improve NAN Handling in RPC
de690b0a6 Use PEP440 compatible versioning
dd4e44931 Improve NAN handling in RPC module
f7502bcc9 slightly update dca_short test
2cae3c42e remove trade database analyzer, clean up a bit
ace962648 Update tests for sorted and nobase
c0d60c63a Optional /whitelist args - sorted, nobase
ed004236c Add float initializer to FtPrecise
e7cb1b737 Wrap Precise into  FtPecise
91d0c9128 improve docs
adc8ee88e Move periodicCache to Utils package
573964b19 Dry Market orders should update "remaining"
ce2c9bf26 Slight renaming of variable
736884c5a Orders should be allowed to have empty fill/remaining values
b5c5a95b6 FTX: Use conditionalOrders endpoint to get proper stop-market order id
4289c5c68 update freqai logo
5a16d5a51 Deactivate database analyzer if user does not use sqlite
e7de81294 Allow user to user pair_dict for persistent storage of custom data
aef086b02 Improved dict typing, timeframe parser, collect dates associated with training data points
02646a4a0 Update scikit-learn in freqai deps
77b3b8a13 Use main exchange instead of creating a separate instance.
20b413478 Properly exclude catboost in ARM
0c7d862aa types-requests bump pre-commit
71c88244f Bump ccxt from 1.91.55 to 1.91.93
97c077171 Bump types-requests from 2.28.6 to 2.28.8
a45a35f38 Bump jsonschema from 4.9.0 to 4.9.1
7fd3f98ae Bump scikit-learn from 1.1.1 to 1.1.2
11a2eb6cc Bump flake8 from 5.0.1 to 5.0.4
ea64f43e5 bring back auto DF resizing for okx
67c722c9c fix asyncio bug
e48e82232 Force response API to js to fix faulty system configs
0b2104fc7 Properly increment the api version
5182f755f Add debug setup documentation
6ded2d5b7 Improve forceexit API test
d3780b931 Add test passing  leverage to execute_entry
d1998f7ed Fix forceexit calling
eff8cd7ec Add leverage to force_entry
daf015d00 extract nested force_exit function to private instance function
82aecc81f Accept parameters to forceexit
aaa534900 Add script for patching conda libta-lib
78e129034 Update docs to specify trading limit behaviour
eb8bde37c Add lightgbm classifier, add classifier check test, fix classifier bug.
bfa859e61 Remove unnecessary method (simplify)
5250189f7 Add Rollback function to Trade
47a30047e Fix typo
b16f57cb0 Minor stylistic fixes
c172ce101 improve flexibility of user defined prediction dataframe
45d68222a Reduce verbosity of Fiat Converter
fdc82f830 add doc section for classifier
f8f1ade16 Reduce function complexity by extracting message sending
268763394 Test iterative sending of /status
b12dd15f4 Send multiple messages in /status  if required
07763d0d4 add classifier, improve model naming scheme
ce8fbbf74 ensure loading historical df matches frequi indices
60d782e5c remove unnecessary function
a42a060ab fix DB once and for all. Make DBSCAN more efficient and robust.
a3799c4d5 start frequi with historical data if available
29b7b014e fix bug in DB path initialization
db1d36794 fix bug associated to fit_live_predictions_candles
26de992d2 ensure user sets startup candles in backtesting mode
05ec5c5e5 generalize database url path for any db type
954540245 Improve defaults for config builder
29e41cc81 Update docs to reflect correct result
7675187c3 Use telegram message length to avoid constants
cffc76954 Fix /profit endpoint calculations for partial sells
c6e121ffb Update tests with correct usdt mock trades
a8541d86f Edit index of `custom_stake_amount`
debc73b65 Edit Typo Custom-stake-amount
df4a5a757 Bump python from 3.10.5-slim-bullseye to 3.10.6-slim-bullseye
51a6b4289 improve DBSCAN performance for subsequent trainings
fe1b8515a fix bug in DBSCAN, update doc
febd80911 Fix typo
29225e4ba add DBSCAN outlier detection feature, add supporting documentation
778833f90 Modify comment in new test-strategies to point out their purpose
95327750d Final abs. profit should not be doubled in rpc messages
eae82d022 fix bug with database url during backtesting. comment out example trade db analysis.
95d3009a9 give user ability to analyze live trade dataframe inside custom prediction model. Add documentation to explain new functionality
ae0d6f63f Version bump ccxt to 1.91.55
895ebbfd1 Exclude aarch64 from catboost requirements
d75e0a982 Fix Flake8 errors after flake update
707a4e7c9 types-requests bump pre-commit
f3154a431 Bump jsonschema from 4.7.2 to 4.9.0
3013282db remove non-catboost stuff from schema
97064a9ce Bump pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1
79b650258 Bump urllib3 from 1.26.10 to 1.26.11
ed230dd75 Bump orjson from 3.7.8 to 3.7.11
372be5425 Bump types-requests from 2.28.3 to 2.28.6
b4ded59c6 Bump flake8 from 4.0.1 to 5.0.1
a75fa26ca Bump scipy from 1.8.1 to 1.9.0
7a696f58f Bump ccxt from 1.91.29 to 1.91.52
946d4c7cf fix trailing whitespace for flake8
4e68626bc ensure convolutional window is prepended for frequi consistency
d83010560 *BREAKING CHANGE* remove unnecessary arguments from populate_any_indicators(), accommodate tests
153336d42 move corr_pairlist expansion to after expand_pairlist()
659870312 Use JSON Schema validation for freaAI schema validation
cbb05354a Add install variant for freqai
a4bada3eb Partial exit using average price (#6545)
61693f6c8 fix tests after changing config_example file
e6ebc0443 make single generalized config for freqai. update docs to reflect that.
09e5fb2f5 Removed description header
c2eaa3d2c add image of algorithmic overview to doc
bad15f077 Simplify fetch_positions by using already existing method
dc82675f0 Add Test for liquidation in stop-loss-reached
d046f0cc5 Improve method wording for liquidation price setter
dba7a7257 Use stop_or_liquidation instead of stop_loss
845cecd38 Add stoploss or liquidation property
4da96bc51 Update docs
15752ce3c Rename set_stoploss method to be fully private
ff4cc5d31 Revamp liquidation test to actually make sense
9852733ef Improve tests to align with modified logic
f57ecb186 Simplify adjust_stop test
8711b7d99 Liquidations cannot be rejected.
995be90f9 Liquidation should be a separate exit type
dd8288c09 expose full parameter set for SVM outlier detection. Set default shuffle to false to improve reproducibility
f22b14078 fix backtesting bug, undo move of label stat calc, fix example strat exit logic
08d3ac7ef add keras and conv_width to schema and documentation
59624181b isort BaseRegressionModel imports
c84d54b35 Fix typing issue, avoid using .get() when unnecessary, convert to fstrings
efbd83c56 Small type and typo fixes in freqai_interface
a2a0d35a2 Update missing typehints
f31106dc6 Minor comment fixes
31ddec834 Add missing test to confirm backtesting won't send messages
0adfa4d9e Add tests for dataprovider send-message methods
7bac05466 Add documentation and clarity for send_msg
229e8864b Add send_msg capability to dataprovider
bc760b7eb Simplify small segment in datadrawer
324e54c01 fix possible memory leak associated with Catboost Pool object
3f149c406 fix return type in BaseTensorFlowModel
ad25a4cb5 reduce number of pair_dict lookups, remove coin_first
fb4e8430c isort auto import sorting
e213d0ad5 isolate data_drawer functions from data_kitchen, accommodate tests, add new test
56b17e6f3 allow user to pass test_size = 0 and avoid using eval sets in prediction models
55cf378ec remove leftover breakpoint from test file
897f18a8c ensure proper integer type casting for timestamps. Add check test for backtesting subdaily time periods
7b105532d fix mypy error and add test for principal component analysis
4abc26b58 add test for follow_mode
c9d46a523 finish bringing follow_mode up to date
ab587747f first fix for follower path bug
520ee3f7a Convert freqAI into packages
1885deb63 More docstring changes
70b7a254a Update some areas to use default docstring formatting
88e10f730 add exception for not passing timerange. Remove hard coded arguments for CatboostPredictionModels. Update docs
fff39eff9 fix multitarget bug
95f5218ce Reenable Catboost test (#7118)
f3d46613e move prediction denormalization into datakitchen. remove duplicate associated code. avoid normalization/denormalization for string dtypes.
81c1aa3c1 Update imports in freqAI sample strategies
8a3cffcd1 Remove remaining CustomModel references
62f7606d2 Update tests to new variant
8fa6e8b4b Remove freqAI model bridge in favor of `self.freqai`
c91e23dc5 let user avoid normalizing labels
50d630a15 remove unnecessary comments from data_drawer.py
a1cff377e add record of contribution to data_kitchen.py
c2d6a0e89 add record of contribution to doc and source
3acc86957 improve the dataframe key description, update outdated parts of doc
36dc9be7a Update some docs wording
5559e605b small PR conversation resolutions
40f00196e use cloudpickle in place of pickle. define Paths once in data_drawer.
accc629e3 set separate table sections in doc
98c8a447b add LightGBMPredictionMultiModel
afcb0bec0 clean up obsolete comments, move remove_features_from_df to datakitchen
0b21750e7 Reorder advanced topics
ac0f48491 add freqai logo to top of doc
3205788bc extend doc to include descriptions of the return values from FreqAI to the strategy
8033e0bf2 add counter to backtesting log so users know how many more pairs and how many more models will need to be trained
183dec866 remove ability to backtest open ended timeranges (safer)
e694ea1cf make sure backtesting gets the populated indicators with slimmed down user strat
ca4dd5864 remove superceded function from datakitchen
8f86b0dea *breaking change* simplify user strat by consolidating feature loops into backend
e7337728b add separator in folder name just incase an asset ends in an integer
c9a6dc88a add parameter list/discriptions to doc
6c5e48dd4 dev-dependencies should include freqAI
a99c12626 help windows builds pass freqai tests. Add freqai to README.md
4e5d60fdc match scikit-learn version to hyperopt required version
921a7ef21 add requirements-freqai.txt to builds
286bd0c40 follow string for adding a strat to tests/strategy/strats
c43935e82 create dedicated minimal freqai test strat
88d769d80 comment out problematic catboost test
d43c14667 add more tests for datakitchen functionalities, add regression tests for freqai_interface train/backtest
ac2e8d760 Added description heading to README.md
9c051958a Feat/freqai (#7105)
714d9534b start adding tests
ed0f8b118 Improve FreqAI documentation (#7072)
921f3899f revert pickle reading for historic predictions
41eeb9917 load pickle file for writing
9d184586f fix bug in historic prediction saving
ca2029a46 fix issue when bot restarts with same identifier, does not load prediction history
4141d165f add BaseTensorFlowModel class
ef409dd34 Add ground work for TensorFlow models, add protections from common mistakes
fea63fba1 Fix saving/loading historic predictions
8ce6b1831 start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available.
3fc92b1b2 Create BaseRegression model - designed to reduce code duplication across currently available models.
607455919 Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!)
819cc9c0e Fully align LightGBM with Catboost
58b18770e Fix LightGBM missing argument in predict method
2e1061af6 Fix faulty LightGBM model
d9acdc976 remove excess, increase no model warning clarity
4cac67fd6 Catch infrequent issue associated with grabbing first candle
eda9464d3 Fix docs test
bd3a6ba2f update backtesting to handle new output framework
8ac8d53c3 All LGBMRegressor model parameters are now set in config
4ff0ef735 fix bug returning multiple targets for training
ffb39a502 black formatting on freqai files
106131ff0 Rehaul organization of return values
93e1410ed first step toward cleaning output and enabling multimodel training per pair
6c7d02cb1 expose nu in the SVM outlier detection via svm_nu in config
7dfbd432d fix config saving bug, assign priorities to pairs in scanning, sleep the scanning loop to reduce CPU usage
68bafa951 archive config to the model folder, filter out features before returning dataframe to strategy (to alleviate frequi issues)`
051b99791 reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel
852706cd6 Fix default behavior for expiration_hours
b35c64b6c fix bug in backtest, typo in example strat
3599d18ff fix bug in follow_mode, thanks @blood4rc
6da7a9885 add docstrings to new functions, remove superceded code
f631ae911 add model expiration feature, fix bug in DI return values
61040c9f8 Fix freqAI dockerfile not running freqai code ...
0b0688a91 ensure scanning purges models
121edc3e4 Add freqAI docker file
c5de0c49e first functional scanning commit
4d472a0ea merging datarehaul into scanning branch
c981ad460 Fix missing space
75a248cf4 Fstring freqAI sample strategy, remove duplicate features
d9b79d94e increase candle update flexibility to allow long sequential trainings that may last more than one candle
66800c7a4 ensure newest candles are always appended
f8f25e36e update example config/strat
15d049cff detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check
cab8f517b add lock to datadrawer
4b26b6aae add lock to any historic data access
3c2e314ee Fix bugs
e6c5e737a Fix other bugs
bf19055e5 Update function spelling
2451ed8c8 Quick bug fix
d6b8801f4 fix follower bug
e8c0dcf9f add debug message to timerange
f2762e3b4 fix bug in return_values()
16b4a5b71 rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.
15a971346 catch infinity values when filtering
7d41542f9 process_only_new_candles in examplestrat should be True, thanks @Bloodhunter4rc
fea39254d fix custom_exit (thanks @paranoidandy)
b37c31cc2 fix ta-lib issue with simultaneous method access
4ac6ef297 make defining period intervals more user friendly and flexible
ace951bf7 another queue bug fix for fresh starts
833d25bda Save data after queue reorg
0b0dd8dd8 Force high priority pair to be trained before anything else.
7523ed825 automatically detect maximum required data based on user fed indicators (to avoid NaNs in dataset for rolling indicators), add new config parameter for backtesting to let users increase their startup_candles to accommodate high timeframe indicators, add docs to explain all. Add new feature for automatic indicator duplication according to user defined intervals (exhibited in example strat and configs now).
9b3b08a2b let follower purge old model files
bac4ced38 Ensure follower predictions are persistent and uniquely stored
70adf5564 Automatically detect and change follower data_path to accommodate remote systems
0306f5ca1 Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk
45f4f0f60 ensure follower sends back null arrays in case leader hasnt created a model file yet
29d2f59f1 fix PCA bug
606f18e5c Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader
5b4c649d4 detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly
e22990238 fix bug in previous commit
a20651efd Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).
2f1a2c1cd allow users to store data in custom formats, update spot config to reflect better target horizon to training period ratio
d59eac332 revert a79032b
a79032bf7 fixing bug in training queue
3f7226327 allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
cc6cae47e allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
4eb4753e2 allow subdaily retraining for backtesting
0aa716205 ensure the prediction is reset in the pair_dict after any trade exit, not just custom_exit
fe36b08fc fix key error in example strat
ce365eb9e improve example strat so that it has dynamic buy and sell logic according to original prediction
4eb29c881 Dont reset pair priority if it doesnt successfully train
83dd45372 catch errors occuring on background thread, and make sure to keep the ball rolling. Improve pair retraining queue.
e54614fa2 remove remnants of single threaded version, ensure pair queue priority is checked before retraining
2a4d1e2d6 fix bug in setting new timerange for retraining
7870a86e9 fix live retraining bug
0bf915054 handle key check correctly
c5a16e91f throw user error if user tries to load models but feeds the wrong features (while using PCA)
b8f9c3557 dirty dirty, dont look here (hacking a flag to avoid reloading leverage_tiers in dry/live)
891fb8771 give load_cached_data_for_updating the right flags to avoid redownloading data in dry/live
65fdebab7 let load_pairs_histories load futures candles in live
c080571b7 help futures go dry/live with auto download feature
8a501831d fix the error logic on previous commit
23c30dbc1 add error for user trying to backtest with backtest_period<1
619320501 fix bug for target_mean/std array merging in backtesting
ff531c416 reduce `complexity` inside start_download_data() in an effort to appease flake8
d79983c79 try to pass flake8
7593339c1 small cleanup
b79d4e887 Allow user to go live and start from pretrained models (after a completed backtest) by simply reusing the `identifier` config parameter while dry/live.
7486d9d9e proper validation of freqai config parameters
7ff325860 remove assertions, log error if user has not assigned freqai in config, fix stratify bug
35bed842c cleanup, add clarity to comments and docstrings
58b5abbaa improve multithreaded training queue system
31ae2b306 alleviate FutureWarning in sklearn about ensuring svm model features are passed with identical order
255d35976 add priority metadata to pairs to avoid a sync of train time + train period
059c28542 paying closer attention to managing live retraining on separate thread without affecting prediction of other coins on master thread
b0d2d13eb improve data persistence/mapping for live/dry. This accommodates quick reloads after crash and handles multi-pair cleanly
e1c068ca6 add config asserts, use .get method with default values for optional functionality, move data_cleaning_* to freqai_interface (away from user custom pred model) since it is controlled by config params.
dede12864 set process_only_new_candles to true in example strat
ee3cdd0ff more cleanup
3587bd82e cleanup superceded code
af0cc21af Enable hourly/minute retraining in live/dry. Suppress catboost folder output. Update config + constants + docs to reflect updates.
42d95af82 Aggregated commit. Adding support vector machine for outlier detection, improve user interface to dry/live, better standardization, fix various other bugs
c5ecf9417 move live retraining to separate thread.
1fae6c9ef keep model accessible in memory to avoid loading objects from disk during live/dry
67eb94c69 download-data will now check if freqai is active in config, and if so will also download data for corr_pairlist
89eacf2f4 Retrain model if FreqAI found a pretrained model but user strategy is not passing the expected features (user has changed the features in the strategy but has passed a the same config[freqai][identifier]). Logger warning output to user.
c708dd318 doc update thanks matthias
c81b96079 Fix some typos
db66b82f6 accept open-ended timeranges from user
d1d451c27 auto populate features based on a prepended % in the strategy (remove feature assignment from config). Update doc/constants/example strategy to reflect change
8664e8f9a create a prediction_models folder where basic prediction models can live (similar to optimize/hyperopt-loss. Update resolver/docs/and gitignore to accommodate change
80dcd88ab allow user to run config from anywhere on their system
9e94d2886 add timerange to backtest commnad
e5759d950 fix typo
f4296173e use bash visual in doc
717df891b use bash visual in doc
a8022c104 give beta testers more information in the doc
a7029e35b ensure informative pairs includes any combination of whitelist - corr_pairlist
9b3e5faeb create more flexible whitelist, avoid duplicating whitelist features into corr_pairlist, update docs
22bd5556e add self-retraining functionality for live/dry
178c2014b appease mypy
a4f5811a5 fix flake8 issue in arguments.py
aae233bd6 try passing the check tests
f653ace24 another attempt at fixing datalength bug
b08c0888b add USERPATH_FREQAIMODELS, remove return values from @abstract methods
b03c7b514 optional style for interfacing freqai with backtesting
e9a7b68bc revert constants.py and add changes
302021809 fix bug on backtest timerange
00ff0c9b9 ensure user defined timerange truncates final backtest so that we arent mismatching data lengths upon return to strategy. Rename DataHandler class to FreqaiDataKitchen
66715c5ba update doc
def71a0af auto build full_timerange and self manage training_timerange
764f9449b fix logger, debug some flake8 appeasements
29c2d1d18 use logger in favor of print
99f7e44c3 flake8 passing, use pathlib in lieu of os.path to accommodate windows/mac OS
2600ba4e7 remove unused remnants
630d20154 remove trained_stake
b40f8f88a cleaning and bug fixing
fc837c4da add freqao backend machinery, user interface, documentation