Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot the seasonality and ACF/PACF #1009

Closed
6 tasks done
trentmc opened this issue May 9, 2024 · 3 comments
Closed
6 tasks done
Assignees
Labels
Type: Enhancement New feature or request

Comments

@trentmc
Copy link
Member

trentmc commented May 9, 2024

Background / motivation

Parent epic: #1006 "Explore SARIMAX modeling"

Top-level goal: Help user answer which of which transform to use {0, B0, B1, B2}, to (a) achieve stationarity; and (b) see how well the signal can decompose into trend / seasonal / residual.

Specific transforms:

  • 0: differencing = 0
  • B0: Box-Cox transform + differencing=0
  • B1: Box-Cox transform + differencing=1
  • B2: Box-Cox transform + differencing=1

Sub-goals: We want it easy to see these:

  • decomposition into trend / seasonal / residual
  • relative strengths of trend vs seasonality vs resid
  • ACF & PACF. That is, autocorrelation function & partial autocorrelation function, for insight into best AR coefficient etc
  • ADF for each transform {0, B0, B1, B2}. Insight: the lowest # that has ADF < 0.05 is the best
  • Then, make it easy for the user to select a transform and see the plots ^ for that given transform

Best ref: jahnvisikligar Master's thesis https://github.com/jahnvisikligar/Masters-Thesis/blob/main/Final_Thesis_code.ipynb

TODOs / DoD

  • Add real crypto data, ready for use by unit tests PR #1016
  • Unit test with plots for each sub-component:
    • decomposition into trend / seasonal / residual PR #1016
    • relative strengths of trend vs seasonality vs resid PR #1025
    • ACF & PACF PR #1026
    • ADF for each transform {0, B0, B1, B2}. (Calc'd it as part of above line. Plots are being done in another issue).

Main prototype: with everything

For a given choice of differencing {0,B0,B1,B2}, show

  • 5 decomposition plots: energy per signal, plus 4 signals: (i) observed feed (ii) "trend" (iii) "seasonal", "residual".
  • autocorrelation (ACF) & partial autocorrelation (PACF) plots

Scan 11  May 2024, 10 55 page 2

Other Prototypes & Examples

Prototype: decomposition into trend / seasonality / residual

329284278-d9bbecce-9f5f-415f-be6e-c8da6d5d04b5

Prototype: For differencing=0,B0,B1,B2, show 5 decomposition plots

Scan 11  May 2024, 10 55 page 1

Prototype: For differencing=0,B0,B1,B2, show ACF & PACF

1011

Example 0: neptune.ai "select model"

Ref: neptune.ai select-model post

from statsmodels.tsa.seasonal import seasonal_decompose
result = seasonal_decompose(co2_data)
result.plot()

example0

Example 1a: Sikligar thesis, Differencing=0

Ref: Sikligar thesis & code

example1a-differencing0

Example 1b: Sikligar thesis, Differencing=1

Ref: Sikligar thesis & code

example1b-differencing1

Example 2: neptune.ai catfish sales

result = seasonal_decompose(lim_catfish_sales,model='additive')
fig = result.plot()

Ref: neptune.ai catfish, in this neptune.ai post/py code.

example2

@trentmc trentmc added the Type: Enhancement New feature or request label May 9, 2024
@trentmc trentmc self-assigned this May 9, 2024
@trentmc trentmc changed the title [UX, sim] Plot a seasonal decomposition of the feed [UX, sim] SARIMAX: Plot a seasonal decomposition of the feed May 9, 2024
@trentmc trentmc changed the title [UX, sim] SARIMAX: Plot a seasonal decomposition of the feed [UX, sim] SARIMAX: Plot seasonal decomposition of the feed May 9, 2024
@trentmc trentmc changed the title [UX, sim] SARIMAX: Plot seasonal decomposition of the feed [UX, sim] SARIMAX: Plot seasonal decomposition of the feed, for differencing=0,1,2 May 9, 2024
@trentmc trentmc changed the title [UX, sim] SARIMAX: Plot seasonal decomposition of the feed, for differencing=0,1,2 [UX, sim] SARIMAX: Plot seasonal decomposition of the feed May 9, 2024
@trentmc
Copy link
Member Author

trentmc commented May 10, 2024

Seasonal decomposition of Binance BTC/USDT close price Feb 1, 2024 - Mar 8, 2024

Screenshot 2024-05-10 at 11 22 22

trentmc added a commit that referenced this issue May 10, 2024
…lots (#1016)

Towards #1009 Seasonal decomposition

- Add real crypto data: 10000 five-min epochs for Binance BTC/USDT & ETH/USDT. In `./pdr_backend/lake/test/merged_ohlcv_df_BTC-ETH_2024-02-01_to_2024-03-08.csv`
- add `aimodel/seasonal.py` which has:
  - `pdr_seasonal_decompose()` -   Wraps statsmodels' seasonal_decompose() with predictoor-specific inputs 
  - `SeasonalPlotdata` - Simple class to manage many inputs going to plot_seasonal."
  - `plot_seasonal(d: SeasonalPlotdata)` - Plot seasonal decomposition of the feed, via 4 figures
- add `aimodel/test/test_seasonal.py` which has:
  - via the methods added: pull in real BTC data, decompose it, and plot it
@trentmc trentmc reopened this May 10, 2024
@trentmc
Copy link
Member Author

trentmc commented May 10, 2024

Part 1 done here: #1016

idiom-bytes added a commit that referenced this issue May 10, 2024
* Fix #889: [Pdr bot] Add approach 3: like approach 2, but 1ss (larger-smaller) (#891)

Fix #889

* Fix #892: [Bug] UnixTimeS & UnixTimeMS.now() ignore time zone (PR #893)

* Fix #892

* Bump pandas from 2.2.1 to 2.2.2 (#888)

Bumps [pandas](https://github.com/pandas-dev/pandas) from 2.2.1 to 2.2.2.
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Commits](pandas-dev/pandas@v2.2.1...v2.2.2)

---
updated-dependencies:
- dependency-name: pandas
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump web3 from 6.16.0 to 6.17.1 (#897)

Bumps [web3](https://github.com/ethereum/web3.py) from 6.16.0 to 6.17.1.
- [Changelog](https://github.com/ethereum/web3.py/blob/v6.17.1/docs/releases.rst)
- [Commits](ethereum/web3.py@v6.16.0...v6.17.1)

---
updated-dependencies:
- dependency-name: web3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump ccxt from 4.2.87 to 4.2.98 (#898)

Bumps [ccxt](https://github.com/ccxt/ccxt) from 4.2.87 to 4.2.98.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](ccxt/ccxt@4.2.87...4.2.98)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump scikit-learn from 1.4.1.post1 to 1.4.2 (#885)

Bumps [scikit-learn](https://github.com/scikit-learn/scikit-learn) from 1.4.1.post1 to 1.4.2.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@1.4.1.post1...1.4.2)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Fix #899 [Bug] ImportError: cannot import name 'ContractName' from 'eth_typing'

* undo last commit, it should go through a PR

* Fix #899 [Bug] ImportError: cannot import name 'ContractName' from 'eth_typing' (PR #900)

Fix #899

* Towards #879: Refactors in 'exchange/', and timezone bug fix (PR #896)

#879 is [Bug] dydx api ignores 'fromISO' time-date spec

Towards that, this PR makes it easier to work with dydx separately from the rest.

- rename `safe_fetch_ohlcv*()` -> `fetch_ohlcv*()`. Why: less verbose, same effect
- move `fetch_ohlcv_ccxt()` into its own module of the same name
- move `fetch_ohlcv_dydx()` ""
- timezone-related bug fix in `fetch_ohlcv_dydx()`
  - when it got the timestamp from the API call, it didn't set it as UTC. 
  - so, I first created a new routine `time_types.py::from_iso_timestr(iso_timestr: str)`, plus unit test
  - where the new routine tacks on timezone info
  - then used that new routine in `fetch_ohlcv_dydx()`

* Fix #901: [Warning] datetime.utcfromtimestamp() is deprecated, scheduled for removal (PR #902)

Fix #901

* #701 - Predictoor Agent with PredSubmitterManager (#877)

* #701 - Update PredictoorSS to include prediction feeds and the data sources for training these feeds (#828)

* Trimming apart old implementation

* Make predict_feed an array

* Rename to predict_feeds

* MultiFeedMixin

* Update key to feeds

* Update structure

* Update verify_feed_dependencies for multiple feeds

* Create PredictFeedMixin

* PredictFeedMixin test

* parse_feed_obj func and fixes

* test_parse_feed_obj and multi predict feeds

* pred_submitter_mgr property

* Convert PredictoorSS to PredictFeedMixin

* Formatting

* Update predictoor_ss_test_dict

* Add prepare_stakes function

* Break on cutoff

* Update check balances to check contracts

* Load pred_submitter_mgr_addr

* Update take_step and remove unused funcs

* Add PredictFeed and PredictFeeds classes

* add feeds property

* Update predictoor_ss_test_dict

* Update predictoor ss tests

* convert to str

* Add feeds_str and fixes

* Update keys

* Black

* Fixes

* Black

* Enforce typing and fixes

* Type fixes

* Type fixes

* Fix & update ppss tests

* Formatting

* Feeds structure

* Remove removed functions tests

* enable assert attributes

* Use list

* Set submitter addr

* Update sim engine

* Update sim engine test

* Update feed list

* Update AI model factory tests

* Use PredictFeeds

* Return PredictFeeds

* Use argfeed

* BLack

* PredictFeed update to have 1 -> N

* Fix test

* feeds_list func for comparison

* Single predict feed

* Enforce types

* Usel ist

* Set self.feeds and remove extra properties

* Call to_list

* Fix tests for single predict feed

* Black

* Update tests for single pred feed and structure

* Add feeds property back

* Update tests to support new structure and single pred feed

* Fix check network tests

* Fix mock args

* Black

* Replace all, arg fix

* Fix args

* Fix tests

* Update dict

* Update dict

* Black

* create_xy on given feeds

* Add filter_feeds_from_candidates to mixin

* Fallback to ss.feeds

* Formatting

* Linter

* Linter

* Revert "#701 - Update PredictoorSS to include prediction feeds and the data s…" (#878)

This reverts commit ac6899b.

* #701 - Multi Feed Predictions and PredSubmitter in Predictoor Agent (#855)

* Trimming apart old implementation

* Make predict_feed an array

* Rename to predict_feeds

* MultiFeedMixin

* Update key to feeds

* Update structure

* Update verify_feed_dependencies for multiple feeds

* Create PredictFeedMixin

* PredictFeedMixin test

* parse_feed_obj func and fixes

* test_parse_feed_obj and multi predict feeds

* pred_submitter_mgr property

* Convert PredictoorSS to PredictFeedMixin

* Formatting

* Update predictoor_ss_test_dict

* Add prepare_stakes function

* Break on cutoff

* Update check balances to check contracts

* Load pred_submitter_mgr_addr

* Update take_step and remove unused funcs

* Add PredictFeed and PredictFeeds classes

* add feeds property

* Update predictoor_ss_test_dict

* Update predictoor ss tests

* convert to str

* Add feeds_str and fixes

* Update keys

* Black

* Fixes

* Black

* Enforce typing and fixes

* Type fixes

* Type fixes

* Fix & update ppss tests

* Formatting

* Feeds structure

* Remove removed functions tests

* enable assert attributes

* Use list

* Set submitter addr

* Update sim engine

* Update sim engine test

* Update feed list

* Update AI model factory tests

* Use PredictFeeds

* Return PredictFeeds

* Use argfeed

* BLack

* PredictFeed update to have 1 -> N

* Fix test

* feeds_list func for comparison

* Single predict feed

* Enforce types

* Usel ist

* Set self.feeds and remove extra properties

* Call to_list

* Fix tests for single predict feed

* Black

* Update tests for single pred feed and structure

* Add feeds property back

* Update tests to support new structure and single pred feed

* Fix check network tests

* Fix mock args

* Black

* Replace all, arg fix

* Fix args

* Fix tests

* Update dict

* Update dict

* Black

* create_xy on given feeds

* Add filter_feeds_from_candidates to mixin

* Fallback to ss.feeds

* Formatting

* Linter

* Linter

* Add missing import

* Add get_predict_feed func

* Add PredictionSlotsData class

* Multi slot support

* Fixes, ready to predict

* Argpair attributes

* Payout function and todos

* Todo sim engine

* Add minimum_timeframe_seconds func

* Add get_min_epoch_s_left to predictoor agent

* Fix balance check test

* Fix empty init test

* Add mock functions

* Fix test_predictoor_agent_calc_stakes2_1feed test

* Move pred_submitter_mgr to ganache conftest

* Pass in pred_submitter_mgr

* Pass in pred_submitter_mgr between mocks

* pred_submitter_mgr test dict

* Fix test_predictoor_agent_calc_stakes2_2feeds test

* Make optional

* Fix tests

* Chain id attr

* Comment out sanity checks

* Update wait_for_transaction_receipt mock

* Black formatting

* Add pair_str

* Update sim engine for single predict feed

* Update sim engine tests

* Update sim engine to include feed in MultisimEngine initialization

* Formatting

* Update sim engine to include feed in MultisimEngine initialization

* Formatting

* Resolve linter issues

* Implementing unique epoch

* Add min_epoch_seconds func

* Remove removed vars

* Black Formatting

* Linter

* Final touchups

* Rename to get

* Todo

* Fix system test predictoor

* Formatting

* fix mypy errors

* Transfer automatically

* Compile

* Approve tokens

* Fix token transfer issue in PredSubmitterMgr.sol

* Compiled contract

* Update test

* Black

* Add info

* Fix .exchange

* Fix order

* make it more explicit

* Update predictoor docs

* Update cli help

* Update

* Update docs

* Remove unused variable

* Use logger.info

* Test PredictFeed and parse_feed_obj

* Add tests for PredictFeeds

* Formatting

* Remove cutoff seconds

* Remove cut_off

* Update mock func name

* Formatting

* Resolve linter issues and bugs

* Update todo comment

* Linter

* Remove types

* Formatting

* Improve docs

* Quotes

* Remove 1s

* Add log message

* Fix typo

* Fix assert

* Fix typo

* Refactor aimodel_data_factory.py to handle feeds and x_dim_len dynamically

* Fix #907: Opportunity: solve #901 depr'cn warning, while working in older py (PR #903)

Fix #907

* Fix #912: [PPSS, Aimodel] predictoor_ss.aimodel_ss.input_feeds is … (#913)

Fix #912: [PPSS, Aimodel] predictoor_ss.aimodel_ss.input_feeds is redundant with new predictoor_ss.feeds.train_on

Includes refactoring to what is now PredictTrainFeedset etc.

* Fix #917: multisim sweeping in ppss.yaml (PR #920)

Co-authored with @calina-c

* Bump web3 from 6.17.1 to 6.17.2 (#925)

Bumps [web3](https://github.com/ethereum/web3.py) from 6.17.1 to 6.17.2.
- [Changelog](https://github.com/ethereum/web3.py/blob/v6.17.2/docs/releases.rst)
- [Commits](ethereum/web3.py@v6.17.1...v6.17.2)

---
updated-dependencies:
- dependency-name: web3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump ccxt from 4.2.98 to 4.3.4 (#924)

Bumps [ccxt](https://github.com/ccxt/ccxt) from 4.2.98 to 4.3.4.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](ccxt/ccxt@4.2.98...4.3.4)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump polars from 0.20.17 to 0.20.22 (#923)

Bumps [polars](https://github.com/pola-rs/polars) from 0.20.17 to 0.20.22.
- [Release notes](https://github.com/pola-rs/polars/releases)
- [Commits](pola-rs/polars@py-0.20.17...py-0.20.22)

---
updated-dependencies:
- dependency-name: polars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump streamlit from 1.32.2 to 1.33.0 (#922)

Bumps [streamlit](https://github.com/streamlit/streamlit) from 1.32.2 to 1.33.0.
- [Release notes](https://github.com/streamlit/streamlit/releases)
- [Commits](streamlit/streamlit@1.32.2...1.33.0)

---
updated-dependencies:
- dependency-name: streamlit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump plotly from 5.20.0 to 5.21.0 (#921)

Bumps [plotly](https://github.com/plotly/plotly.py) from 5.20.0 to 5.21.0.
- [Release notes](https://github.com/plotly/plotly.py/releases)
- [Changelog](https://github.com/plotly/plotly.py/blob/master/CHANGELOG.md)
- [Commits](plotly/plotly.py@v5.20.0...v5.21.0)

---
updated-dependencies:
- dependency-name: plotly
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* For #927, in predictoor.md: streamline how t plot

* For #927, in trader.md, Streamline how to plot in simulation

* Fix #929 [Sim] predictoor.md & trader.md say "Warning: simulation is temporarily disabled", but sim works

* Undo change to ppss.yaml, that was accidentally put into previous commit.

* Update vps.md

Add nohup

* Update vps.md: update nohup instructions

* #918 - Remove ganache dependency from predictoor tests (#934)

* Use a mock for pred submitter manager

* Remove ganache dependency from predictoor tests

* Formatting

* #930 - Remove CLI approach arg from predictoor (#931)

* Set predictoor parser to _ArgParser_PPSS_NETWORK

* Remove approach

* Set approach in system test

* Remove approach from command

* Remove approach text check

* Fix #933 [Bug][Pdr Submitter Mgr] Predictions are submitted before <= 60s (#935)

* Fix epoch_s_thr

* Revert

* Smart prediction submisison

* Fix epoch_s_thr and add epoch cache in predictoor_agent.py

* type annotation for "epoch_cache"

* Logging and clean code

* Fix system tests

* Add a test for test_calc_stakes_across_feeds

* Mock calc_stakes_2ss_model

* Improve logging

* Divide into lines

* Approve only cand feeds

* Improve logs

* Divide lines

* Fix type

* Formatting

* Split lines

* Fix typo

* Convert info to debug

This should be at debug level and used to see what value are being submitted to the smart contract

* Update predictoor.md (#926)

Co-authored-by: Idiom <69865342+idiom-bytes@users.noreply.github.com>

* Bump ccxt from 4.3.4 to 4.3.11 (#947)

Bumps [ccxt](https://github.com/ccxt/ccxt) from 4.3.4 to 4.3.11.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](ccxt/ccxt@4.3.4...4.3.11)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump statsmodels from 0.14.1 to 0.14.2 (#946)

Bumps [statsmodels](https://github.com/statsmodels/statsmodels) from 0.14.1 to 0.14.2.
- [Release notes](https://github.com/statsmodels/statsmodels/releases)
- [Changelog](https://github.com/statsmodels/statsmodels/blob/main/CHANGES.md)
- [Commits](statsmodels/statsmodels@v0.14.1...v0.14.2)

---
updated-dependencies:
- dependency-name: statsmodels
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump eth-keys from 0.5.0 to 0.5.1 (#945)

Bumps [eth-keys](https://github.com/ethereum/eth-keys) from 0.5.0 to 0.5.1.
- [Changelog](https://github.com/ethereum/eth-keys/blob/main/CHANGELOG.rst)
- [Commits](ethereum/eth-keys@v0.5.0...v0.5.1)

---
updated-dependencies:
- dependency-name: eth-keys
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump freezegun from 1.4.0 to 1.5.0 (#944)

Bumps [freezegun](https://github.com/spulec/freezegun) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/spulec/freezegun/releases)
- [Changelog](https://github.com/spulec/freezegun/blob/master/CHANGELOG)
- [Commits](spulec/freezegun@1.4.0...1.5.0)

---
updated-dependencies:
- dependency-name: freezegun
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump polars from 0.20.22 to 0.20.23 (#943)

Bumps [polars](https://github.com/pola-rs/polars) from 0.20.22 to 0.20.23.
- [Release notes](https://github.com/pola-rs/polars/releases)
- [Commits](pola-rs/polars@py-0.20.22...py-0.20.23)

---
updated-dependencies:
- dependency-name: polars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Fix #879 - Include toISO in dydx api requests `fetch_ohlcv_dydx` (#951)

* Add TIMEFRAME_TO_DYDX_RESOLUTION_SECONDS into constants

* Update fetch_ohlcv_dydx.py to include _time_delta_from_timeframe function and toISO arg

* Update test_fetch_ohlcv_dydx.py to fix fromISO issue and update tests

* Add test cases for _time_delta_from_timeframe function

* Fix #942 - Move pred_submitter_mgr under bot_only (#950)

* Move pred_submitter_mgr under bot_only

* Formatting

* Convert streamlit dashboard to Dash (#938)

* Convert streamlit dashboard to Dash. Main cases.

* Scaffold for lineplot response with var imps selection.

* Allow running with custom multi_id and ports.

* Adds slider for state selection.

* Reorganise files to separate layout, callbacks and utils.

* Adds argparse and readme updates.

* Linter fixes.

* Draft add 1-var interactivity.

* Adds deselecting.

* style update to make plots take up the entire screen width

* update justify content attribute

* add width to graph components

* removed todo comment

---------

Co-authored-by: trentmc <gtrent@gmail.com>
Co-authored-by: Norbert <katunanorbert@gmail.com>

* Fix #963: [Sim, log] Main summary log line takes up >1 line, even in ultrawide stretched terminal (#964)

Fix #963

* Fix #890: [Aimodel] Add Jaime's techniques: balance class weights in logistic regression, calibrate with isotonic (#959)

Fix #890

* Fixes to sim plotting (#962)

* Adds waiting around unpickling error
* Adjust colours in 3-line plots.
* Adjust labels for slider.

* Fix #965: add log loss metric to logs (#966)

* Fix #965

* Fix #970 [CLI, sim plots] READMEs should say to open the url in the browser

* Fix #977: [CLI] 'pdr view accounts' is too verbose (#978)

Fix #977

* #971 - Organize payout slots and contract addresses based on shared slots (#972)

Bug fix, no breaking changes. 
Updated payout function to organize payout slots and contract addresses based on shared slots before passing them to the smart contract.

* Change sim plots entrypoint, add tests (#979)

* Adds new entrypoint for dash.
* Add tests for dashboard.
* Update md files.
* fix multiple plots on the same row are not shrinking
---------

Co-authored-by: Norbert <katunanorbert@gmail.com>

* Issue 804 - Thread multisim (#919)

* Make multisim threaded.

* Feature #981 - Integrate dydx exchange and sim (#987)

Adds DydxExchange class.
Implements DydxExchange into exchange_mgr.py and other tiny compatibility fixes in fetch_ohlcv_dydx.py

* Adds sim line logger. (#988)

* Adds sim line logger.

* Remove extra command line in fund_accounts parser. (#986)

* Arrange logs and inits. (#985)

* Plot log loss. (#984)

* Bump mypy from 1.9.0 to 1.10.0 (#994)

Bumps [mypy](https://github.com/python/mypy) from 1.9.0 to 1.10.0.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](python/mypy@1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump eth-typing from 4.1.0 to 4.2.2 (#995)

Bumps [eth-typing](https://github.com/ethereum/eth-typing) from 4.1.0 to 4.2.2.
- [Changelog](https://github.com/ethereum/eth-typing/blob/main/docs/release_notes.rst)
- [Commits](ethereum/eth-typing@v4.1.0...v4.2.2)

---
updated-dependencies:
- dependency-name: eth-typing
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump dash from 2.16.1 to 2.17.0 (#996)

Bumps [dash](https://github.com/plotly/dash) from 2.16.1 to 2.17.0.
- [Release notes](https://github.com/plotly/dash/releases)
- [Changelog](https://github.com/plotly/dash/blob/dev/CHANGELOG.md)
- [Commits](plotly/dash@v2.16.1...v2.17.0)

---
updated-dependencies:
- dependency-name: dash
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Bump plotly from 5.21.0 to 5.22.0 (#998)

Bumps [plotly](https://github.com/plotly/plotly.py) from 5.21.0 to 5.22.0.
- [Release notes](https://github.com/plotly/plotly.py/releases)
- [Changelog](https://github.com/plotly/plotly.py/blob/master/CHANGELOG.md)
- [Commits](plotly/plotly.py@v5.21.0...v5.22.0)

---
updated-dependencies:
- dependency-name: plotly
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>

* Fix #990: Add predictoor log line. (#999)

* Process dates with natural language processing. (#1004)

* Towards #1009: Seasonal decomposition: Part 1/2: backend, unit test plots (#1016)

Towards #1009 Seasonal decomposition

- Add real crypto data: 10000 five-min epochs for Binance BTC/USDT & ETH/USDT. In `./pdr_backend/lake/test/merged_ohlcv_df_BTC-ETH_2024-02-01_to_2024-03-08.csv`
- add `aimodel/seasonal.py` which has:
  - `pdr_seasonal_decompose()` -   Wraps statsmodels' seasonal_decompose() with predictoor-specific inputs 
  - `SeasonalPlotdata` - Simple class to manage many inputs going to plot_seasonal."
  - `plot_seasonal(d: SeasonalPlotdata)` - Plot seasonal decomposition of the feed, via 4 figures
- add `aimodel/test/test_seasonal.py` which has:
  - via the methods added: pull in real BTC data, decompose it, and plot it

* Fixing merge

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Trent McConaghy <gtrent@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Călina Cenan <calina@cenan.net>
Co-authored-by: trizin <25263018+trizin@users.noreply.github.com>
Co-authored-by: Robin Lehmann <w1kke@users.noreply.github.com>
Co-authored-by: Norbert <37236152+KatunaNorbert@users.noreply.github.com>
Co-authored-by: Norbert <katunanorbert@gmail.com>
@trentmc trentmc changed the title [UX, sim] SARIMAX: Plot seasonal decomposition of the feed [Sim, Analytics] SARIMAX: Plot seasonal decomposition of the feed May 11, 2024
@trentmc trentmc changed the title [Sim, Analytics] SARIMAX: Plot seasonal decomposition of the feed [Sim plots] Plot seasonal decomposition of the feed May 11, 2024
@trentmc trentmc changed the title [Sim plots] Plot seasonal decomposition of the feed [Sim plots] For {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF May 11, 2024
@trentmc trentmc changed the title [Sim plots] For {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF [Sim plots] ARIMA-style signal analysis: for {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF May 11, 2024
@trentmc trentmc changed the title [Sim plots] ARIMA-style signal analysis: for {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF [Sim plots] ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF May 11, 2024
@trentmc trentmc changed the title [Sim plots] ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot seasonality and ACF/PACF [Sim plots] ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot the seasonality and ACF/PACF May 11, 2024
trentmc added a commit that referenced this issue May 11, 2024
trentmc added a commit that referenced this issue May 11, 2024
@trentmc trentmc changed the title [Sim plots] ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot the seasonality and ACF/PACF ARIMA-style feed analysis: for {BoxCox=T/F, diff=0,1,2}, plot the seasonality and ACF/PACF May 13, 2024
@trentmc
Copy link
Member Author

trentmc commented May 13, 2024

Results for data: merged_ohlcv_df_BTC-ETH_2024-02-01_to_2024-03-08.csv, using BTC/USDT c 5m.

BoxCox=F, diff=0

Screenshot 2024-05-13 at 09 11 52

BoxCox=T, diff=0

Screenshot 2024-05-11 at 21 31 33

BoxCox=T, diff=1

Screenshot 2024-05-13 at 09 12 09

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant