Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Tracking tangle abstraction rework (#1049)
* Add InputSource and some other glue * Restructuring tangle abstraction and fixing errors (#1050) * Restructuring and fix errors * wip * Add `MilestoneRange` (#1051) * Add `MilestoneRange` * PR comments * `*Message` types (#1052) * `*Message` types * rename * fix * fmt * Impl `milestone_stream` and `cone_stream` (#1053) * Impl `milestone_stream` and `cone_stream` * Make `ledger_updates` async again * Clippy and fmt * use `RangeBounds` * fix `Send` * wip * Remove `unspent_outputs` * remove memory stuff * Impl ledger queries for MongoDb input source (#1056) * Impl `milestone_stream` and `cone_stream` * Make `ledger_updates` async again * Clippy and fmt * use `RangeBounds` * Impl ledger updates query * fix `Send` * Impl unspent outputs query for mongodb * Format * Small refactor * Nit * Clippy + Format * Collect into HashMap directly * Return ledger updates as stream Co-authored-by: Jochen Görtler <jochen.goertler@iota.org> Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> * Add impls for INX and in-memory data (#1054) * Add impls for INX and in-memory data * Update src/tangle/sources/inx.rs Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> * Fix merge issues * clippy Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> * Implement incremental in-memory analytics (#1061) * Wip * fmt * wip * wip * Add `BaseTokenActivityAnalytics` analytics * wip * wip * fmt * Remove `inputs` from `handle_block` * restructure * Add ledger outputs * Implement daily active addresses * address some pr comments * fix base token * documentation * Add moar analytics (#1066) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use `BlockData` struct in `BlockAnalytics` trait (#1073) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use BlockData type in BlockAnalytics trait * Review comments * Initial implementation of fill-analytics (and inx too 🤷) (#1070) * Initial implementation of fill-analytics (and inx too 🤷) * PR suggestions * Handle missing outputs when getting analytics * impl protocol params analytic * More merges * impl milestone size * impl more analytics * more impls * impl milestone stream for mongo * impl mongo ledger updates source * Cleanup and feature gating * implement in-memory unlock condition analytics (#1079) * implement in-memory unlock condition analytic * actually use new impl * Small fix * reduce redundancy * Cleanup and unpublify analytics * clippy + docs --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * `Vec`-based `LedgerUpdateStore` (#1091) * `Vec`-based `LedgerUpdateStore` * revert error * Cache ledger updates (#1093) Cache ledger updates in the milestone * implement foundry activity analytics and cleanup output activity impl (#1090) * Remove old analytics queries (#1094) * Implement analytics using dyn Traits (#1095) * Implement analytics using dyn Traits * Simplify Measurement * Reduce reuse recycle * Add analytics context * Clippy * More cleanup. Fix usage of inner fields * AddFields helper trait to invert call * Rename `OuputAmount` to `TokenAmount` (#1096) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix ledger state for new analytics * fox docs * Fix analytic tests and clippy * Fix in-memory analytics (#1099) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more * Implement analytics using dyn Traits * Add analytics context * wip * wip * errors * Only calculate analytics for included transactions * timing * Better database connection * fix ci * fmt * remove dependency on INX * fmt (comment) * refactor num tasks * fix overflow? * fix again... * fix measurement names * fix feature * remove feature gate --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix unit test * clippy * Cleanup tangle impls * Add INX backend for fill analytics (#1115) * Add INX backend for fill analytics * Add --help docs for fill-analytics * Simplify * Lifetimes * Less moves * Small refactor * Set input source default * Fixes everything * Fix live INX analytics * Cleanup --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Cleanup error types --------- Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> Co-authored-by: /alex/ <alexander.schmidt@iota.org> * Cleanup * Update src/analytics/ledger/base_token.rs Co-authored-by: /alex/ <alexander.schmidt@iota.org> --------- Co-authored-by: /alex/ <alexander.schmidt@iota.org> Co-authored-by: Alexandcoats <alexandcoats@gmail.com> * Allow more CLI variables in ENV (#1124) Improve CLI env availability * Add token distribution analytic (#1123) * Starting impl * Integrate token distribution into address balance analytic * Ensure complete distribution * Fix comment * Small refactor * Add dashboard * cleanup * Variable buckets * Little more cleanups --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * wrap values in analytics (#1121) * Wip * fmt * wip * wip * Add `BaseTokenActivityAnalytics` analytics * wip * wip * fmt * Remove `inputs` from `handle_block` * restructure * Add ledger outputs * Implement daily active addresses * address some pr comments * fix base token * documentation * Add moar analytics (#1066) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use `BlockData` struct in `BlockAnalytics` trait (#1073) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use BlockData type in BlockAnalytics trait * Review comments * Initial implementation of fill-analytics (and inx too 🤷) (#1070) * Initial implementation of fill-analytics (and inx too 🤷) * PR suggestions * Handle missing outputs when getting analytics * impl protocol params analytic * More merges * impl milestone size * impl more analytics * more impls * impl milestone stream for mongo * impl mongo ledger updates source * Cleanup and feature gating * implement in-memory unlock condition analytics (#1079) * implement in-memory unlock condition analytic * actually use new impl * Small fix * reduce redundancy * Cleanup and unpublify analytics * clippy + docs --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * `Vec`-based `LedgerUpdateStore` (#1091) * `Vec`-based `LedgerUpdateStore` * revert error * Cache ledger updates (#1093) Cache ledger updates in the milestone * implement foundry activity analytics and cleanup output activity impl (#1090) * Remove old analytics queries (#1094) * Implement analytics using dyn Traits (#1095) * Implement analytics using dyn Traits * Simplify Measurement * Reduce reuse recycle * Add analytics context * Clippy * More cleanup. Fix usage of inner fields * AddFields helper trait to invert call * Rename `OuputAmount` to `TokenAmount` (#1096) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix ledger state for new analytics * fox docs * Fix analytic tests and clippy * Fix in-memory analytics (#1099) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more * Implement analytics using dyn Traits * Add analytics context * wip * wip * errors * Only calculate analytics for included transactions * timing * Better database connection * fix ci * fmt * remove dependency on INX * fmt (comment) * refactor num tasks * fix overflow? * fix again... * fix measurement names * fix feature * remove feature gate --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix unit test * clippy * Cleanup tangle impls * Add INX backend for fill analytics (#1115) * Add INX backend for fill analytics * Add --help docs for fill-analytics * Simplify * Lifetimes * Less moves * Small refactor * Set input source default * Fixes everything * Fix live INX analytics * Cleanup --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Cleanup error types --------- Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> Co-authored-by: /alex/ <alexander.schmidt@iota.org> * Cleanup * impl wrapping analytics --------- Co-authored-by: Jochen Görtler <jochen.goertler@iota.org> Co-authored-by: /alex/ <alexander.schmidt@iota.org> Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> * Create base token test case (#1122) * Create base token test case * Weird comment * cleanup * Add transaction size distribution analytic (#1117) * Wip * fmt * wip * wip * Add `BaseTokenActivityAnalytics` analytics * wip * wip * fmt * Remove `inputs` from `handle_block` * restructure * Add ledger outputs * Implement daily active addresses * address some pr comments * fix base token * documentation * Add moar analytics (#1066) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use `BlockData` struct in `BlockAnalytics` trait (#1073) * Add ledger size analytic * Remove context * Add output activity analytic (part 1) * Filter asset * Handle implicit ids * Add output activity analytic (part 2) * Document unwraps * Impl TryFrom OutputId for Nft and Alias ids * Cleanup and docs * Clippy * Add milestone size analytic * Fix docs * Slice instead of Vec * Fix CI * Fix CI (this time?) * usize instead of u64 * format * Use BlockData type in BlockAnalytics trait * Review comments * Initial implementation of fill-analytics (and inx too 🤷) (#1070) * Initial implementation of fill-analytics (and inx too 🤷) * PR suggestions * Handle missing outputs when getting analytics * impl protocol params analytic * More merges * impl milestone size * impl more analytics * more impls * impl milestone stream for mongo * impl mongo ledger updates source * Cleanup and feature gating * implement in-memory unlock condition analytics (#1079) * implement in-memory unlock condition analytic * actually use new impl * Small fix * reduce redundancy * Cleanup and unpublify analytics * clippy + docs --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * `Vec`-based `LedgerUpdateStore` (#1091) * `Vec`-based `LedgerUpdateStore` * revert error * Cache ledger updates (#1093) Cache ledger updates in the milestone * implement foundry activity analytics and cleanup output activity impl (#1090) * Remove old analytics queries (#1094) * Implement analytics using dyn Traits (#1095) * Implement analytics using dyn Traits * Simplify Measurement * Reduce reuse recycle * Add analytics context * Clippy * More cleanup. Fix usage of inner fields * AddFields helper trait to invert call * Rename `OuputAmount` to `TokenAmount` (#1096) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix ledger state for new analytics * fox docs * Fix analytic tests and clippy * Fix in-memory analytics (#1099) * Implement analytics using dyn Traits * Add analytics context * Rename `OuputAmount` to `TokenAmount` * Rename `value` to `amount` * fmt * rename more * Implement analytics using dyn Traits * Add analytics context * wip * wip * errors * Only calculate analytics for included transactions * timing * Better database connection * fix ci * fmt * remove dependency on INX * fmt (comment) * refactor num tasks * fix overflow? * fix again... * fix measurement names * fix feature * remove feature gate --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Fix unit test * clippy * Cleanup tangle impls * Add INX backend for fill analytics (#1115) * Add INX backend for fill analytics * Add --help docs for fill-analytics * Simplify * Lifetimes * Less moves * Small refactor * Set input source default * Fixes everything * Fix live INX analytics * Cleanup --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> * Cleanup error types --------- Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> Co-authored-by: /alex/ <alexander.schmidt@iota.org> * Cleanup * Starting impl * Rename analytics choice * Format * Fix calculation * Rework analytic (#1125) * Rework analytic * dashboard tweaks * more tweaks * fine, clippy, fine * rename * rename * Add more buckets * Rename * Refactor --------- Co-authored-by: Alexander Schmidt <alexander.schmidt@iota.org> * Fix dashboard 1 * Fix dashboard 2 --------- Co-authored-by: Jochen Görtler <jochen.goertler@iota.org> Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> Co-authored-by: Alexandcoats <alexandcoats@gmail.com> * Small refactor (#1130) --------- Co-authored-by: Alexander Schmidt <alexander.schmidt@iota.org> * Fix dashboard 1 * Fix dashboard 2 * Fix dashboard 3 * Rename panel title; Remove unit * Change panel titles --------- Co-authored-by: Jochen Görtler <jochen.goertler@iota.org> Co-authored-by: Jochen Görtler <grtlr@users.noreply.github.com> Co-authored-by: Alexandcoats <alexandcoats@gmail.com> * fix(api): refactor block queries to avoid lookups (#1133) * Refactor block queries to avoid lookups * 🙏 * Add shimmer test data * Fix paths * PR suggestion * Revert applied query * feat(analytics): rename value fields to amount (#1132) rename value fields to amount * chore(docs): Update env var docs (#1136) Update docs with latest env vars * fix(metrics): Start measurement before fetching milestone (#1137) Start timer before milestone fetch * fix(analytics): fill daily analytics (#1131) * fix daily addresses * Separate daily/per-milestone, cleanup traits * Remove unnecessary tags * Change daily to interval * Fix another bug in chunking * parallelism improvements for mongo source * Capitalization * cleanup * Simplify chunking (#1139) * Set up buckets only once for Transaction Size analytic (#1138) * Set up buckets only once * No more hashmaps * bump indexes by one * Remove once_cell * suggestions and comments * clippy --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> --------- Co-authored-by: /alex/ <alexander.schmidt@iota.org> * Clap cleanup * fix(analytic): Fix base token analytic (#1144) Sum it up * feat(analytics): unit tests (#1145) * Add unit tests for analytics with shimmer test data. Simplify analytics. * Fix transaction size influx query. Cleanup tests for clippy. * Further simplification of analytics trait * Remove pub * chore(deps): bump deps (#1147) bump dependencies --------- Co-authored-by: Alex Coats <alexandcoats@gmail.com> Co-authored-by: /alex/ <alexander.schmidt@iota.org>
- Loading branch information
3 people
committed
Feb 28, 2023
1 parent
14a67e9
commit 8870176