Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregation support for ledger dumps #3490

Merged
merged 4 commits into from
Aug 3, 2022
Merged

Conversation

dmkozh
Copy link
Contributor

@dmkozh dmkozh commented Jul 19, 2022

Description

This adds support of group by and some aggregation operators for simple aggregation operations.

Also added output format improvements for JSON dumps and more docs/examples.

Checklist

  • Reviewed the contributing document
  • Rebased on top of master (no merge commits)
  • Ran clang-format v8.0.0 (via make format or the Visual Studio extension)
  • Compiles
  • Ran all tests
  • If change impacts performance, include supporting evidence per the performance document

@@ -2,31 +2,32 @@
// under the Apache License, Version 2.0. See the COPYING file at the root
// of this distribution or at http://www.apache.org/licenses/LICENSE-2.0

#include "util/xdrquery/XDRMatcher.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong commit? I see that you have a "Rename XDRMatcher.h/cc to XDRQuery to be more general" later on

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

docs/software/ledger_query_examples.md Show resolved Hide resolved
@@ -546,7 +546,7 @@ exit /b 0
<ClCompile Include="..\..\src\util\test\SchedulerTests.cpp" />
<ClCompile Include="..\..\src\util\XDRCereal.cpp" />
<ClCompile Include="..\..\src\util\xdrquery\test\XDRQueryTests.cpp" />
<ClCompile Include="..\..\src\util\xdrquery\XDRMatcher.cpp" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this commit should probably be the first in your PR as the other commits seem to depend on the new name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did the rename after all the changes, but I guess this got messed up when I was squashing some intermediate commits. Updated the history.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you need to refresh timestamps for commits as to not confuse Github: right now that commit looks like the last one in the list but it's the first one in your chain.

The way to do this, is to rebase by running git rebase --force-rebase --exec "sleep 2" --exec "git commit --amend --date=now" master

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rename is supposed to be the last commit and it seems like it is both in git hub and git logs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes, I saw that 🤦‍♂️ . In my mind you were going to rename first before making the other changes. Looks good

…ction.

This allows to support group by and aggregation in `dump-ledger` command.
The output is dumped to CSV as it's always flat. I'm not adding JSON format for this for the sake of simplicity.

Also updated the JSON dump format to be more `jq`-friendly.
@MonsieurNicolas
Copy link
Contributor

r+ 2b1dce5

@latobarita latobarita merged commit 9ca42fb into stellar:master Aug 3, 2022
v19.4.0 automation moved this from In progress to Done Aug 3, 2022
@dmkozh dmkozh deleted the xdrquery_agg branch August 17, 2022 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants