+per #16541 initial version of the Persistence Query module #18051

ktoso · 2015-07-22T10:39:55Z

Ladies and Gentlemen,
Thanks to the Akka Streams 1.0 release last week, we now would like to present this new experimental module for initial feedback, addressing a long history of discussions and awesome feedback we have gathered from the community (see the great summary: Akka Persistence on the Query Side: The Conclusion).

Introducing the Akka Streams based improved Query Side to Akka Persistence.
This is only the initial PR, more examples, polish and example implementations will be pushed soon.

The current form of the API is the culmination of these discussions with the community, a very long "maturing in the back of your mind" process and a few days spent together around ScalaDays for polishing and trying it out in various example/mock implementations with @patriknw and @dotta.

The API is intentionally very "open" (we provide the marker traits Query[T, M], Hint, and a number of basic case classes), because we do not want to restrict some of the great journals out there - instead, we want to encourage various implementations to expose that kind of API which is best suited for the underlying store. For example, SQL based journals may provide SqlQuery("SELECT * ...") if that's what they want to expose. Journals will have to document their behaviours very thoroughly.

Other important information:

this module (akka-persistence-query-experimental) will be (and stay) experimental in Akka 2.4
we'll provide a number of example ReadJournal implementations, however we currently do not intend to maintain one for production usage - we ask the community to help out, similarily like it has happened organically with Akka Persistence.
integration with the new EventAdapters known from the write-side in Akka 2.4 is under discussion/development Integrate EventAdapters to the Query side #18050

ktoso · 2015-07-22T10:40:23Z

Refs #16541

akka-ci · 2015-07-22T11:20:06Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/3269/

jrudolph · 2015-07-22T13:33:43Z

Error lines in https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/3269/:

  1659 Unable to find credentials for [Artifactory Realm @ proxy-ch.typesafe.com].
  1660 Unable to find credentials for [Artifactory Realm @ proxy-ch.typesafe.com].
 12321 (akka-docs/sphinx:generatePdf) Sphinx pdf generation failed.  See debug output for details.
 12322 Total time: 713 s, completed Jul 22, 2015 1:19:57 PM

ahjohannessen · 2015-07-22T15:34:30Z

akka-docs/rst/scala/persistence-query.rst

+query has an optionally supported offset parameter (of type ``Long``) which the journals can use to implement resumable-streams.
+For example a journal may be able to use a WHERE clause to begin the read starting from a specific row, or in a datastore
+that is able to order events by insertion time it could treat the Long as a timestamp and select only older events.
+Again, specific sapabilities are specific to the journal you are using, so you have to


capabilities

2m · 2015-07-24T12:18:03Z

LGTM. Very clean API!

rkuhn · 2015-07-27T08:55:18Z

akka-docs/rst/scala/code/docs/persistence/query/MyEventsByTagPublisher.scala

+//#events-by-tag-publisher
+class MyEventsByTagPublisher(tag: String, offset: Long, refreshInterval: FiniteDuration)
+  extends ActorPublisher[EventEnvelope] {
+  import MyEventsByTagPublisher._


I’d declare the case object Continue in here to avoid having to pull the companion into the docs as well.

patriknw · 2015-08-12T17:12:55Z

akka-docs/rst/java/persistence-query.rst

+
+``EventsByPersistenceId`` is a query equivalent to replaying a :ref:`PersistentActor <event-sourcing>`,
+however, since it is a stream it is possible to keep it alive and watch for additional incoming events persisted by the
+persistent actor identified by the given ``persistenceId``. Most journal will have to revert to polling in order to achieve


s/journal/journals/

patriknw · 2015-08-12T17:24:29Z

LGTM, after final touch

ahjohannessen · 2015-08-12T17:24:55Z

Gotta love collaboration, feedback is great 👍 😄

ktoso · 2015-08-12T18:25:18Z

Addressed all feedback, merging after validation (failure was my javadoc mistake)

akka-ci · 2015-08-12T18:40:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/3379/

akka-ci · 2015-08-12T18:44:50Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/3380/

akka-ci · 2015-08-12T20:02:41Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/3381/

+per #16541 initial version of the Persistence Query module

xuwei-k · 2015-08-16T17:57:34Z

lol

@@ -0,0 +1,172 @@
+adoc-api/akka/persistence/query/scaladsl/ReadJournal.html...


unnecessary log file?

Thanks for noticing @xuwei-k, removed 9b6b433
(used this file to debug javadoc failure using ./scripts/find-javadoc-error.pl)

Refs #18051

akka-ci added the validating PR is currently being validated by Jenkins label Jul 22, 2015

ktoso added t:stream t:persistence-query and removed t:stream labels Jul 22, 2015

akka-ci added needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed validating PR is currently being validated by Jenkins labels Jul 22, 2015

ahjohannessen reviewed Jul 22, 2015
View reviewed changes

rkuhn reviewed Jul 27, 2015
View reviewed changes

patriknw reviewed Aug 12, 2015
View reviewed changes

ktoso force-pushed the wip-16541-persistence-query branch 2 times, most recently from 9b404d8 to e0c7aec Compare August 12, 2015 18:21

akka-ci added validating PR is currently being validated by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) labels Aug 12, 2015

akka-ci added validating PR is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed validating PR is currently being validated by Jenkins labels Aug 12, 2015

+per #16541 add missing java samples for persistence query

3314de4

ktoso force-pushed the wip-16541-persistence-query branch from e0c7aec to 3314de4 Compare August 12, 2015 18:43

akka-ci added the needs-attention Indicates a PR validation failure (set by CI infrastructure) label Aug 12, 2015

akka-ci added validating PR is currently being validated by Jenkins tested PR that was successfully built and tested by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR is currently being validated by Jenkins labels Aug 12, 2015

ktoso added a commit that referenced this pull request Aug 12, 2015

Merge pull request #18051 from akka/wip-16541-persistence-query

5a6ee8b

+per #16541 initial version of the Persistence Query module

ktoso merged commit 5a6ee8b into master Aug 12, 2015

xuwei-k reviewed Aug 16, 2015
View reviewed changes

ktoso added a commit that referenced this pull request Aug 16, 2015

=pro Remove unneccessary log file

9b6b433

Refs #18051

rkuhn deleted the wip-16541-persistence-query branch January 13, 2016 11:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

+per #16541 initial version of the Persistence Query module #18051

+per #16541 initial version of the Persistence Query module #18051

ktoso commented Jul 22, 2015

ktoso commented Jul 22, 2015

akka-ci commented Jul 22, 2015

jrudolph commented Jul 22, 2015

ahjohannessen Jul 22, 2015

2m commented Jul 24, 2015

rkuhn Jul 27, 2015

ktoso Aug 11, 2015

patriknw Aug 12, 2015

patriknw commented Aug 12, 2015

ahjohannessen commented Aug 12, 2015

ktoso commented Aug 12, 2015

akka-ci commented Aug 12, 2015

akka-ci commented Aug 12, 2015

akka-ci commented Aug 12, 2015

xuwei-k Aug 16, 2015

ktoso Aug 16, 2015

		@@ -0,0 +1,172 @@
		adoc-api/akka/persistence/query/scaladsl/ReadJournal.html...

+per #16541 initial version of the Persistence Query module #18051

+per #16541 initial version of the Persistence Query module #18051

Conversation

ktoso commented Jul 22, 2015

ktoso commented Jul 22, 2015

akka-ci commented Jul 22, 2015

jrudolph commented Jul 22, 2015

ahjohannessen Jul 22, 2015

Choose a reason for hiding this comment

2m commented Jul 24, 2015

rkuhn Jul 27, 2015

Choose a reason for hiding this comment

ktoso Aug 11, 2015

Choose a reason for hiding this comment

patriknw Aug 12, 2015

Choose a reason for hiding this comment

patriknw commented Aug 12, 2015

ahjohannessen commented Aug 12, 2015

ktoso commented Aug 12, 2015

akka-ci commented Aug 12, 2015

akka-ci commented Aug 12, 2015

akka-ci commented Aug 12, 2015

xuwei-k Aug 16, 2015

Choose a reason for hiding this comment

ktoso Aug 16, 2015

Choose a reason for hiding this comment