New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make a better way to use multiple databases at once #400
Comments
Make it possible to run different code parallelly on different (or not) databases |
|
But it works on with Pg:
|
I'm thinking of accepting Positionals and not Positionals, so when Positional, it will respect the order. Otherwise, it will try to run everything in parallel... |
% perl6 -I. -e '
use Red <red-do>;
model Bla is table<not_bla> { has UInt $.id is serial; has Str $.name is column }
red-defaults pg => \("Pg", :default), sqlite => \("SQLite");
red-do <pg sqlite> => { Bla.^create-table: :if-not-exists };
say red-do (<pg sqlite> => { Bla.^load: 6 }), "sqlite" => { Bla.^load: 7 }, (pg => { Bla.^load: 8 })
'
(Bla.new(id => 6, name => "b") Nil Nil Bla.new(id => 8, name => "c")) |
This question would also involve
For readability purposes
Convenience alias for the above The internal implementation of the asynchronicity is not relevant. Perhaps Routines
Calling any of them would result in creating a new For example:
For convenience and syntax sugar Similarly, routines A side note:
Then if used like:
It would result in running the blocks in parallel. The point is in not using quotes around database names turning them into named parameters instead of a list of
|
I seem to be out of sync here, but that because I started writing the comment about 7 hours ago and finished just now. :) |
This supplies/suppliers should be shared between red-do’s or should exist only on the red-do where it was created? Sent with GitHawk |
Suppliers and channels are only for the I think |
@vrurg would you mind to write an example of use to that supply/channel? I couldn't "see" that yes... |
I mean, I keep with something like this on my head: supply {
red-do <db1 db2> => { whenever start { Model.^all.grep: {...} } -> $result { .emit for $result } }
} |
No, that's too much! Say, we need to duplicate records from one db into many:
And that's all. For this to work you would need to create a BTW, note that tap is an async co-routine. So, basically, the above example doesn't even need |
red-do :parallel,
db-source => {
red-emit "dup", $_ for Model.^all
},
<db-dest1 db-dest2> => {
red-tap "dup", -> $record {
Mode.^create: $record
}
} Maybe this way we avoid to duplicate code... |
I don't think this would be often needed. I've done it this way just to make the example clean. In real life it would be something like:
or, if it happens to be many destinations:
I see no need to overcomplicate in this case. |
@vrurg now this is possible (the pg was pre populated):
|
Maybe we should have a supply on the Driver that would emit everything that happens on that driver... and the user could grep what it wants. red-do {
.events.grep({ .event-type eq "create" and .model === MyModel }).tap: $logger.debug: "Created: { .obj }"
} and it would create a log entry ( Maybe the event class could be something like: class Event {
has enum <create delete update> $.event-type;
has Red::Model:U $.model;
has Red::Model $.obj;
has %.change;
} or maybe it should be a model to make it possible to the user store it on a database... |
It's something I would expect to be quite appreciated in a big production. I would add that perhaps it also makes sense to have supply of events for all active drivers allowing for unified processing of everything. Whoever needs just one stream – subscribes to individual driver. Whoever needs everything could use With regard to the A downside of this approach is possible performance cost, though it shouldn't be very influential in a parallel model as even emission and processing could take place in their own, likely shared, thread. But a way to set what kind of events a consumer is interested in would undoubtedly be reasonable to have. No idea what is PS. It feels like loading you with extra work. ;) |
I really like the idea of passing the AST. class Event {
has Red::Driver $.db;
has Str $.db-name;
has Str $.driver-name;
has Str $.name;
has $.data;
} And on automatic events About the |
Or maybe: class Event {
has Red::Driver $.db;
has Str $.db-name;
has Str $.driver-name;
has Str $.name;
has $.data;
has Red::AST $.ast;
has Red::Model $.object;
has X::Red $.error
} |
And ‘red-tap “bla”, {...}` should tap: Red.events.grep(*.name eq “bla”).map(*.data) And that would not need to be restricted to a single |
I could have been not clear enough, but I really didn't meant to get rid of given $event.data {
when Red::Ast { ... }
when Failure { ... } # or X::Red
...
} The original model object which was the source for an class Event {
has Red::Driver $.db;
has Str $.db-name;
has Str $.driver-name;
has Str $.name;
has $.data;
has Model $.origin;
} Using the system-wide event supplier for The original point of limiting data streams by enclosing One thing which worries me a bit is use of share
|
Started:
I've added the error field because it would be good to have the AST and the error in case of error. |
I've removed |
We'll need a place to add the bind information of the AST... |
As much as I thought about it, Event Is requiring another attribute for this. |
Maybe a Red::Event::AST and a Red::Event::Generic? That way we could use the type to filter... class Red::Event {
has Red::Driver $.db;
has Str $.db-name;
has Str $.driver-name;
}
class Red::Event::AST is Red::Event {
has Red::AST $.ast;
has @.bind;
has Red::Model $.orig;
}
class Red::Event::Generic is Red::Event {
has $.data;
}
Sent with GitHawk |
Does it really makes sense multiplying entities here? Or, if you don't want to pollute pure
Though its a bit of complication from inside, for a user it'd be clear that if Besides, this approach is easy to extend for other data types if necessary. |
use Red;
Red.events.tap: -> $ev { say $ev.data }
my $*RED-DB = database "SQLite";
model Bla {
has UInt $.id is id;
has Str $.name is column is rw
}
Bla.^create-table: :if-not-exists;
my $bla = Bla.^create: :name<test>;
$bla.name = "bla";
$bla.^save;
Bla.^all.grep({ .id < 10 and .name.starts-with: "bla" }).Seq;
sleep 5
|
BTW, how hard is it to get actual SQL from AST for an end-user? |
Just question of calling the driver's translate method passing the AST. And I was wrong... We do not store the bindings... it's just a thing after the AST translation. |
|
Just perfect for human-readable logging and possibly additional debugging! |
Maybe I should change the $*RED-DEBUG to use that... Sent with GitHawk |
Perhaps. But debug could require more than just dumping database events. Debug stream might include intermediate messages of state changes, dumps of local variables or whatever else could be considered useful. Also, file names and line numbers for the purpose of distinguishing identical messages. I can't say much in this area because I don't really know what exactly debugging does in Red. But my guess would be that additional debug event would be useful. The event could be emitted by a
The good thing about incorporating debugging into event subsystem is that it allows a user to plug in any reporting mechanism he/she likes. For example, events could be redirected to a WebSocket as JSON and be used by a front-end developer to observe what's going on in the backend. PS. That's funny that I also thought about events partially duplicating what debug does. But didn't want to overload you with something beyond the current task. But it was too much on the surface to get around... ;) |
now, every db meta-method can receive |
https://colabti.org/irclogger/irclogger_log/perl6?date=2019-09-17#l157
The text was updated successfully, but these errors were encountered: