Replication support #182

dyemanov · 2019-01-08T04:57:21Z

Still work in progress, but suitable as feature preview for Beta 1. Discussion will follow in fb-devel, questions may be raised here as well.

AppVeyorBot · 2019-01-08T05:35:20Z

❌ Build firebird 1.0.387 failed (commit fe1b5f00eb by @dyemanov)

aafemt · 2019-01-08T12:31:56Z

doc/README.replication.md

+
+## Setting up the master side
+
+Replication is configured using a single configuration file: replication.conf. It allows to define global settings as well as per-database settings. All the possible options are listed inside replication.conf, descriptions are provided as comments there. For per-database configuration, full database name must be specified \(aliases or wildcards are not allowed\) inside the {database} section.


Why separate config file with the same syntax as databases.conf instead of databases.conf itself? It has no sense to distribute database's parameters into several files.
I would understand if it was plugin's config file, but now it AFAIU is read by engine itself.

The reasons are partially historical (initial implementation was for v2.5 without per-database configuration) and partially intentional - to separate purely optional feature from the "core" configuration. But it can be changed if your suggestion is supported by others.

I think that for users would be more convenient to have all database-related settings in one place.

aafemt · 2019-01-08T12:39:06Z

src/include/firebird/FirebirdInterface.idl

+
+// Replication interfaces
+
+interface ReplicatedRecord : Versioned


I see no method to get record's format or any other way to get separate fields values. Does it exist?

It doesn't exist yet, but I plan to add it after Beta. Your suggestions are appreciated.

Avalerion uses this:

class IField { public: virtual bool hasData() = 0; virtual const char* getName() = 0; virtual int getType() = 0; virtual int getScale() = 0; virtual int getLength() = 0; virtual int getCharset() = 0; virtual bool isNull() = 0; virtual void* getData() = 0; virtual bool next() = 0; }; class IRecord { public: virtual const char* getSchemaName() = 0; virtual const char* getTableName() = 0; virtual long long getNumber() = 0; virtual IField* getField() = 0; };

Where IField is actually enumerator. I would like to have also "bool isKey()" in it.

aafemt · 2019-01-10T12:15:24Z

src/include/firebird/FirebirdInterface.idl

+	Status getStatus();
+
+	ReplicatedTransaction startTransaction(int64 number);
+	boolean cleanupTransaction(int64 number);


Some comments about "what this method is supposed to do and when is called" would be very appreciated.

aafemt · 2019-01-10T12:16:50Z

I see interfaces for CDC, but I don't see a new plugin type for CDC. It was supposed to be a plugin, wasn't it?

aafemt · 2019-01-10T13:13:36Z

src/include/firebird/FirebirdInterface.idl

+
+	boolean storeBlob(ISC_QUAD blobId, ReplicatedBlob blob);
+
+	boolean executeSql(const string sql);


What is character set of this string? Could it be fixed UTF-8, please?..

It's fixed UTF-8.

aafemt · 2019-01-10T13:18:53Z

A lot of changes seems to be completely unrelated to replication. Could they go to a separate PR?

dyemanov · 2019-01-13T14:09:07Z

I see interfaces for CDC, but I don't see a new plugin type for CDC. It was supposed to be a plugin, wasn't it?

Yes, it's expected after Beta 1. You may provide a pull request.

dyemanov · 2019-01-13T14:11:15Z

A lot of changes seems to be completely unrelated to replication. Could they go to a separate PR?

They're all related, IIRC (even if indirectly).

aafemt · 2019-01-13T14:33:31Z

They're all related, IIRC (even if indirectly).

I really cannot see relation between replication and renaming of Win32DirItr into Win32DirIterator for example.

dyemanov · 2019-01-13T14:43:52Z

They're all related, IIRC (even if indirectly).

I really cannot see relation between replication and renaming of Win32DirItr into Win32DirIterator for example.

Yep, sorry for that. I used to code a different class in other forks and discovered Win32DirItr only during backporting. As it wasn't used in our codebase before replication, I've made some adjustments (including better readability). I understand they may complicate the review process, but I'd rather prefer to keep them together with replication.

aafemt · 2019-01-13T15:19:36Z

It wasn't used because the codebase contains TWO classes for directory iteration.

dyemanov · 2019-01-13T15:50:05Z

It wasn't used because the codebase contains TWO classes for directory iteration.

The one inside PathUtils was created first (circa FB 1.5), then Jim ignored it and created ScanDir, then I ignored both and created DirectoryWalker ;-) I've fixed my mistake now, but I'm not going to fix the remaining duplication (at least not in this PR).

AlexPeshkoff · 2019-01-13T15:52:27Z

On 1/13/19 6:50 PM, Dmitry Yemanov wrote: It wasn't used because the codebase contains TWO classes for directory iteration. The one inside PathUtils was created first (circa FB 1.5), then Jim ignored it and created ScanDir, then I ignored both and created DirectoryWalker ;-) I've fixed my mistake now, but I'm not going to fix the remaining duplication (at least not in this PR).

Yes, that dup definitely has nothing to do with replication.

aafemt · 2019-01-13T15:58:08Z

And that's exactly what I said: this cleanup and renaming should better go to a separate PR.

AppVeyorBot · 2019-01-14T11:05:58Z

❌ Build firebird 1.0.396 failed (commit 86bbe9a5a1 by @dyemanov)

asfernandes · 2019-01-24T15:00:06Z

builds/install/misc/replication.conf

+	# Connection string to the replica database (used for synchronous replication only).
+	# Expected format:
+	#
+	# [<login>:<password>@]<database connection string>


What about login and password with "special" characters as : and @?

Currently, you may omit both and use ISC_USER/ISC_PASSWORD instead. I can also add support for the quotes, i.e. "myn@me":"my:pwd". But speaking honestly, I'm not satisfied with the current solution, maybe you guys will raise some clever idea.

Hmm... is it setting with value only, no name ?

Ah, i see it below. It is "sync_replica".
Why not break it into 3 parts: login\password\connection string ?

@hvlad, do you mean three different settings? How to define multiple replicas in this case? Now you can specify:
sync_replica user1:pwd1@/my/first/replica
sync_replica user2:pwd2@/my/second/replica
sync_replica user3:pwd3@/my/third/replica
This may be not very elegant, but with three settings per each replica it's going to be a mess.

Just as idea, maybe we could use nested sections:
database = /my/db
{
sync_replica = /my/first/replica
{
user = user1
password = pwd1
## may be other connection options
}
}
but I don't know whether such nesting is supported by our config classes.

Exactly, nested sections should work

dyemanov · 2019-01-25T09:11:25Z

PR was merged but feel free to post any further reviews here, they will be noted/answered.

aafemt · 2019-02-20T11:55:50Z

Isn't upgrading of record format on source side just a waste of time, especially in REPL_store()?

aafemt · 2019-02-21T11:32:17Z

src/jrd/replication/Replicator.h

+				return m_replicator->startSavepoint(this) ? FB_TRUE : FB_FALSE;
+			}
+
+			FB_BOOLEAN releaseSavepoint()


Firebird 2.5 uses "relative" comit/rollback of savepoints which caused some bugs. Firebird 4 uses "absolute" rollback to savepoint with given number. Shouldn't this piece of code be changed accordingly?

AFAIU, you speak about looper savepoints. But these methods behave as SAVEPOINT, RELEASE SAVEPOINT and ROLLBACK TO SAVEPOINT statements. How they're implemented internally is irrelevant, provided that all changes between "start" and "rollback" are undone.

I just see that in ensureSavepoint() you start whole stack of savepoints that is unnecessary, IMHO.

Priorly replicated savepoints are not replicated again. The rest of the stack is needed to release/rollback them later. I'm open to any optimization ideas here.

The rest of the stack is needed to release/rollback them later.

Nope. It is easy to handle release of "never started savepoints" by full rollback. Here is Avalerion's sequence of calls:

Transaction stared savepoint 1 - ignored by replication module because transaction is still read-only

select started savepoint 2 - the same

select release savepoint 2 - the same

update started savepoint 3 - still nothing

update changed a record - replication modile at last is initialized, allocating replicator (ReplTransaction in Avalerion) and call updateRecord()

update rollback or release savepoint 3 - plugin::rollback/releaseSavepoint() is called and replication plugin scan internal list of savepoints (in asynchronous mode only) looking for number 3. When not found, it assumes that whole work from beginning is done or rolled back and act accordingly.

In your case at point 5 (AFAIU) startSavepoint() in replication module will be called several times which is avoidable. Savepoint is replicated at moment when plugin::releaseSavepoint() has been successfully called so additional marking of it as "replicated" is unnecesary because the flag is immediatelly cleaned up by Savepoint::release() which is called right after that.

dyemanov · 2019-02-21T15:53:20Z

Isn't upgrading of record format on source side just a waste of time, especially in REPL_store()?

If the record is already in the latest format, upgrade is skipped. Are you worried about the redundant MET_current() call?

aafemt · 2019-02-21T16:26:16Z

No, it seems to be lightweight enough, but I'm worried about contrary case in which "internal" part of replication does work that may be not necessary for "external" part (i.e. plugin). I understand, that this code is oriented to your particular implementation of capture plugin, but there can be others. Some of them may be even interested in old BLOB content (or not interested at all).

aafemt · 2019-02-21T18:17:12Z

src/jrd/replication/Publisher.cpp

+		// Replicate the entire stack of active savepoints (excluding priorly replicated),
+		// starting with the oldest ones
+
+		HalfStaticArray<Savepoint*, 16> stack;


This stack has no point: because of "relative" savepoint handling you need only count of them but not order. On success result doesn't matter and in case of any error complete replicator is destroyed and "replicated" flag in savepoint is irrelevant.

Agreed about the count only (the very-first implementation was passing savepoint IDs, the stack still remains). The "replicated" flag is relevant to avoid starting priorly started (upper level) savepoints. Imagine the first update executing inside savepoint frame {1, 2, 4, 10}, then savepoint 10 is released and the next update is executed inside savepoint frame {1, 2, 4, 12}. Savepoints {1, 2, 4} were already started and should not be replicated again.

I said that it is irrelevant in the case if error happen.
No error retirned from startSavepoint() - whole savepoint stack is marked as "replicated" and no problem here.
On error all savepoints are rolled back and dropped.

Initial implementation of replication

c10142d

aafemt reviewed Jan 8, 2019

View reviewed changes

aafemt reviewed Jan 10, 2019

View reviewed changes

Postfixes for configuration and Windows builds

0463ff7

dyemanov added 9 commits January 14, 2019 14:25

Misc Windows adjustments

371f242

Forgotten changes from prior commits

3273d69

Last round (fingers crossed) of fixing the Windows build

3b7d933

Merge remote-tracking branch 'upstream/master' into replication

8be85ce

Improve self-documentation

4639a55

Misc improvements

5e63119

Merge remote-tracking branch 'upstream/master' into replication

5b1977d

Updates the docs

a0d8e1d

More configuration

2a210b0

asfernandes reviewed Jan 24, 2019

View reviewed changes

dyemanov added 2 commits January 24, 2019 22:11

Frontported some remaining pieces

4ae1da4

Installation post-fixes

c955a0f

Special permission for replication

573414f

dyemanov merged commit 932ca51 into FirebirdSQL:master Jan 25, 2019

aafemt reviewed Feb 21, 2019

View reviewed changes


		## Setting up the master side

		Replication is configured using a single configuration file: replication.conf. It allows to define global settings as well as per-database settings. All the possible options are listed inside replication.conf, descriptions are provided as comments there. For per-database configuration, full database name must be specified \(aliases or wildcards are not allowed\) inside the {database} section.


		// Replication interfaces

		interface ReplicatedRecord : Versioned


		boolean storeBlob(ISC_QUAD blobId, ReplicatedBlob blob);

		boolean executeSql(const string sql);

Uh oh!

Replication support #182

Replication support #182

Uh oh!

Conversation

dyemanov commented Jan 8, 2019

Uh oh!

AppVeyorBot commented Jan 8, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aafemt commented Jan 10, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aafemt commented Jan 10, 2019

Uh oh!

dyemanov commented Jan 13, 2019

Uh oh!

dyemanov commented Jan 13, 2019

Uh oh!

aafemt commented Jan 13, 2019

Uh oh!

dyemanov commented Jan 13, 2019

Uh oh!

aafemt commented Jan 13, 2019 via email

Uh oh!

dyemanov commented Jan 13, 2019

Uh oh!

AlexPeshkoff commented Jan 13, 2019 via email

Uh oh!

aafemt commented Jan 13, 2019 via email

Uh oh!

AppVeyorBot commented Jan 14, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dyemanov commented Jan 25, 2019

Uh oh!

aafemt commented Feb 20, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dyemanov Feb 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dyemanov commented Feb 21, 2019

dyemanov Feb 21, 2019 •

edited

Loading

aafemt commented Feb 21, 2019 •

edited

Loading