Firebird doesn't respect the MaxParallelWorkers parameter value

[iCelo-git]

MaxParallelWorkers

Firebird doesn't respect the MaxParallelWorkers parameter value when restoring the database. I didn't notice the problem with backup
It was tested on Linux, Windows, on Firebird 5.0.3 and 5.0.4 snapshot.
On my settings I have MaxParallelWorkers = 8. No matter if I run simple gbak or gbak with -service localhost/3050:service_mgr
Firebird definitely uses a value provided in the parameter, ignoring MaxParallelWorkers parameter:

gbak:use up to 12 parallel workers
gbak:transportable backup -- data in XDR format
gbak:		backup file is compressed
gbak:backup version is 10
gbak: WARNING:Wrong parallel workers value 12, valid range are from 1 to 8

When I ran it with -par 2000, gbak used 6 GB of RAM. When gbak ran with -par 16384, it ate all 64 GB of the server's RAM.

[hvlad]

gbak is not an engine and should not and doesn't use engine settings from firebrid.conf.
gbak's additional workers have nothing common with engine's workers.
Note also that gbak could have no access to the engine's firebird.conf

Please, read the docs carefully. In doc/README.gbak it is stated explicitly that:

Note, gbak not uses firebird.conf by itself and ParallelWorkers setting does not affect its operations.

BTW, you see the warning about "Wrong parallel workers value" that was issued by the engine after gbak attaches to it.

[iCelo-git]

Well, you are correct the MaxParallelWorkers from firebird.conf has no effect on gbak, as it is stated also in documentation and I understand that gbak is not the engine by itself. However what is the purpose of having such parameter in the first place? As one of the biggest enhancements in Firebird 5.0 is the multithreading in backup operations and introduction of MaxParallelWorkers and ParallelWorkers in firebird.conf feels like the way to manage that. Also having gbak just accepting whatever value in -par feels kind of dangerous as it can kill the firebird service (running as SuperServer) or even the whole server OS (that's what I encountered).

[hvlad]

As one of the biggest enhancements in Firebird 5.0 is the multithreading in backup operations and introduction of MaxParallelWorkers and ParallelWorkers in firebird.conf feels like the way to manage that.

gbak's multithreading is controlled by its command-line switch -PARALLEL.
This value is used to set number of additional attachments that gbak establishes to read\write database data by backup\restore.
Every such additional attachment works in own thread and all of them is controlled by gbak.
Engine have no idea about its usage, this is regular user attachments from engine POV.

engine's multithreading is controlled by firebird.conf and connections parameters, passed via DPB by applications.
engine uses parallel workes for some tasks like sweep or index creation.
gbak utilise this engine's ability by passing value of its own partameter set via command-line switch -PARALLEL.

Also having gbak just accepting whatever value in -par feels kind of dangerous as it can kill the firebird service (running as SuperServer) or even the whole server OS (that's what I encountered).

This can be addressed at gbak side, but it have nothing common with subj.

[tomaszdubiel18]

Shouldn't this be reopened? An issue for normal backup is understandable, but have you noticed the same problem is present when invoking with -service localhost/3050:service_mgr? Then firebird is responsible for the task, not gbak. RAM grows on firebird process and firebird can spot here that the value is bigger than in MaxParallelWorkers and to run it with MaxParallelWorkers value.

[hvlad]

Shouldn't this be reopened?

This - definitely not. New issue with correct claim - sure.

An issue for normal backup is understandable, but have you noticed the same problem is present when invoking with -service localhost/3050:service_mgr? Then firebird is responsible for the task, not gbak.

It is still gbak, even if it run inside Firebird process.

[tomaszdubiel18]

OK, please tell me how I should report it, what problem exactly to describe. Why does it matter for me? Because doing remote backup with service manager can blow up the server. That is some security hole.

[hvlad]

Ask to add limit for max value of -PARALLEL switch value.

I have no good idea what value it should be, so it is up to you to offer good candidate that should satisfy most users ;)

Also, we can use MaxParallelWorkers setting from firebird.conf if and only if gbak runs as a service, but I'm not sure if it is good idea to make such mixed usage of setting for (close but) different purposes.

As for security hole... if everybody is allowed to run services and use crazy parameters - yes, this is security hole.
Do you offer to remove -replace switch, for example ?

[tomaszdubiel18]

I see a place for improvement.
If gbak is able to attach to firebird process (which has access to firebird.conf) just by using services manager, then why not to use it exactly to read MaxParallelWorkers? It seems an ideal parameter to restrict max parallel workers. Why does it have to be limited for parallel operations apart from gbak? I will propose it in a new ticket.
We will discuss it there :-)
When talking about security - does running services require admin role? Normal user wouldn't be able to restore a database via service manager?

[hvlad]

When talking about security - does running services require admin role?

One can't connect to the service manager without correct Firebird auth.
I have no idea why you mention admin role here.

Normal user wouldn't be able to restore a database via service manager?

Database restore (create) is privileged operation.

[AlexPeshkoff]

Database restore does not require admin role - it requires some privileges that are certainly granted to privilege role but may also be granted to any user.

[mrotteveel]

I think that at minimum it requires CREATE_DATABASE and USE_GBAK_UTILITY.

[tomaszdubiel18]

I would see it as some kind of DDOS problem. User with given priviliges may use this to cause permanent unavailability of the service and overall problems with functioning of the server.

[mrotteveel]

I would see it as some kind of DDOS problem. User with given priviliges may use this to cause permanent unavailability of the service and overall problems with functioning of the server.

As I said on firebird-java, a normal user could already do that by creating a lot of normal connections to an existing database. Limiting gbak will not solve this, and otherwise such a malicious actor could simply restore multiple databases, etc.

[tomaszdubiel18]

OK, however now it is very simple to do that. Just run gbak :-) that's all.