[ Enhancement ] Backup and restore from stream

[davidducos]

Initial implementation of stream backups

[davidducos]

This is an example of how you can execute mydumper and myloader using stream:

root@ubuntu-focal:~/stream/mydumper# rm -rf backup/ backup2/* ; ./mydumper -o backup -G -R -L log -v 4 -B test --stream --no-delete | ./myloader -d backup2 -o -B test2 -v 4 --stream --no-delete; ls backup backup2
** Message: 14:40:20.422: Step 1 completed, Databases created
** Message: 14:40:20.427: Step 1 completed, databases created
** Message: 14:40:20.445: Step 2 completed, tables created
** Message: 14:40:20.490: Thread 2 restoring table `test2`.`perftest` from backup2/test.perftest-schema.sql
** Message: 14:40:20.490: Dropping table or view (if exists) `test2`.`perftest`
** Message: 14:40:20.516: Creating table `test2`.`perftest` from backup2/test.perftest-schema.sql
** Message: 14:40:20.520: Step 3 completed, load data finished
** Message: 14:40:20.763: Thread 4 restoring `test2`.`perftest` part 0 of 0 from test.perftest.00000.sql. Progress 1 of 1 .
** Message: 14:40:20.763: Thread 1 ending
** Message: 14:40:20.763: Thread 3 ending
** Message: 14:40:20.763: Thread 2 ending
** Message: 14:40:22.234: Thread 4 ending
** Message: 14:40:22.234: Step 4 completed
** Message: 14:40:22.234: Import timings:
** Message: 14:40:22.234: Data      	| Index    	| Total   	| Table
** Message: 14:40:22.234: Step 5 started
** Message: 14:40:22.234: Starting table checksum verification
backup:
metadata  test-schema-create.sql  test.perftest-schema.sql  test.perftest.00000.sql  test.perftest.metadata

backup2:
metadata  test-schema-create.sql  test.perftest-schema.sql  test.perftest.00000.sql  test.perftest.metadata

[breisig]

Will this get merged anytime soon for stream support?

[davidducos]

Hi @breisig ,
I expect to be released in v0.11.3 which is planned to mid-Nov but I hope this will be merge this week to master.
Take into account that you can test branch stream, as latest commit fix several issues.

[davidducos]

If you use nc, you might need to use -N on the sender side to send the EOF, if not, it might be possible that the connection stay up.