Skip to content

Commit

Permalink
Support database-level dump/restore (#108)
Browse files Browse the repository at this point in the history
Added support for dumping a single Babelfish logical database. Changes are
summarized in following bullet points:

    Added a new option --bbf-database-name in both pg_dumpall and pg_dump to
    let user provide the name of logical database to be dumped.
    Fetch all the physical schema names corresponding to the given logical database
    and add all of them into schema_include_patterns list. With this, existing dump
    logic will make sure to only dump all these schemas as well as all the objects
    contained by these schemas.
    Function bbf_selectDumpableTableData takes care of explicitly marking all
    Babelfish catalog table data to be dumped.
    We need to selectively dump the Babelfish catalog table data corresponding to
    given logical database which is being handled by function getCursorForBbfCatalogTableData
    for COLUMN-INSERT and fixCopyCommand for COPY. These function prepares
    query using joins and conditions to filter the data.
    We skip dumping the dbid column as it might conflict with the existing database's dbid
    so we generate the new dbid during restore. It has been handled for both COLUMN-INSERT
    and COPY options:
    a. COPY: Implemented a hook fill_missing_values_in_copyfrom_hook which will use the
    sequence to generate the new dbid.
    b. COLUMN-INSERT: Used existing pre_parse_analyze_hook hook to handle this case.
    Added a new file dumpall_babel_utils.c to implement Babelfish specific logic to
    selectively dump users. New file is needed since existing dump_babel_utils.c can't
    be used with pg_dumpall utility.
    Logins do not belong to a single database so we will not be dumping the logins so it and
    we will only dump default users of the database (dbo, db_owner and db_guest).
    Due to this CREATE and INSERT might fail for certain logins but it should be fine.
    Current implementation does not dump foreign servers and foreign data wrappers as
    these are global objects and does not belong to a single logical database.
    Note: A logical database with certain migration mode (single-db/multi-db) can only be
    restored on the Babelfish server with the same migration mode. Which means a single-db
    database can be restored only on single-db Babelfish server, same thing applies to multi-db.
    Although the builtin databases like master can be restored on any kind of server.
  • Loading branch information
rishabhtanwar29 committed Mar 14, 2023
1 parent 0e93f3f commit 6440423
Show file tree
Hide file tree
Showing 9 changed files with 603 additions and 12 deletions.
5 changes: 5 additions & 0 deletions src/backend/commands/copyfromparse.c
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,8 @@ if (1) \
/* NOTE: there's a copy of this in copyto.c */
static const char BinarySignature[11] = "PGCOPY\n\377\r\n\0";

/* hooks */
fill_missing_values_in_copyfrom_hook_type fill_missing_values_in_copyfrom_hook = NULL;

/* non-export function prototypes */
static bool CopyReadLine(CopyFromState cstate);
Expand Down Expand Up @@ -1023,6 +1025,9 @@ NextCopyFrom(CopyFromState cstate, ExprContext *econtext,
&nulls[defmap[i]]);
}

if (fill_missing_values_in_copyfrom_hook)
fill_missing_values_in_copyfrom_hook(cstate->rel, values, nulls);

return true;
}

Expand Down
4 changes: 2 additions & 2 deletions src/bin/pg_dump/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ pg_dump: pg_dump.o common.o pg_dump_sort.o dump_babel_utils.o $(OBJS) | submake-
pg_restore: pg_restore.o $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_restore.o $(OBJS) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)

pg_dumpall: pg_dumpall.o dumputils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_dumpall.o dumputils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
pg_dumpall: pg_dumpall.o dumputils.o dumpall_babel_utils.o $(WIN32RES) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) pg_dumpall.o dumputils.o dumpall_babel_utils.o $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)

install: all installdirs
$(INSTALL_PROGRAM) pg_dump$(X) '$(DESTDIR)$(bindir)'/pg_dump$(X)
Expand Down

0 comments on commit 6440423

Please sign in to comment.