Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for backing up MariaDB compressed and encrypted #200

Closed
wants to merge 4 commits into from

Conversation

Projects
None yet
4 participants
@janlindstrom
Copy link

janlindstrom commented May 3, 2016

tablespaces and tables. Furhermore, add support for backing
up MariaDB encrypted redo-logs.

Add support for backing up MariaDB compressed and encrypted
tablespaces and tables. Furhermore, add support for backing
up MariaDB encrypted redo-logs.

@gl-sergei gl-sergei self-assigned this May 4, 2016

@gl-sergei

This comment has been minimized.

Copy link
Member

gl-sergei commented Jun 25, 2016

Hi,

Here is what I found:

  • there is a memory leak in mariadb_check_tablespace_encryption
  • patch won't compile on 32-bit platforms with following error: mariadb.cc:103: error: format ‘%lu’ expects type ‘long unsigned int’, but argument 2 has type ‘ib_uint64_t’

Backing up of encrypted and compressed tables seems working. Prepare doesn't work. Will it be separate patch? I didn't check backing up with redo log encryption enabled yet.

Fixed memory leak on mariadb_check_tablespace_encryption function and
fixed 32-bit platform compiler error mariadb.cc:103: error: format ‘%lu’
expects type ‘long unsigned int’, but argument 2 has type ‘ib_uint64_t’.
@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented Jun 30, 2016

Hi,

Thank you for you review. Fixed issues on

janlindstrom@0ad6809

Prepare will be fixed on MariaDB server with separate patch.

@gl-sergei

This comment has been minimized.

Copy link
Member

gl-sergei commented Jul 11, 2016

Just an update. I merged this patch to 2.4 (which is based on 5.7) and it seems to work OK, except one debug assertion when page_size is 8k:

2016-07-05 08:02:36 0x7f3b5a28e740  InnoDB: Assertion failure in thread 139892892428096 in file fsp0space.h line 141
InnoDB: Failing assertion: fsp_flags_is_valid(fsp_flags)

I had no time to look at it yet.


DBUG_EXECUTE_IF("fsp_flags_is_valid_failure", return(false););

/* If there is flags that are MariaDB extended flags,
read unused and page size from MariaDB positions. */
if ((unused && mariadb_unused == 0) ||

This comment has been minimized.

Copy link
@gl-sergei

gl-sergei Jul 12, 2016

Member

I think unused != 0 && mariadb_unused == 0 would be more readable

This comment has been minimized.

Copy link
@janlindstrom
/* If there is flags that are MariaDB extended flags,
read unused and page size from MariaDB positions. */
if ((unused && mariadb_unused == 0) ||
(page_ssize && (page_ssize < UNIV_PAGE_SIZE_MIN ||

This comment has been minimized.

Copy link
@gl-sergei

gl-sergei Jul 12, 2016

Member

same for page_ssize != 0

This comment has been minimized.

Copy link
@janlindstrom
read unused and page size from MariaDB positions. */
if ((unused && mariadb_unused == 0) ||
(page_ssize && (page_ssize < UNIV_PAGE_SIZE_MIN ||
page_ssize > UNIV_PAGE_SIZE_MAX))) {

This comment has been minimized.

Copy link
@gl-sergei

gl-sergei Jul 12, 2016

Member

Shouldn't it be (page_ssize < UNIV_PAGE_SSIZE_MIN || page_ssize > UNIV_PAGE_SSIZE_MAX) instead of (page_ssize < UNIV_PAGE_SIZE_MIN || page_ssize > UNIV_PAGE_SIZE_MAX)?

This comment has been minimized.

Copy link
@janlindstrom

janlindstrom Aug 1, 2016

Author

There is no UNIV_PAGE_SSIZE_MIN, but making this more clear.

@gl-sergei

This comment has been minimized.

Copy link
Member

gl-sergei commented Jul 12, 2016

Should

/********************************************************************//**
Extract the page size from tablespace flags.
@return page size of the tablespace in bytes */
UNIV_INLINE
ulint
fsp_flags_get_page_size(
/*====================*/
    ulint   flags); /*!< in: tablespace flags */

also be updated?

However, I am not sure if xtrabackup using it during the copy.

Xtrabackup using FSP_FLAGS_GET_ZIP_SSIZE to determine page size. Does this field have the same position in MariaDB?

@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented Aug 1, 2016

FSP_FLAGS_ZIP_SSIZE position is not changed.

@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented Aug 1, 2016

I used latest Percona server 5.5 and configured it to use 8K page size, then tried to use 2.3 xtrabackup and it also fails on fsp_flags_is_valid() because:

  } else if (UNIV_PAGE_SIZE != UNIV_PAGE_SIZE_ORIG && !page_ssize) {
            fprintf(stderr, "InnoDB: Error: Tablespace flags %lu corrupted page_ssize %lu max %lu:%d\n",
                    flags, page_ssize, UNIV_PAGE_SIZE, UNIV_PAGE_SIZE_ORIG);
            return(false);

Looking into code, I do not see and ha_innodb::create ever storing page_ssize even if it is != UNIV_PAGE_SIZE_ORIG.
}

@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented Aug 1, 2016

Percona Server 5.6 has following in fil0fil.cc:

/* Add the UNIV_PAGE_SIZE to the table flags and write them to the
tablespace header. */
flags = fsp_flags_set_page_size(flags, UNIV_PAGE_SIZE);
@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented Aug 2, 2016

Found and fixed the error on page ssize determination when MariaDB flags were used.

@io41

This comment has been minimized.

Copy link

io41 commented Apr 7, 2017

Is there any plans to update this PR @janlindstrom ?
@gl-sergei if this gets updated against master, are you willing to merge it?

Thanks!

@janlindstrom

This comment has been minimized.

Copy link
Author

janlindstrom commented May 8, 2017

Removing this PR due to (1) it is based on old MariaDB version and (2) no actual progress.

@io41

This comment has been minimized.

Copy link

io41 commented May 10, 2017

In case anyone else finds this PR for the same reason I did, here's where the train is going: https://mariadb.com/resources/blog/mariadb-backup-released-mariadb-server-10123

@hydrapolic

This comment has been minimized.

Copy link

hydrapolic commented May 11, 2017

Thanks @io41 for the update, haven't noticed in the changelog.

This will be tricky (https://jira.mariadb.org/browse/MDEV-9566):
Missing:
Update xtrabackup sources (not sure how to do that easily and repeatable)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.