BUG#21653352 : Utilities fails with Server SQL mode set to ANSI QUOTES

The utilities fail to execute when the sql_mode=ANSI_QUOTES is set on the
MySQL Server. While the bug report mysqlrpladmin failing due to a internal
query that can not be executed when the server is set to
sql_mode=ANSI_QUOTES
because of the use of double quotes. In addition this sql_mode mask the
hashed password with the string '<secret>' causing errors while parsing a
grant statement which limits gathering user's permission and more
utilities
to fail with this mode.

This patch contains the following changes to fix the issues mentioned
above:
-The regular expression to parse the grant statements has been updated to
 support statements with the hashed password masked.
-The queries on common\replication.py has been modified to avoid use of
 double quotes (") and replaced with single quotes (') capable to run
 regardless of the sql_mode set in the server.
-New regular expressions had been added to parse the object identifiers
 for the use of double quotes (").

With this patch the users do not need to specify the sql_mode they use
in the server, this is queried from the servers given to the utility, but
users need to be careful to use the escape sequences when the objects
identifier contains backtick quotes. While a backtick quote needed to be
escape by a second backtick, the ANSI quotes sql_mode does not require such
escape.

Servers with sql_mode set to ANSI_QUOTES or a mixed sql_mode can now
be used  with the following utilities mysqldbcompare, mysqldbcopy,
mysqldbexport, mysqldbimport, mysqldiff, mysqlgrants, and mysqlrpladmin.
The mysqlrplsync requires to all or none of the servers to be set with
theSQL mode set to ANSI_QUOTES.

New MUT tests has been added in this patch to exercise the changes.

v.4

mysql-test/mutlib/mutlib.py

@@ -1,5 +1,5 @@
 #
-# Copyright (c) 2010, 2015 Oracle and/or its affiliates. All rights reserved.
+# Copyright (c) 2010, 2016 Oracle and/or its affiliates. All rights reserved.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -1585,7 +1585,7 @@ def drop_db(server, db, debug=False):
         if not res:
             return True  # Ok to exit here as there weren't any dbs to drop
         try:
-            q_db = quote_with_backticks(db)
+            q_db = quote_with_backticks(db, server.select_variable("SQL_MODE"))
             server.exec_query("DROP DATABASE {0}".format(q_db))
         except UtilError as err:
             if debug:

mysql-test/r/compare_db_errors_ansi_quotes.result

@@ -0,0 +1,131 @@
+Test case 1 - Invalid --server1 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Server1 connection values invalid
+Test case 2 - Invalid --server2 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Server2 connection values invalid
+Test case 3 - missing backticks inventory.inventory 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Cannot parse the specified database(s): 'inventory.inventory'. Please verify that the database(s) are specified in a valid format (i.e., db1[:db2]) and that backtick quotes are properly used when required. The use of backticks is required if non alphanumeric characters are used for database names. Parsing the specified database results in db1 = 'inventory' and db2 = 'inventory'.
+Test case 4 - non existing database '`inventory.inventory`'
+# server1 on localhost: ... connected.
+# server2 on localhost: ... connected.
+ERROR: The database inventory.inventory does not exist.
+Test case 5 - invalid format :inventory 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Cannot parse the specified database(s): ':inventory'. Please verify that the database(s) are specified in a valid format (i.e., db1[:db2]) and that backtick quotes are properly used when required.
+Test case 6 - invalid value for --span-key-size=A 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: option --span-key-size: invalid integer value: 'A'
+Test case 7 - invalid value for --span-key-size=16.6 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: option --span-key-size: invalid integer value: '16.6'
+Test case 8 - span size equal to zero: --span-key-size=0 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: The value 0 specified for option --span-key-size is too small and would cause inaccurate results, please retry with a bigger value or the default value of 8.
+Test case 9 - size too low for --span-key-size=-4 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: The value -4 specified for option --span-key-size is too small and would cause inaccurate results, please retry with a bigger value or the default value of 8.
+Test case 10 - size too high for --span-key-size=33 
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: The value 33 specified for option --span-key-size is too big. It must be smaller or equal than 32 (size of the key hash values for comparison).
+Test case 11 - no options
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: You must specify at least one database to compare or use the --all option to compare all databases.
+Test case 12 - No pri key
+# server1 on localhost: ... connected.
+# server2 on localhost: ... connected.
+# Checking databases inventory on server1 and inventory on server2
+#
+#                                                   Defn    Row     Data   
+# Type      Object Name                             Diff    Count   Check  
+# ------------------------------------------------------------------------- 
+# TABLE     box                                     pass    pass    -       
+#           - Compare table checksum                                FAIL    
+#           - Find row differences                                  SKIP    
+#
+# The table box does not have an usable Index or primary key.
+
+# TABLE     box_2                                   pass    pass    -       
+#           - Compare table checksum                                FAIL    
+#           - Find row differences                                  SKIP    
+#
+# The table box_2 does not have an usable Index or primary key.
+
+
+# Database consistency check failed.
+#
+# ...done
+Test case 13 - Invalid --character-set
+# server1 on localhost: ... 
+ERROR: Character set 'unsupported_charset' unsupported.
+Test case 14 - Invalid --use-indexes and different indexes
+# server1 on localhost: ... connected.
+# server2 on localhost: ... connected.
+# Checking databases inventory on server1 and inventory on server2
+#
+#                                                   Defn    Row     Data   
+# Type      Object Name                             Diff    Count   Check  
+# ------------------------------------------------------------------------- 
+# TABLE     box                                     SKIP    pass    -       
+#           - Compare table checksum                                SKIP    
+#           - Find row differences                                  SKIP    
+#
+# The table box does not have an usable Index or primary key.
+
+# TABLE     box_2                                   SKIP    pass    -       
+#           - Compare table checksum                                SKIP    
+#           - Find row differences                                  SKIP    
+#
+# The table box_2 does not have an usable Index or primary key.
+
+# TABLE     box_3                                   SKIP    pass    -       
+#           - Compare table checksum                                SKIP    
+#           - Find row differences                                  FAIL    
+#
+The specified index "invalid_index" was not found in table inventory.box_3
+
+# TABLE     box_4                                   SKIP    pass    -       
+#           - Compare table checksum                                SKIP    
+#           - Find row differences                                  FAIL    
+#
+Indexes are not the same.
+
+
+# Database consistency check failed.
+#
+# ...done
+Test case 15 - Cannot use --all with a database argument.
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: You cannot use the --all option with a list of databases.
+Test case 16 - Option --server1 is required.
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Option '--server1' is required.
+Test case 17 - Option --server2 is required.
+Usage: mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
+
+mysqldbcompare: error: Option '--server2' is required.
+Test case 18 - Cannot compare all databases on the same server.
+# server1 on 127.0.0.1: ... connected.
+# server2 on localhost: ... connected.
+ERROR: Specified servers are the same (server1=127.0.0.1:PORT1 and server2=localhost:PORT1). Cannot compare all databases on the same server.
+Test case 19 - Comparing the same object on the same server.
+# server1 on 127.0.0.1: ... connected.
+# server2 on localhost: ... connected.
+ERROR: Comparing the same object on the same server.
+Test case 20 - Cannot compare all databases on the same server and with the same connection string.
+# server1 on localhost: ... connected.
+# server2 on localhost: ... connected.
+ERROR: Specified servers are the same (server1=localhost:PORT1 and server2=localhost:PORT1). Cannot compare all databases on the same server.