Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100755 177 lines (135 sloc) 4.731 kb
691c5bf First commit
Chris Cummer authored
1 #!/usr/bin/env ruby
2
3 require 'fileutils'
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
4 require 'yaml'
691c5bf First commit
Chris Cummer authored
5
6
7 # ------------------------------ Classes ------------------------------ #
8
9
10 class MySQL2SqliteConverter
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
11
12 def initialize(args)
13 @config = {
14 :database => nil,
15 :username => nil,
16 :password => nil,
17 :overwrite => true,
18 :tables => nil,
19 :mysqldump => 'mysqldump',
20 }
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
21
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
22 init_result = (args.length == 1) ? init_from_yaml(args) : init_from_command_line(args)
814d5bb Passwords in the YAML config file are now optional
Chris Cummer authored
23
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
24 if (init_result && @config[:database] && @config[:username])
1b33644 Cleaned up various lines
Chris Cummer authored
25 @file_deletion_delay = 10
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
26 @output_file, @sqlite_database = @config[:database] + ".sql", @config[:database] + ".sqlite"
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
27 else
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
28 raise 'Invalid configuration'
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
29 end
691c5bf First commit
Chris Cummer authored
30 end
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
31
691c5bf First commit
Chris Cummer authored
32 def mysql_to_sqlite()
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
33 handle_existing_files()
34
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
35 arr = []
691c5bf First commit
Chris Cummer authored
36
c81a83a Ability to define specific tables for export via YAML config file
Chris Cummer authored
37 mysqldump_str = generate_mysqldump_str()
38
39 IO.popen( mysqldump_str ) do |pipe|
691c5bf First commit
Chris Cummer authored
40 pipe.each_line do |line|
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
41 next if contains_disallowed_sql( line )
691c5bf First commit
Chris Cummer authored
42
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
43 line = translate_sql_differences( line )
44 line = translate_character_differences( line )
691c5bf First commit
Chris Cummer authored
45
46 arr << line
47 end
48 end
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
49
50 unless arr.empty?
51 @line_count = arr.length
52
53 complete_str = arr.join( '' )
54 complete_str.gsub!( /,\n\);/, "\);\n" )
4a20774 @Shumkov fix \r\n issue
Shumkov authored
55 complete_str.gsub!( /\\r\\n/, "\r\n" )
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
56
57 puts "Writing out to: #{@output_file}"
58 File.open( @output_file, 'w') { |f| f.write( complete_str ) }
59
60 puts "Writing out to: #{@sqlite_database}"
61 return system( "cat #{@output_file} | sqlite3 #{@sqlite_database}" )
62 end
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
63 end
64
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
65 private
66
67 def init_from_yaml(args)
68 if (File.exists?(args[0]))
69 config = YAML::load_file(args[0])
70 # Backward compatibility with old version of configs
71 config = config['config'] if config['config']
72
73 config.each do |key, value|
74 key = key.to_sym
75 if @config.has_key? key
8069631 Fixes password append issue (missing a required space)
Chris Cummer authored
76 @config[key] = value
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
77 else
78 raise 'Invalid option'
79 end
80 end
81 true
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
82 end
83 end
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
84
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
85 def init_from_command_line( args )
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
86 if (args.length >= 2)
87 @config[:database], @config[:username], = args[0], args[1]
88 @config[:password] = args[2] if args[2]
89 @config[:overwrite] = args[3] if args[3] && ("1" == args[3] || "true" == args[3])
90 true
91 end
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
92 end
93
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
94 def handle_existing_files()
95 handle_existing_file( @output_file )
96 handle_existing_file( @sqlite_database )
97 end
98
99 def handle_existing_file( file )
100 # TODO: Replace this with a query to the user, defaulting to Y
101 if ( File.exists?( file ) )
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
102 if ( true != @config[:overwrite] )
f8697bc Added option to over-ride file over-write warnings
Chris Cummer authored
103 (1..@file_deletion_delay).each do |count|
104 puts "WARNING: File #{file} already exists and will be over-written in #{@file_deletion_delay - count} seconds. Press ctl-C to Quit"
105 sleep 1
106 end
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
107 end
108
109 FileUtils.rm( file )
110 end
111 end
112
113
c81a83a Ability to define specific tables for export via YAML config file
Chris Cummer authored
114 def generate_mysqldump_str()
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
115 table_str = ( nil != @config[:tables] ) ? @config[:tables].join( ' ' ) : ''
116
117 mysqldump_str = "#{@config[:mysqldump]} -u #{@config[:username]} --compact --compatible=ansi --complete-insert --skip-extended-insert --default-character-set=binary #{@config[:database]} " + table_str
8069631 Fixes password append issue (missing a required space)
Chris Cummer authored
118 mysqldump_str += " -p#{@config[:password]}" if @config[:password]
c81a83a Ability to define specific tables for export via YAML config file
Chris Cummer authored
119
120 return mysqldump_str
121 end
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
122
123
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
124 # Don't attempt to include lines of MySQL data that Sqlite doesn't recognize
125 def contains_disallowed_sql( line )
126 return true if line.include?( 'KEY "' )
127 return true if line.include?( 'UNIQUE KEY ' )
128 return true if line.include?( 'PRIMARY KEY ' )
129 return false
130 end
131
132
133 # Replaces the MySQL terms with Sqlite-friendly terms
134 def translate_sql_differences( line )
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
135 line.gsub!( /UNSIGNED /, '' )
136 line.gsub!( /AUTO_INCREMENT/, ' primary key' )
137 line.gsub!( /SMALLINT\([0-9]*\)/, 'integer' )
138 line.gsub!( /TINYINT\([0-9]*\)/, 'integer' )
139 line.gsub!( /INT\([0-9]*\)/, 'integer' )
140 line.gsub!( /CHARACTER SET [^ ]+/, '' )
141 line.gsub!( /ENUM\([^)]*\)/, 'varchar(255)' )
142 line.gsub!( /ON UPDATE [^,]*/, '' )
143 line.gsub!( /COLLATE [^\s]+/, '')
691c5bf First commit
Chris Cummer authored
144
7922b40 Added hack to add missing comma after text field type.
Chris Cummer authored
145 line.gsub!( /" text/, '" text,' )
146
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
147 return line
148 end
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
149
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
150 # Replace other syntactic differences
151 def translate_character_differences( line )
152 line.gsub!( /\`/, '"' )
153 line.gsub!( /\\'/, '\'\'' )
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
154
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
155 return line
691c5bf First commit
Chris Cummer authored
156 end
157 end
158
159
160 # ------------------------------ Main ------------------------------ #
161
162
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
163 if __FILE__ == $0
164 begin
165 conv = MySQL2SqliteConverter.new(ARGV)
166 rescue => e
167 puts 'ERROR: ' + e
168 puts
eb0462b Now accepts YAML file for configuration; some more error handling around...
Chris Cummer authored
169 puts "Usage: ./mysql2sqlite.rb database_name username password "
170 puts " or: ./mysql2sqlite.rb config_file.yaml"
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
171 else
172 result = conv.mysql_to_sqlite()
691c5bf First commit
Chris Cummer authored
173
452b3f0 @Shumkov refactor initialize and configuration
Shumkov authored
174 puts ( result ) ? "Done." : "Error."
175 end
8ac683d Refactored some of the main class to compartmentalize
Chris Cummer authored
176 exit
691c5bf First commit
Chris Cummer authored
177 end
Something went wrong with that request. Please try again.