add alter() & preview_alter()

[kimehta]

A method for altering the definitions of tables with data #110

Only supports table comment, and secondary attribute changes (ADD,DROP,CHANGE).
new attributes are nullable if no default value is specified.

This is the best approach I have aside from maybe using a third party library or doing the same parsing on mysql ddl.

[kimehta]

failing because of indentation error. haven't updated test cases for alter yet.

[kimehta]

still need to do something about make methods, add strict mode support to avoid data corruption.

[coveralls]

Coverage decreased (-0.3%) to 90.503% when pulling a8539cd on kimehta:dev into f41d474 on datajoint:master.

[coveralls]

Coverage decreased (-2.5%) to 88.317% when pulling eb64107 on kimehta:dev into f41d474 on datajoint:master.

[dimitri-yatsenko]

add nosetests

[kimehta]

add nosetests

Like this? Should i include tests for all(dj.imported,dj.computed,dj.manual) as well?

datajoint-python/tests/test_relation.py

Lines 246 to 299 in d807be0

	def test_alter_unsupported(self):
	"""alter unsupported(primary, foreign key, index) test on dj.lookup with data"""
	definition = """
	# Basic information about animal subjects used in experiments
	# removed 'unique index (real_id, species)' from secondary
	subject_id :int # {op} no change. unique subject id
	new_pri: int # shouldn't exist
	---
	-> dep: voila # shouldn't matter
	real_id :varchar(40) # real-world name. Omit if the same as subject_id
	species = "mouse" :enum('mouse', 'monkey', 'human')
	date_of_birth :date
	subject_notes :varchar(4000)
	"""
	assert_false(self.subject.preview_alter(definition))
	def test_alter_supported(self):
	"""alter supported test on dj.lookup with data"""
	#asser (TABLE COMMENT,CHANGE, CHANGE(rename), ADD, DROP) in that order
	new_definition = """
	subject_id :int # unique subject id
	---
	real_id :varchar(40) # real-world name. Omit if the same as subject_id
	species = 'monkey' :enum('mouse', 'monkey', 'human', 'fish')
	dob :date #{date_of_birth}
	salary :int # minimum wage for monkeys
	unique index (real_id, species)
	"""
	old_definition = """
	# Basic information about animal subjects used in experiments
	subject_id :int # unique subject id
	---
	real_id :varchar(40) # real-world name. Omit if the same as subject_id
	species = "mouse" :enum("mouse", "monkey", "human")
	date_of_birth :date #{dob}
	subject_notes :varchar(4000)
	unique index (real_id, species)
	"""
	self.subject.alter(new_definition)
	assert_false(self.subject.heading.table_info['comment'])
	assert_true('fish' in self.subject.heading.attributes['species'].type
	and 'monkey' == self.subject.heading.attributes['species'].default.strip('"'))
	assert_true('date_of_birth' not in self.subject.heading.attributes
	and 'dob' in self.subject.heading.attributes)
	assert_true('salary' in self.subject.heading.attributes
	and self.subject.heading.attributes['salary'].type.strip('"') == 'int'
	and not self.subject.heading.attributes['salary'].nullable
	and not self.subject.heading.attributes['salary'].default)
	assert_true('subject_notes' not in self.subject.heading.attributes)
	#revert
	self.subject.alter(old_definition)
	self.subject.insert(self.subject.contents,replace=True)

[kimehta]

the alter commands generated by my code are the same for each version of python.
Travis ci passing python 3.7, failing 3.6. the only difference(i think) is mysql version(5.7 & 5.6 respectively).
I tried running generated alters on db-fiddle and it works fine on both mysql versions.
pymysql version are the same for each python.
the code works fine on my machine for different version of python with datajoint mysql docker.

I am trying to find the cause but if anyone would like to take a look on my forked builds:

there 2 separate alters. 1 to alter and 2nd to reverse the first alter. I am getting error when executing the 2nd alter.

#17 is passing without modifying enum values, but with reorder support.(python 3.6)
#14 is passing with modified enum, but without reorder(after colum) support.(python 3.6)
#15 failing 3.6 with modified enum and reorder support.
python 3.7 is passing everything.
https://travis-ci.com/kimehta/datajoint-python/builds
here "passing" is not throwing mysql error#1292 incorrect date value '0000-00-00' at row 1. I am purposefully failing everything to collect debug info.

oddly enough, as std output in the job log shows, table contents for each python are the same after the 1st alter and there's no '0000-00-00' date value, despite the error.

[dimitri-yatsenko]

closing for a pending resubmission