Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Added parallel migrations #551

Merged
merged 1 commit into from

6 participants

@mpscholten
Collaborator

This is a port of propelorm/Propel#777 to Propel2, without the backwards compatibility. This feature is request in #145. Basically it's just tracking of all executed migrations instead of the latest one.

Only problem is that I'm unable to write tests. I need to call $migrationManager->setConnections(..) in the tests but I'm not sure where to get the connection settings from. If someone could help me with this, I'll try to write some tests.

I would really like to see this feature in Propel2 because it's really hard to use propel migrations if you have multiple feature branches.

@willdurand
Owner

:+1:

@staabm
Collaborator

please correct me if I am wrong: this PR removes the "old" way of doing migrations and does everything in parallel..?

propelorm/Propel#777 added the parallel part optionally (opt-in like approach)?

@willdurand
Owner

Yep, because you are no more tied to the migrations' timestamps anymore, if I understand correctly.

@mpscholten
Collaborator

added the parallel part optionally (opt-in like approach)?

Yes I dropped this because I think there's no benefit of keeping 2 ways of doing migrations.

@marcj
Owner

Can you also write some tests for that?

@mpscholten
Collaborator

I need to call $migrationManager->setConnections(..) in the tests but I'm not sure where to get the connection settings from. If someone could help me with this, I'll try to write some tests.

I tried already :-) but I don't know how to set up the MigrationManager object in my tests properly. I need to call setConnections after creating the new object, but I don't know where to get the connection informations in the test case.

@mpscholten
Collaborator

But where do I get the generator config? $generatorConfig->getBuildConnections($input->getOption('input-dir'));

@marcj
Owner
$generatorConfig = new GeneratorConfig();

:-)

@mpscholten
Collaborator

Thanks mate :) but $generatorConfig->getBuildConnections() is an empty array now

@marcj
Owner

Yes, because you have to pass a directory of a fixture you want to use.

$generatorConfig->getBuildConnections(__DIR__. '/../../path/to/tests/Fixtures/migration/');
@mpscholten
Collaborator

Thanks, tests are ready now. Time for review

@gharlan

Hmm, all SQLite builds failed..

@mpscholten
Collaborator

Fixed. Maybe later we can port the tests back to propel1 so we can merge propelorm/Propel#777

@mpscholten
Collaborator

Any updates here? :-)

@willdurand
Owner

LGTM @marcj

@willdurand willdurand added this to the alpha-3 milestone
@jaugustin
Collaborator

it's ok for me to,
@willdurand could you merge this PR ?
I am working on a new migration command and some improvement and it would be better if I could use this ;)

@willdurand
Owner

it needs to be rebased.

@mpscholten
Collaborator

Rebase done

@marcj
Owner

Squash please :)

@mpscholten mpscholten Added parallel migrations
Fixed missing method

Added MigrationManager tests and fixed a bug in MigrationManager

Fix for Only variables should be passed by reference

Fixed migrationTableExists on sqlite
a1c9668
@mpscholten
Collaborator

Squashed :shipit:

@marcj marcj merged commit 24bc29e into propelorm:master
@marcj
Owner

Good job :dancers:

This was referenced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 18, 2014
  1. @mpscholten

    Added parallel migrations

    mpscholten authored
    Fixed missing method
    
    Added MigrationManager tests and fixed a bug in MigrationManager
    
    Fix for Only variables should be passed by reference
    
    Fixed migrationTableExists on sqlite
This page is out of date. Refresh to see the latest.
View
2  src/Propel/Generator/Command/MigrationDownCommand.php
@@ -142,7 +142,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
$datasource
));
- $manager->updateLatestMigrationTimestamp($datasource, $previousTimestamp);
+ $manager->removeMigrationTimestamp($datasource, $nextMigrationTimestamp);
if ($input->getOption('verbose')) {
$output->writeln(sprintf(
View
2  src/Propel/Generator/Command/MigrationStatusCommand.php
@@ -131,7 +131,7 @@ protected function execute(InputInterface $input, OutputInterface $output)
' %s %s %s',
$timestamp == $oldestMigrationTimestamp ? '>' : ' ',
$manager->getMigrationClassName($timestamp),
- $timestamp <= $oldestMigrationTimestamp ? '(executed)' : ''
+ !in_array($timestamp, $validTimestamps) ? '(executed)' : ''
));
}
}
View
69 src/Propel/Generator/Manager/MigrationManager.php
@@ -116,14 +116,13 @@ public function getMigrationTable()
return $this->migrationTable;
}
- public function getOldestDatabaseVersion()
+ public function getAllDatabaseVersions()
{
if (!$connections = $this->getConnections()) {
throw new \Exception('You must define database connection settings in a buildtime-conf.xml file to use migrations');
}
- $oldestMigrationTimestamp = null;
- $migrationTimestamps = array();
+ $migrationTimestamps = array();
foreach ($connections as $name => $params) {
$conn = $this->getAdapterConnection($name);
$platform = $this->getGeneratorConfig()->getConfiguredPlatform($conn);
@@ -136,29 +135,27 @@ public function getOldestDatabaseVersion()
try {
$stmt = $conn->prepare($sql);
$stmt->execute();
- if ($migrationTimestamp = $stmt->fetchColumn()) {
- $migrationTimestamps[$name] = $migrationTimestamp;
+
+ while ($migrationTimestamp = $stmt->fetchColumn()) {
+ $migrationTimestamps[] = $migrationTimestamp;
}
} catch (\PDOException $e) {
$this->createMigrationTable($name);
- $oldestMigrationTimestamp = 0;
+ $migrationTimestamps = [];
}
}
- if (null === $oldestMigrationTimestamp && $migrationTimestamps) {
- sort($migrationTimestamps);
- $oldestMigrationTimestamp = array_shift($migrationTimestamps);
- }
+ sort($migrationTimestamps);
- return $oldestMigrationTimestamp;
+ return $migrationTimestamps;
}
public function migrationTableExists($datasource)
{
$conn = $this->getAdapterConnection($datasource);
$sql = sprintf('SELECT version FROM %s', $this->getMigrationTable());
- $stmt = $conn->prepare($sql);
try {
+ $stmt = $conn->prepare($sql);
$stmt->execute();
return true;
@@ -192,24 +189,36 @@ public function createMigrationTable($datasource)
}
}
- public function updateLatestMigrationTimestamp($datasource, $timestamp)
+ public function removeMigrationTimestamp($datasource, $timestamp)
{
$platform = $this->getPlatform($datasource);
$conn = $this->getAdapterConnection($datasource);
- $conn->transaction(function () use ($conn, $platform) {
- $sql = sprintf('DELETE FROM %s', $this->getMigrationTable());
- $stmt = $conn->prepare($sql);
- $stmt->execute();
- $sql = sprintf('INSERT INTO %s (%s) VALUES (?)',
+ $conn->transaction(function () use ($conn, $platform, $timestamp) {
+ $sql = sprintf('DELETE FROM %s WHERE %s = ?',
$this->getMigrationTable(),
$platform->quoteIdentifier('version')
);
$stmt = $conn->prepare($sql);
$stmt->bindParam(1, $timestamp, \PDO::PARAM_INT);
$stmt->execute();
+ $conn->commit();
});
}
+ public function updateLastMigrationTimestamp($datasource, $timestamp)
+ {
+ $platform = $this->getPlatform($datasource);
+ $conn = $this->getAdapterConnection($datasource);
+ $sql = sprintf('INSERT INTO %s (%s) VALUES (?)',
+ $this->getMigrationTable(),
+ $platform->quoteIdentifier('version')
+ );
+ $stmt = $conn->prepare($sql);
+ $stmt->bindParam(1, $timestamp, \PDO::PARAM_INT);
+ $stmt->execute();
+ $conn->commit();
+ }
+
public function getMigrationTimestamps()
{
$path = $this->getWorkingDirectory();
@@ -229,14 +238,7 @@ public function getMigrationTimestamps()
public function getValidMigrationTimestamps()
{
- $oldestMigrationTimestamp = $this->getOldestDatabaseVersion();
- $migrationTimestamps = $this->getMigrationTimestamps();
- // removing already executed migrations
- foreach ($migrationTimestamps as $key => $timestamp) {
- if ($timestamp <= $oldestMigrationTimestamp) {
- unset($migrationTimestamps[$key]);
- }
- }
+ $migrationTimestamps = array_diff($this->getMigrationTimestamps(), $this->getAllDatabaseVersions());
sort($migrationTimestamps);
return $migrationTimestamps;
@@ -249,14 +251,7 @@ public function hasPendingMigrations()
public function getAlreadyExecutedMigrationTimestamps()
{
- $oldestMigrationTimestamp = $this->getOldestDatabaseVersion();
- $migrationTimestamps = $this->getMigrationTimestamps();
- // removing already executed migrations
- foreach ($migrationTimestamps as $key => $timestamp) {
- if ($timestamp > $oldestMigrationTimestamp) {
- unset($migrationTimestamps[$key]);
- }
- }
+ $migrationTimestamps = array_intersect($this->getMigrationTimestamps(), $this->getAllDatabaseVersions());
sort($migrationTimestamps);
return $migrationTimestamps;
@@ -372,4 +367,10 @@ public static function getUser()
return '';
}
+
+ public function getOldestDatabaseVersion()
+ {
+ $versions = $this->getAllDatabaseVersions();
+ return array_pop($versions);
+ }
}
View
156 tests/Propel/Tests/Generator/Manager/MigrationManagerTest.php
@@ -0,0 +1,156 @@
+<?php
+
+namespace Propel\Tests\Generator\Manager;
+
+use Propel\Generator\Config\GeneratorConfig;
+use Propel\Generator\Manager\MigrationManager;
+use Propel\Tests\TestCase;
+
+class MigrationManagerTest extends TestCase
+{
+ /**
+ * @return MigrationManager
+ */
+ private function createMigrationManager(array $migrationTimestamps)
+ {
+ $generatorConfig = new GeneratorConfig();
+
+ $generatorConfig->setBuildProperty('projectDir', __DIR__ . '/../../../../Fixtures/migration/');
+ $generatorConfig->setBuildProperty('buildtimeConfFile', 'runtime-conf.xml');
+ $connections = $generatorConfig->getBuildConnections();
+
+ $migrationManager = $this->getMock('Propel\Generator\Manager\MigrationManager', ['getMigrationTimestamps']);
+ $migrationManager->setGeneratorConfig($generatorConfig);
+ $migrationManager->setConnections($connections);
+ $migrationManager->setMigrationTable('migration');
+ $migrationManager
+ ->expects($this->any())
+ ->method('getMigrationTimestamps')
+ ->will($this->returnValue($migrationTimestamps));
+
+ // make sure there is no other table named migration
+ $migrationManager->getAdapterConnection('migration')->query('DROP TABLE IF EXISTS migration');
+
+
+ return $migrationManager;
+ }
+
+ public function testMigrationTableWillBeCreated()
+ {
+ $migrationManager = $this->createMigrationManager([]);
+ $this->assertFalse($migrationManager->migrationTableExists('migration'));
+
+ $migrationManager->createMigrationTable('migration');
+ $this->assertTrue($migrationManager->migrationTableExists('migration'));
+ }
+
+ public function testGetAllDatabaseVersions()
+ {
+ $databaseVersions = [1, 2, 3];
+ $migrationManager = $this->createMigrationManager([]);
+ $migrationManager->createMigrationTable('migration');
+
+ foreach ($databaseVersions as $version) {
+ $migrationManager->updateLastMigrationTimestamp('migration', $version);
+ }
+
+ $this->assertEquals($databaseVersions, $migrationManager->getAllDatabaseVersions());
+ }
+
+ public function testGetValidMigrationTimestamps()
+ {
+ $localTimestamps = [1, 2, 3, 4];
+ $databaseTimestamps = [1, 2];
+ $expectedMigrationTimestamps = [3, 4];
+
+ $migrationManager = $this->createMigrationManager($localTimestamps);
+ $migrationManager->createMigrationTable('migration');
+
+ foreach ($databaseTimestamps as $timestamp) {
+ $migrationManager->updateLastMigrationTimestamp('migration', $timestamp);
+ }
+
+ $this->assertEquals($expectedMigrationTimestamps, $migrationManager->getValidMigrationTimestamps());
+ }
+
+ public function testRemoveMigrationTimestamp()
+ {
+ $localTimestamps = [1, 2];
+ $databaseTimestamps = [1, 2];
+
+ $migrationManager = $this->createMigrationManager($localTimestamps);
+ $migrationManager->createMigrationTable('migration');
+
+ foreach ($databaseTimestamps as $timestamp) {
+ $migrationManager->updateLastMigrationTimestamp('migration', $timestamp);
+ }
+
+ $this->assertEquals([], $migrationManager->getValidMigrationTimestamps());
+ $migrationManager->removeMigrationTimestamp('migration', 2);
+ $this->assertEquals([2], $migrationManager->getValidMigrationTimestamps());
+ }
+
+ public function testGetAlreadyExecutedTimestamps()
+ {
+ $timestamps = [1, 2];
+
+ $migrationManager = $this->createMigrationManager($timestamps);
+ $migrationManager->createMigrationTable('migration');
+
+ $this->assertEquals([], $migrationManager->getAlreadyExecutedMigrationTimestamps());
+
+ foreach ($timestamps as $timestamp) {
+ $migrationManager->updateLastMigrationTimestamp('migration', $timestamp);
+ }
+
+ $this->assertEquals($timestamps, $migrationManager->getAlreadyExecutedMigrationTimestamps());
+ }
+
+ public function testIsPending()
+ {
+ $localTimestamps = [1, 2];
+
+ $migrationManager = $this->createMigrationManager($localTimestamps);
+ $migrationManager->createMigrationTable('migration');
+
+ $migrationManager->updateLastMigrationTimestamp('migration', 1);
+ $this->assertTrue($migrationManager->hasPendingMigrations());
+
+ $migrationManager->updateLastMigrationTimestamp('migration', 2);
+ $this->assertFalse($migrationManager->hasPendingMigrations());
+ }
+
+ public function testGetOldestDatabaseVersion()
+ {
+ $timestamps = [1, 2];
+ $migrationManager = $this->createMigrationManager($timestamps);
+ $migrationManager->createMigrationTable('migration');
+
+ $this->assertNull($migrationManager->getOldestDatabaseVersion());
+ foreach ($timestamps as $timestamp) {
+ $migrationManager->updateLastMigrationTimestamp('migration', $timestamp);
+ }
+ $this->assertEquals(2, $migrationManager->getOldestDatabaseVersion());
+ }
+
+ public function testGetFirstUpMigrationTimestamp()
+ {
+ $migrationManager = $this->createMigrationManager([1, 2, 3]);
+ $migrationManager->createMigrationTable('migration');
+
+ $migrationManager->updateLastMigrationTimestamp('migration', 1);
+
+ $this->assertEquals(2, $migrationManager->getFirstUpMigrationTimestamp());
+ }
+
+ public function testGetFirstDownMigrationTimestamp()
+ {
+ $migrationManager = $this->createMigrationManager([1, 2, 3]);
+ $migrationManager->createMigrationTable('migration');
+
+ $migrationManager->updateLastMigrationTimestamp('migration', 1);
+ $migrationManager->updateLastMigrationTimestamp('migration', 2);
+
+ $this->assertEquals(2, $migrationManager->getFirstDownMigrationTimestamp());
+ }
+}
Something went wrong with that request. Please try again.