Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

split the references into 2 tables to allow using defered FKs in the future #75

Merged
merged 1 commit into from Jun 7, 2013
Merged

split the references into 2 tables to allow using defered FKs in the future #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
@@ -1,6 +1,19 @@
Changelog
=========

* **2013-06-07**: [#75] split the references into 2 tables
* This splits the phpcr_nodes_foreignkeys into two separate tables
* Improves performance and allows using native deferred FK capabilities in the future
* Migration steps
* run ``bin/jackalope jackalope:init:dbal --dump-sql``
* Copy and execute all tables, indexes etc related to ``phpcr_nodes_references`` and ``phpcr_nodes_weakreferences``.
* Run the following SQL statements:
* INSERT INTO phpcr_nodes_references ( source_id, source_property_name, target_id )
SELECT source_id, source_property_name, target_id FROM phpcr_nodes_foreignkeys WHERE type = 9;
* INSERT INTO phpcr_nodes_weakreferences ( source_id, source_property_name, target_id )
SELECT source_id, source_property_name, target_id FROM phpcr_nodes_foreignkeys WHERE type = 10;
* DROP TABLE phpcr_nodes_foreignkeys;

* **2013-06-01**: [#109] ensure data is stored as UTC and returned in the default TZ
* This enables consistent search query behavior regardless of the timezone used
* Any existing stored nodes need to be modified, so that they are stored again
Expand Down
137 changes: 74 additions & 63 deletions src/Jackalope/Transport/DoctrineDBAL/Client.php
View file
Open in desktop
Expand Up @@ -140,6 +140,14 @@ class Client extends BaseTransport implements QueryTransport, WritingInterface,
*/
private $referencesToUpdate = array();

/**
* @var array
*/
private $referenceTables = array(
PropertyType::REFERENCE => 'phpcr_nodes_references',
PropertyType::WEAKREFERENCE => 'phpcr_nodes_weakreferences',
);

/**
* @var array
*/
Expand Down Expand Up @@ -636,15 +644,17 @@ private function syncBinaryData($nodeId, $binaryData)
}
}

private function syncForeignKeys()
private function syncReferences()
{
// do not update references that are going to be deleted anyways
$toUpdate = array_diff_assoc($this->referencesToUpdate, $this->referencesToDelete);

if ($this->referencesToUpdate) {
$query = 'DELETE FROM phpcr_nodes_foreignkeys WHERE source_id IN (?)';
// do not update references that are going to be deleted anyways
$toUpdate = array_diff_assoc($this->referencesToUpdate, $this->referencesToDelete);

try {
$this->conn->executeUpdate($query, array(array_keys($toUpdate)), array(Connection::PARAM_INT_ARRAY));
foreach ($this->referenceTables as $table) {
$query = "DELETE FROM $table WHERE source_id IN (?)";
$this->conn->executeUpdate($query, array(array_keys($toUpdate)), array(Connection::PARAM_INT_ARRAY));
}
} catch (DBALException $e) {
throw new RepositoryException('Unexpected exception while cleaning up after saving', $e->getCode(), $e);
}
Expand All @@ -653,18 +663,15 @@ private function syncForeignKeys()
foreach ($references['properties'] as $name => $data) {
foreach ($data['values'] as $value) {
try {
$targetId = $this->pathExists(self::getNodePathForIdentifier($value));

$params = array(
'source_id' => $nodeId,
'source_property_name' => $name,
'target_id' => $targetId,
'type' => $data['type'],
'target_id' => $this->pathExists(self::getNodePathForIdentifier($value)),
);

$this->conn->insert('phpcr_nodes_foreignkeys', $params);
$this->conn->insert($this->referenceTables[$data['type']], $params);
} catch (ItemNotFoundException $e) {
if (PropertyType::REFERENCE == $data['type']) {
if (PropertyType::REFERENCE === $data['type']) {
throw new ReferentialIntegrityException(
"Trying to store reference to non-existant node with path '$value' in node '{$references['path']}' property '$name'."
);
Expand All @@ -675,38 +682,49 @@ private function syncForeignKeys()
}
}

// TODO on RDBMS that support deferred FKs we could skip this step
if ($this->referencesToDelete) {
$nodeIds = array_keys($this->referencesToDelete);
$params = array(PropertyType::REFERENCE, $nodeIds, $nodeIds);
$params = array(array_keys($this->referencesToDelete));

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think a few comments would help a lot here to understand the code. if i get it right, we first delete all references originating from nodes we deleted, as we can always delete them. then we check if there are references pointing to the node being deleted, which after we deleted source references from this changeset means there still are references from nodes that are not deleted in the same save.

what about deleting the weak references too?

// due to the outer join we cannot filter on workspace_name, but this is ok
// remove all PropertyType::REFERENCE with a source_id on a deleted node
try {
$query = "DELETE FROM phpcr_nodes_references WHERE source_id IN (?)";
$this->conn->executeUpdate($query, $params, array(Connection::PARAM_INT_ARRAY));
} catch (DBALException $e) {
throw new RepositoryException('Unexpected exception while cleaning up deleted nodes', $e->getCode(), $e);
}

// ensure that there are no PropertyType::REFERENCE pointing to nodes that will be deleted
// Note: due to the outer join we cannot filter on workspace_name, but this is ok
// since within a transaction there can never be missing referenced nodes within the current workspace
// make sure the target node is not in the list of nodes being deleted, to allow deletion in same request
$query = 'SELECT fk.target_id
FROM phpcr_nodes_foreignkeys fk
LEFT OUTER JOIN phpcr_nodes n ON fk.target_id = n.id
WHERE fk.type = ?
AND fk.target_id IN (?)
AND fk.source_id NOT IN (?)';

$stmt = $this->conn->executeQuery($query, $params, array(\PDO::PARAM_INT, Connection::PARAM_INT_ARRAY, Connection::PARAM_INT_ARRAY));
$query = 'SELECT DISTINCT r.target_id
FROM phpcr_nodes_references r
LEFT OUTER JOIN phpcr_nodes n ON r.target_id = n.id
WHERE r.target_id IN (?)';

$stmt = $this->conn->executeQuery($query, $params, array(Connection::PARAM_INT_ARRAY));
$missingTargets = $stmt->fetchAll(\PDO::FETCH_COLUMN);
if ($missingTargets) {
$paths = array();
foreach ($missingTargets as $id) {
$paths[] = $this->referencesToDelete[$id];
if (isset($this->referencesToDelete[$id])) {
$paths[] = $this->referencesToDelete[$id];
}
}

throw new ReferentialIntegrityException("Cannot delete '".implode("', '", $paths)."': A reference points to this node or a subnode");
}

$query = 'DELETE FROM phpcr_nodes_foreignkeys WHERE target_id IN (?)';
// clean up all references
try {
$this->conn->executeUpdate($query, array($nodeIds), array(Connection::PARAM_INT_ARRAY));
foreach ($this->referenceTables as $table) {
$query = "DELETE FROM $table WHERE target_id IN (?)";
$this->conn->executeUpdate($query, $params, array(Connection::PARAM_INT_ARRAY));
}
} catch (DBALException $e) {
throw new RepositoryException('Unexpected exception while cleaning up deleted nodes', $e->getCode(), $e);
}

}
}

Expand Down Expand Up @@ -1132,18 +1150,19 @@ protected function deleteNode($path)
}

$nodeId = $this->pathExists($path);

if (!$nodeId) {
throw new ItemNotFoundException("No node found at ".$path);
}

$params = array($path, $path."/%", $this->workspaceName);

// TODO on RDBMS that support deferred FKs we could skip this step
$query = 'SELECT id, path FROM phpcr_nodes WHERE (path = ? OR path LIKE ?) AND workspace_name = ?';
$stmt = $this->conn->executeQuery($query, $params);
$this->referencesToDelete += $stmt->fetchAll(\PDO::FETCH_UNIQUE | \PDO::FETCH_COLUMN);
$query = 'DELETE FROM phpcr_nodes WHERE (path = ? OR path LIKE ?) AND workspace_name = ?';

try {
$query = 'DELETE FROM phpcr_nodes WHERE (path = ? OR path LIKE ?) AND workspace_name = ?';
$this->conn->executeUpdate($query, $params);
} catch (DBALException $e) {
throw new RepositoryException('Unexpected exception while deleting node ' . $path, $e->getCode(), $e);
Expand Down Expand Up @@ -1190,36 +1209,29 @@ protected function deleteProperty($path)
throw new ItemNotFoundException("No item found at ".$path);
}

if ('/' == $nodePath) {
// root node is a special case
$propertyName = substr($path, 1);
} else {
$propertyName = str_replace($nodePath . '/', '', $path);
}

$query = 'SELECT props FROM phpcr_nodes WHERE id = ?';
$xml = $this->conn->fetchColumn($query, array($nodeId));

$dom = new \DOMDocument('1.0', 'UTF-8');
$dom->loadXml($xml);

$found = false;
$propertyName = PathHelper::getNodeName($path);
foreach ($dom->getElementsByTagNameNS('http://www.jcp.org/jcr/sv/1.0', 'property') as $propertyNode) {
if ($propertyName == $propertyNode->getAttribute('sv:name')) {
$found = true;
// would be nice to have the property object to ask for type
// but its in state deleted, would mean lots of refactoring
if ($propertyNode->hasAttribute('sv:type') &&
('reference' == strtolower($propertyNode->getAttribute('sv:type'))
|| 'weakreference' == strtolower($propertyNode->getAttribute('sv:type'))
)
) {
$query = 'DELETE FROM phpcr_nodes_foreignkeys
WHERE source_id = ? AND source_property_name = ?';
try {
$this->conn->executeUpdate($query, array($nodeId, $propertyName));
} catch (DBALException $e) {
throw new RepositoryException("Unexpected exception while deleting foreign key of reference property $path", $e->getCode(), $e);
if ($propertyNode->hasAttribute('sv:type')) {
$type = strtolower($propertyNode->getAttribute('sv:type'));
if (in_array($type, array('reference', 'weakreference'))) {
$table = $this->referenceTables['reference' === $type ? PropertyType::REFERENCE : PropertyType::WEAKREFERENCE];
try {
$query = "DELETE FROM $table WHERE source_id = ? AND source_property_name = ?";
$this->conn->executeUpdate($query, array($nodeId, $propertyName));
} catch (DBALException $e) {
throw new RepositoryException('Unexpected exception while cleaning up deleted nodes', $e->getCode(), $e);
}
}
}
$propertyNode->parentNode->removeChild($propertyNode);
Expand Down Expand Up @@ -1846,8 +1858,8 @@ public function getBinaryStream($path)
$this->assertLoggedIn();

$nodePath = PathHelper::getParentPath($path);
$propertyName = ltrim(str_replace($nodePath, '', $path), '/'); // i dont know why trim here :/
$nodeId = $this->pathExists($nodePath);
$propertyName = PathHelper::getNodeName($path);

$data = $this->conn->fetchAll(
'SELECT data, idx FROM phpcr_binarydata WHERE node_id = ? AND property_name = ? AND workspace_name = ?',
Expand Down Expand Up @@ -1896,7 +1908,8 @@ public function query(Query $query)

if (null !== $offset && null == $limit &&
($this->conn->getDatabasePlatform() instanceof MySqlPlatform
|| $this->conn->getDatabasePlatform() instanceof SqlitePlatform)
|| $this->conn->getDatabasePlatform() instanceof SqlitePlatform
)
) {
$limit = PHP_INT_MAX;
}
Expand Down Expand Up @@ -2068,21 +2081,19 @@ public function getWeakReferences($path, $name = null)
private function getNodeReferences($path, $name = null, $weakReference = false)
{
$targetId = $this->pathExists($path);
$params = array($targetId);

$type = $weakReference ? PropertyType::WEAKREFERENCE : PropertyType::REFERENCE;

$query = "SELECT CONCAT(n.path, '/', fk.source_property_name) as path, fk.source_property_name FROM phpcr_nodes n".
' INNER JOIN phpcr_nodes_foreignkeys fk ON n.id = fk.source_id'.
' WHERE fk.target_id = ? AND fk.type = ?';
$properties = $this->conn->fetchAll($query, array($targetId, $type));

$references = array();
foreach ($properties as $property) {
if (null === $name || $property['source_property_name'] == $name) {
$references[] = $property['path'];
}
$table = $weakReference ? $this->referenceTables[PropertyType::WEAKREFERENCE] : $this->referenceTables[PropertyType::REFERENCE];
$query = "SELECT CONCAT(n.path, '/', r.source_property_name) FROM phpcr_nodes n
INNER JOIN $table r ON n.id = r.source_id
WHERE r.target_id = ?";
if (null !== $name) {
$query.= " AND source_property_name = ?";
$params[] = $name;
}
return $references;

$stmt = $this->conn->executeQuery($query, $params);
return $stmt->fetchAll(\PDO::FETCH_COLUMN);
}

/**
Expand Down Expand Up @@ -2170,7 +2181,7 @@ public function prepareSave()
*/
public function finishSave()
{
$this->syncForeignKeys();
$this->syncReferences();
$this->referencesToUpdate = $this->referencesToDelete = array();
$this->conn->commit();
}
Expand Down
26 changes: 18 additions & 8 deletions src/Jackalope/Transport/DoctrineDBAL/RepositorySchema.php
View file
Open in desktop
Expand Up @@ -58,14 +58,24 @@ static public function create()
$binary->setPrimaryKey(array('id'));
$binary->addUniqueIndex(array('node_id', 'property_name', 'workspace_name', 'idx'));

$foreignKeys = $schema->createTable('phpcr_nodes_foreignkeys');
$foreignKeys->addColumn('source_id', 'integer');
$foreignKeys->addColumn('source_property_name', 'string', array('length' => 220));
$foreignKeys->addColumn('target_id', 'integer');
$foreignKeys->addColumn('type', 'smallint');
$foreignKeys->setPrimaryKey(array('source_id', 'source_property_name', 'target_id'));
$foreignKeys->addIndex(array('target_id'));
$foreignKeys->addForeignKeyConstraint($nodes, array('source_id'), array('id'), array('onDelete' => 'CASCADE'));
$references = $schema->createTable('phpcr_nodes_references');
$references->addColumn('source_id', 'integer');
$references->addColumn('source_property_name', 'string', array('length' => 220));
$references->addColumn('target_id', 'integer');
$references->setPrimaryKey(array('source_id', 'source_property_name', 'target_id'));
$references->addIndex(array('target_id'));
$references->addForeignKeyConstraint($nodes, array('source_id'), array('id'), array('onDelete' => 'CASCADE'));
// TODO: this should be reenabled on RDBMS with deferred FK support
//$references->addForeignKeyConstraint($nodes, array('target_id'), array('id'));

$weakreferences = $schema->createTable('phpcr_nodes_weakreferences');
$weakreferences->addColumn('source_id', 'integer');
$weakreferences->addColumn('source_property_name', 'string', array('length' => 220));
$weakreferences->addColumn('target_id', 'integer');
$weakreferences->setPrimaryKey(array('source_id', 'source_property_name', 'target_id'));
$weakreferences->addIndex(array('target_id'));
$weakreferences->addForeignKeyConstraint($nodes, array('source_id'), array('id'), array('onDelete' => 'CASCADE'));
$weakreferences->addForeignKeyConstraint($nodes, array('target_id'), array('id'), array('onDelete' => 'CASCADE'));

$types = $schema->createTable('phpcr_type_nodes');
$types->addColumn('node_type_id', 'integer', array('autoincrement' => true));
Expand Down