Skip to content

Commit

Permalink
MONDRIAN: [MONDRIAN-952] Adds support for non-collapsed snowflaked di…
Browse files Browse the repository at this point in the history 
…mensions. Also adds test cases and extends the documentation.

[git-p4: depot-paths = "//open/mondrian/": change = 14725]
  • Loading branch information
@lucboudreau
lucboudreau committed Oct 29, 2011
1 parent 44a78ac commit 22633db
Show file tree
Hide file tree
Showing 15 changed files with 1,137 additions and 97 deletions.
164 changes: 164 additions & 0 deletions doc/aggregate_tables.html
View file
Expand Up @@ -30,6 +30,7 @@ <h3>Contents</h3>
<ol>
<li><a href="#A_simple_aggregate_table">A simple aggregate table</a></li>
<li><a href="#Another_aggregate_table">Another aggregate table</a></li>
<li><a href="#Non_Collapsed_Aggregate_Levels">Non Collapsed Aggregate Levels</a></li>
</ol>
<li><a href="#Defining_aggregate_tables">Defining aggregate tables</a></li>
<li><a href="#Building_aggregates">Building aggregate tables</a></li>
Expand Down Expand Up @@ -302,6 +303,169 @@ <h1>2.2 Another aggregate table<a name="Another_aggregate_table">&nbsp;</a></h1>
table, but all other columns remain in the <code>Product</code> and <code>Mfr</code>

dimension tables.</p>

<!--
############################################
## 2.3 Non-collapsed aggregate levels #####
############################################ -->
<h1>2.3 Non Collapsed Aggregate Levels<a name="Non_Collapsed_Aggregate_Levels">&nbsp;</a></h1>

<p>Another design option for aggregate tables is to use non collapsed levels.
Consider the following table structure:</p>

<p>
<img border="0" src="images/aggregate_tables_4.png"></p>

<p>and the corresponding XML:</p>

<blockquote>

<code>&lt;<a href="#XML_Cube">Cube</a> name=&quot;Sales&quot;&gt;<br>
&nbsp;&nbsp;&lt;<a href="#XML_Table">Table</a> name=&quot;sales&quot;&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggName">AggName</a>
name=&quot;agg_3&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggFactCount">AggFactCount</a> column=&quot;cnt&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggMeasure">AggMeasure</a> name=&quot;[Measures].[Unit
Sales]&quot; column=&quot;sls&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Year]&quot;
column=&quot;yer&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Quarter]&quot;
column=&quot;qtr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Month]&quot;
column=&quot;mth&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Channel.Network].[Brand]&quot;
column=&quot;brn&quot; collapsed=&quot;false&quot;/&gt;<br>



&nbsp;&nbsp;&nbsp; &lt;/<a href="#XML_AggName">AggName</a>&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggName">AggName</a>
name=&quot;agg_3&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggFactCount">AggFactCount</a> column=&quot;cnt&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggMeasure">AggMeasure</a> name=&quot;[Measures].[Unit
Sales]&quot; column=&quot;sls&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Year]&quot;
column=&quot;yer&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Quarter]&quot;
column=&quot;qtr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Time].[Month]&quot;
column=&quot;mth&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="#XML_AggLevel">AggLevel</a> name=&quot;[Channel.Distributor].[Brand]&quot;
column=&quot;brn&quot; collapsed=&quot;false&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;/<a href="#XML_AggName">AggName</a>&gt;<br>


&nbsp;&nbsp;&lt;/<a href="#XML_Table">Table</a>&gt;<br>

<br>
&nbsp; &lt;<a href="schema.html#XML_Dimension">Dimension</a> name=&quot;Channel&quot;&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Hierarchy">Hierarchy</a> hasAll=&quot;true&quot;
name=&quot;Network&quot; primaryKey=&quot;prod&quot; primaryKeyTable=&quot;prod&quot;&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;brn&quot;
rightKey=&quot;brn&quot; rightAlias=&quot;brn_mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;prod&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;brn&quot;
rightKey=&quot;brn&quot; rightAlias=&quot;brn_mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;brn_mfr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;mfr&quot;
rightKey=&quot;mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;brn_mfr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;mfr_net&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Network&quot; table=&quot;mrf_net&quot; column=&quot;net&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Manufacturer&quot; table=&quot;mfr_brn&quot; column=&quot;brn&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Brand&quot; table=&quot;brn_mfr&quot; column=&quot;brn&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Product&quot; table=&quot;prd&quot; column=&quot;brd&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Hierarchy">Hierarchy</a>&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Hierarchy">Hierarchy</a> hasAll=&quot;true&quot;
name=&quot;Distributor&quot; primaryKey=&quot;prod&quot; primaryKeyTable=&quot;prod&quot;&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;brn&quot;
rightKey=&quot;brn&quot; rightAlias=&quot;brn_mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;prod&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;brn&quot;
rightKey=&quot;brn&quot; rightAlias=&quot;brn_mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;brn_mfr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Join">Join</a> leftKey=&quot;mfr&quot;
rightKey=&quot;mfr&quot;&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;brn_mfr&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Table">Table</a>
name=&quot;mfr_dist&quot;/&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Join">Join</a>&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Distributor&quot; table=&quot;mrf_dist&quot; column=&quot;dist&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Manufacturer&quot; table=&quot;mfr_brn&quot; column=&quot;brn&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Brand&quot; table=&quot;brn_mfr&quot; column=&quot;brn&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &lt;<a href="schema.html#XML_Level">Level</a>
name=&quot;Product&quot; table=&quot;prd&quot; column=&quot;brd&quot;/&gt;<br>

&nbsp;&nbsp;&nbsp; &lt;/<a href="schema.html#XML_Hierarchy">Hierarchy</a>&gt;<br>

&nbsp; &lt;/<a href="schema.html#XML_Dimension">Dimension</a>&gt;<br>
<br>
&nbsp;&nbsp;&lt;!-- Rest of the cube definition --&gt;<br>
&lt;/<a href="#XML_Cube">Cube</a>&gt;</code>

</blockquote>

<p>The cube described above uses snoflaked tables for the <code>[Channel]</code> dimension.
That dimension has two separate hierarchies and they both share the table <code>brn_mfr</code>.
Because both hierarchies have a level sharing the same table and column, we can take advantage
of the <code>collapsed</code> option for <code>AggLevel</code> elements, and create an aggregate
table that can be used for both hierarchies simultaneously.
</p>

<p>By setting <code>collapsed</code> to false, Mondrian knows that the keys of the top two levels
of the hierarchies are not part of the aggregate table. They have to be joined when resolving queries
which use that particular aggregate table.</p>

<p>This also works with implicit aggregate rules. Let's assume that the <code>AggName</code>
elements are removed from the schema. If Mondrian scans this table in search of a suitable
aggregate tables to use, it will notice that the top levels are missing, and that it is therefore
dealing with a non-collapsed hierarchy. Mondrian will automatically create an inner join to the
other tables so it can effectively take advantage of the aggregate table.</p>

<p>As with regular <code>AggLevel</code> elements, it is not necessary to include the bottom
levels of the hierarchies. In the example above, we have ommitted the last level, <code>[Product]</code></p>

<!--
######################################
## 3. Defining aggregate tables #####
Expand Down
Binary file added doc/images/aggregate_tables_4.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions src/main/mondrian/olap/Mondrian.xml
View file
Expand Up @@ -1844,13 +1844,22 @@ Revision is $Id$
The name of the Dimension Hierarchy level.
</Doc>
</Attribute>
<Attribute name="collapsed" required="false" type="Boolean" default="true">
<Doc>
Whether this is a collapsed level. The parents of that level
are also present in the aggregation table.
</Doc>
</Attribute>
<Code>
public String getNameAttribute() {
return name;
}
public String getColumnName() {
return column;
}
public boolean isCollapsed() {
return collapsed;
}
</Code>
</Element>

Expand Down
102 changes: 98 additions & 4 deletions src/main/mondrian/rolap/aggmatcher/AggStar.java
View file
Expand Up @@ -10,6 +10,7 @@
package mondrian.rolap.aggmatcher;

import mondrian.olap.*;
import mondrian.olap.MondrianDef.AggLevel;
import mondrian.recorder.MessageRecorder;
import mondrian.resource.MondrianResource;
import mondrian.rolap.*;
Expand All @@ -25,6 +26,7 @@
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.*;

import javax.sql.DataSource;

/**
Expand Down Expand Up @@ -206,6 +208,18 @@ private static void collectLevels(
*/
private final BitKey distinctMeasureBitKey;
private final AggStar.Table.Column[] columns;
/**
* A map of bit positions to columns which need to
* be joined and are not collapsed. If the aggregate table
* includes an {@link AggLevel} element which is not
* collapsed, it will appear in that list. We use this
* list later on to create the join paths in AggQuerySpec.
*/
private final Map<Integer, AggStar.Table.Column> levelColumnsToJoin;

/**
* An approximate number of rows present in this aggregate table.
*/
private final int approxRowCount;

AggStar(
Expand All @@ -222,6 +236,7 @@ private static void collectLevels(
this.distinctMeasureBitKey = bitKey.emptyCopy();
this.aggTable = new AggStar.FactTable(aggTable);
this.columns = new AggStar.Table.Column[star.getColumnCount()];
this.levelColumnsToJoin = new HashMap<Integer, AggStar.Table.Column>();
}

/**
Expand Down Expand Up @@ -404,7 +419,10 @@ public AggStar.Table.Level lookupLevel(final int bitPos) {
* the array of columns.
*/
public AggStar.Table.Column lookupColumn(final int bitPos) {
return columns[bitPos];
if (columns[bitPos] != null) {
return columns[bitPos];
}
return levelColumnsToJoin.get(bitPos);
}

/**
Expand Down Expand Up @@ -653,22 +671,29 @@ private ForeignKey(
*/
final class Level extends Column {
private final RolapStar.Column starColumn;
private final boolean collapsed;

private Level(
final String name,
final MondrianDef.Expression expression,
final int bitPosition,
RolapStar.Column starColumn)
RolapStar.Column starColumn,
boolean collapsed)
{
super(name, expression, starColumn.getDatatype(), bitPosition);
this.starColumn = starColumn;
this.collapsed = collapsed;
AggStar.this.levelBitKey.set(bitPosition);
}

@Override
public SqlStatement.Type getInternalType() {
return starColumn.getInternalType();
}

public boolean isCollapsed() {
return collapsed;
}
}

/** The name of the table in the database. */
Expand Down Expand Up @@ -869,7 +894,8 @@ protected void convertColumns(final RolapStar.Table rTable) {
name,
expression,
bitPosition,
column);
column,
false);
addLevel(level);
}
}
Expand Down Expand Up @@ -1268,8 +1294,76 @@ private void loadLevel(final JdbcSchema.Table.Column.Usage usage) {
name,
expression,
bitPosition,
usage.rColumn);
usage.rColumn,
usage.collapsed);
addLevel(level);
/*
* If we are dealing with a non-collapsed level, we have to
* modify the bit key of the AggStar and create a column
* object for each parent level so that the AggQuerySpec
* can correctly link up to the other tables.
*/
if (!usage.collapsed) {
// We must also update the bit key with
// the parent levels of any non-collapsed level.
RolapLevel parentLevel =
(RolapLevel)usage.level.getParentLevel();
while (!parentLevel.isAll()) {
/*
* Find the bit for this AggStar's bit key for each parent
* level. There is no need to modify the AggStar's bit key
* directly here, because the constructor of Column
* will do that for us later on.
*/
final BitKey bk = AggStar.this.star.getBitKey(
new String[] {
parentLevel.getKeyExp().getTableAlias()},
new String[] {
((MondrianDef.Column)parentLevel.getKeyExp())
.getColumnName()});
final int bitPos = bk.nextSetBit(0);
if (bitPos == -1) {
throw new MondrianException(
"Failed to match non-collapsed aggregate level with a column from the RolapStar.");
}
/*
* Now we will create the Column object to return to the
* AggQuerySpec. We will use the convertTable() method
* because it is convenient and it is capable to convert
* our base table into a series of parent-child tables
* with their join paths figured out.
*/
DimTable columnTable =
convertTable(
AggStar.this.star.getColumn(bitPosition)
.getTable(),
null);
/*
* Make sure to return the last child table, since
* AggQuerySpec will take care of going up the
* parent-child hierarchy and do all the work for us.
*/
while (columnTable.getChildTables().size() > 0) {
columnTable = columnTable.getChildTables().get(0);
}
final DimTable finalColumnTable = columnTable;
levelColumnsToJoin.put(
bitPos,
new Column(
((MondrianDef.Column)parentLevel.getKeyExp())
.getColumnName(),
parentLevel.getKeyExp(),
AggStar.this.star.getColumn(bitPos).getDatatype(),
bitPos)
{
public Table getTable() {
return finalColumnTable;
}
});
// Do the next parent level.
parentLevel = (RolapLevel) parentLevel.getParentLevel();
}
}
}

public void print(final PrintWriter pw, final String prefix) {
Expand Down

0 comments on commit 22633db

Please sign in to comment.