Skip to content

Commit

Permalink
MONDRIAN
Browse files Browse the repository at this point in the history 
       Added IgnoreMap to DefaultRulesSchema which optionally allows one 
       to specify one or more regular expressions such that if a
       candidate aggregate table column matches then the column is not
       allowed to match against a level or measure fact table column.
       Documentation is on the developer_notes page.
       The Mondrian DefaultRules do not include the IgnoreMap, but if
       you wish to create your own you can include an IgnoreMap element.
       This is useful if your aggregates are created using 
       materialized views since they always have supporting columns.

[git-p4: depot-paths = "//open/mondrian/": change = 7000]
  • Loading branch information
Richard Emberson committed Jun 23, 2006
1 parent c1d3f77 commit ef71ccb
Show file tree
Hide file tree
Showing 4 changed files with 206 additions and 7 deletions.
97 changes: 90 additions & 7 deletions doc/developer_notes.html
View file
Expand Up @@ -171,23 +171,26 @@ <h2><a name="Agg_default_rules">Default aggregate table recognition rules</a></h
are specified by creating an instance of the rule schema
found in the file:
<code>
MONDRIAN_HOME/src/main/rolap.aggmatcher/DefaultRulesSchema.xml.
MONDRIAN_HOME/src/main/rolap/aggmatcher/DefaultRulesSchema.xml.
</code>
The instance of this schema that is built into the
<code>
mondrian.jar
</code>
after a build is in the same directory,
<code>
MONDRIAN_HOME/src/main/rolap.aggmatcher/DefaultRules.xml.
MONDRIAN_HOME/src/main/rolap/aggmatcher/DefaultRules.xml.
</code>
</p>
<p>
There are five different default rules that are used to match and map
There are six different default rules that are used to match and map
a candidate
aggregate table: table name, fact count column, foreign key column,
level column and measure column. All of these rules are defined
by creating an instance of the DefaultRulesSchema.xml grammar.
aggregate table: table name, ignore column, fact count column, foreign
key column, level column and measure column. All of these rules are
defined by creating an instance of the DefaultRulesSchema.xml grammar.
The DefaultRulesSchema.xml instance, the DefaultRules.xml file mentioned
above, that by default is built as part of the mondrian.jar does not
contain an ignore column rule.
This grammar has base/supporting classes that are common to the above
rules. In
<code>
Expand Down Expand Up @@ -323,14 +326,17 @@ <h2><a name="Agg_default_rules">Default aggregate table recognition rules</a></h
as the regular expression.
</p>
<p>
Both the level and measure column matching elements have one or more
The ignore, asdf level and measure column matching elements have one or more
<code>
Regex
</code>
child elements. These allow for specifying multiple possible matches
(if any match, then its a match).
The
<code>
IgnoreMap,
</code>
<code>
LevelMap
</code>
and
Expand Down Expand Up @@ -383,6 +389,83 @@ <h2><a name="Agg_default_rules">Default aggregate table recognition rules</a></h
<p>
The
<code>
IgnoreMap
</code>
element has NO template parameter names. Each
<code>
Regex
</code>
value is simply a regular expression. As an example (Mondrian by
default does not include an
<code>
IgnoreMap
</code>
by default), a regular expression that matches all aggregate table columns
then end with '_DO_NOT_USE' would be:
</p>
<blockquote>
.*_DO_NOT_USE<br>
</blockquote>
</p>
<p>
One might want to use an
<code>
IgnoreMap
</code>
element to filter out aggregate columns if, for example, the aggregate
table is a materialized view, since with each "normal" column
of such a materialized view there is an associated support column
used by the database which has no significance to Mondrian.
In the process of recognizing aggregate tables, Mondrian logs
a warning message for each column whose use can not be determined.
Materialized views have so many of these support columns
that if, in fact, there was a column whose use was desired
but was not recognized (for instance, the column name is
misspelt) all of the materialized view column warning message
mask the one warning message that one really needs to see.
</p>
<p>
The
<code>
IgnoreMap
</code>
regular expressions are applied before any of the other column
matching actions. If one sets the
<code>
IgnoreMap
</code>
regular expression to, for example,
<blockquote>
.*<br>
</blockquote>
</p>
<p>
then all columns are marked as "ignore" and there are no other
columns left to match anything else. One must be very careful
when choosing
<code>
IgnoreMap
</code>
regular expressions not just for your current columns but for
columns that might be created in the future. Its best to document
this usage in your organization.
</p>
<p>
The following is what the element might look like in a
DefaultRules.xml file:
<blockquote>
<pre>
&lt;IgnoreMap id="ixx" &gt;
&lt;Regex id="physical" charcase="ignore"&gt;
.*_DO_NOT_USE
&lt;/Regex&gt;
&lt;/IgnoreMap&gt;
</pre>
</blockquote>
</p>
<p>
The
<code>
LevelMap
</code>
element has the four template parameter names (hardcoded):
Expand Down
14 changes: 14 additions & 0 deletions src/main/mondrian/rolap/aggmatcher/DefaultRecognizer.java
View file
Expand Up @@ -53,11 +53,25 @@ DefaultRules getRules() {
* Get the Matcher to be used to match columns to be ignored.
*/
protected Recognizer.Matcher getIgnoreMatcher() {
Recognizer.Matcher matcher = getRules().getIgnoreMatcher();
if (matcher != null) {
// an ignore matcher was defined in the recognizer xml file.
return matcher;
} else {
// return a do-noting matcher
return new Recognizer.Matcher() {
public boolean matches(String name) {
return false;
}
};
}
/*
return new Recognizer.Matcher() {
public boolean matches(String name) {
return false;
}
};
*/
}

/**
Expand Down
17 changes: 17 additions & 0 deletions src/main/mondrian/rolap/aggmatcher/DefaultRules.java
View file
Expand Up @@ -249,6 +249,7 @@ protected static DOMWrapper makeDOMWrapper(
private final DefaultDef.AggRules rules;
private final Map factToPattern;
private final Map foreignKeyMatcherMap;
private Recognizer.Matcher ignoreMatcherMap;
private Recognizer.Matcher factCountMatcher;
private String tag;

Expand Down Expand Up @@ -324,6 +325,22 @@ public Recognizer.Matcher getTableMatcher(final String tableName) {
* Gets the {@link mondrian.rolap.aggmatcher.Recognizer.Matcher} for the
* fact count column.
*/
public Recognizer.Matcher getIgnoreMatcher() {
if (ignoreMatcherMap == null) {
// get default AggRule
DefaultDef.AggRule rule = getAggRule();
DefaultDef.IgnoreMap ignoreMatch = rule.getIgnoreMap();
ignoreMatcherMap = ignoreMatch.getMatcher();
}
return ignoreMatcherMap;
}

/**
* Gets the {@link mondrian.rolap.aggmatcher.Recognizer.Matcher} for
* columns that should be ignored.
*
* @return
*/
public Recognizer.Matcher getFactCountMatcher() {
if (factCountMatcher == null) {
// get default AggRule
Expand Down
85 changes: 85 additions & 0 deletions src/main/mondrian/rolap/aggmatcher/DefaultRulesSchema.xml
View file
Expand Up @@ -55,6 +55,9 @@ Revision is $Id$
<Array name="measureMaps" type="MeasureMap">
<Doc>All shared MeasureMap.</Doc>
</Array>
<Array name="ignoreMaps" type="IgnoreMap">
<Doc>All shared IgnoreMap.</Doc>
</Array>
<Array name="aggRules" type="AggRule" min="1">
<Doc>
All AggRules (at least one).
Expand Down Expand Up @@ -90,6 +93,7 @@ Revision is $Id$
validate(tableMatches, msgRecorder);
validate(levelMaps, msgRecorder);
validate(measureMaps, msgRecorder);
validate(ignoreMaps, msgRecorder);
validate(aggRules, msgRecorder);
} finally {
msgRecorder.popContextName();
Expand All @@ -105,6 +109,13 @@ Revision is $Id$
}
}
public boolean hasIgnoreMap(String id) {
return (lookupIgnoreMap(id) != null);
}
public IgnoreMap lookupIgnoreMap(String id) {
return (IgnoreMap) lookupBase(id, ignoreMaps);
}
public boolean hasFactCountMatch(String id) {
return (lookupFactCountMatch(id) != null);
}
Expand Down Expand Up @@ -157,6 +168,9 @@ Revision is $Id$
return null;
}
public IgnoreMap[] getIgnoreMaps() {
return ignoreMaps;
}
public FactCountMatch[] getFactCountMatches() {
return factCountMatches;
}
Expand Down Expand Up @@ -873,6 +887,23 @@ Revision is $Id$
int end = 0;
int previousEnd = 0;
int start = template.indexOf("${", end);
// if no templateNames, then there better not be
// any ${}s
if (templateNames.length == 0) {
if (start == -1) {
// everything is ok
templateParts = new String[1];
templateParts[0] = template;
templateNamePos = new int[0];
} else {
String msg = "Bad template \"" +
template +
"\", no ${} entries but there are "+
"template names" ;
msgRecorder.reportError(msg);
}
return;
}
while (count < MAX_SIZE) {
if (start == -1) {
if (count == 0) {
Expand Down Expand Up @@ -1231,6 +1262,26 @@ Revision is $Id$
</Code>
</Element>

<!-- IgnoreMapRef ================================================== -->
<Element type="IgnoreMapRef" class="Ref" >
<Code>
public void validate(final AggRules rules,
final mondrian.recorder.MessageRecorder msgRecorder) {
msgRecorder.pushContextName(getName());
try {
if (! rules.hasIgnoreMap(getRefId())) {
String msg = "No IgnoreMap has id equal to refid \"" +
getRefId() +
"\"";
msgRecorder.reportError(msg);
}
} finally {
msgRecorder.popContextName();
}
}
</Code>
</Element>

<!-- FactCountMatchRef =============================================== -->
<Element type="FactCountMatchRef" class="Ref" >
<Code>
Expand Down Expand Up @@ -1355,6 +1406,23 @@ Revision is $Id$
]]></Code>
</Element>

<Element type="IgnoreMap" class="RegexMapper" >
<Doc>
This is the template used to specify columns to be ignored.
There are NO template names. One simply uses a regular
expression.
</Doc>
<Code><![CDATA[
private static final String[] TEMPLATE_NAMES = new String[] { };
protected String[] getTemplateNames() {
return TEMPLATE_NAMES;
}
protected Recognizer.Matcher getMatcher() {
return getMatcher(new String[]{});
}
]]></Code>
</Element>

<!-- AggRule ===================================================== -->
<Element type="AggRule" class="Base">
<Doc>
Expand All @@ -1380,6 +1448,15 @@ Revision is $Id$
</Doc>
</Attribute>

<Object name="ignoreMap" type="IgnoreMap" required="false">
<Doc>
</Doc>
</Object>
<Object name="ignoreMapRef" type="IgnoreMapRef" required="false">
<Doc>
</Doc>
</Object>

<Object name="factCountMatch" type="FactCountMatch" required="false">
<Doc>
</Doc>
Expand Down Expand Up @@ -1467,6 +1544,11 @@ Revision is $Id$
final mondrian.recorder.MessageRecorder msgRecorder) {
msgRecorder.pushContextName(getName());
try {
if (isRef(rules, msgRecorder, ignoreMap,
ignoreMapRef, "IgnoreMap")) {
ignoreMap =
rules.lookupIgnoreMap(ignoreMapRef.getRefId());
}
if (isRef(rules, msgRecorder, factCountMatch,
factCountMatchRef, "FactCountMatch")) {
factCountMatch = rules.lookupFactCountMatch(
Expand Down Expand Up @@ -1516,6 +1598,9 @@ Revision is $Id$
public MeasureMap getMeasureMap() {
return measureMap;
}
public IgnoreMap getIgnoreMap() {
return ignoreMap;
}
]]></Code>
</Element>

Expand Down

0 comments on commit ef71ccb

Please sign in to comment.