Skip to content

Commit

Permalink
MONDRIAN: First cut at supporting large dimensions by automatically i…
Browse files Browse the repository at this point in the history
…terating (contributed by Luis F. Canals and Jorge Lopez).

    Many changes to internal APIs to return lists of members rather than arrays.
    Obsolete property LargeDimensionThreshold.
    Add schema property Dimension.highCardinality.
    Reinstate (but not enable) code to optimize lists of constraints for aggregate distinct count.
    Introduce MemberListCalc and TupleListCalc, subclasses of ListCalc for stronger typing.
    Update copyright notices & trim spaces.

[git-p4: depot-paths = "//open/mondrian/": change = 11049]
  • Loading branch information
julianhyde committed May 11, 2008
1 parent cd2f797 commit 3b50bf1
Show file tree
Hide file tree
Showing 102 changed files with 5,440 additions and 2,068 deletions.
11 changes: 0 additions & 11 deletions doc/configuration.html
Expand Up @@ -163,17 +163,6 @@ <h3>1.1 Property list<a name="Property_list">&nbsp;</a></h3>
If evaluation exceeds this depth (for example, while evaluating a very
complex calculated member), Mondrian will throw an error.</td>
</tr>
<tr>
<td>
<code><a href="api/mondrian/olap/MondrianProperties.html#LargeDimensionThreshold">
mondrian.rolap.LargeDimension Threshold</a></code></td>
<td>int</td>
<td>100</td>
<td>Determines when a dimension is considered &quot;large&quot;. If a dimension has
more than this number of members, Mondrian uses a
<a href="api/mondrian/rolap/SmartMemberReader.html">smart member reader</a>.</td>

</tr>
<tr>
<td>
<code><a href="api/mondrian/olap/MondrianProperties.html#SparseSegmentCountThreshold">
Expand Down
6 changes: 2 additions & 4 deletions doc/schema.html
Expand Up @@ -5,7 +5,7 @@
== Agreement, available at the following URL:
== http://www.opensource.org/licenses/cpl.html.
== Copyright (C) 2001-2002 Kana Software, Inc.
== Copyright (C) 2002-2007 Julian Hyde
== Copyright (C) 2002-2008 Julian Hyde
== All Rights Reserved.
== You must accept the terms of that agreement to use this software.
== jhyde, 24 September, 2002
Expand Down Expand Up @@ -913,9 +913,7 @@ <h1>4.2 Join optimization<a name="Join_optimization">&nbsp;</a></h1>
queries:</p>

<ul>
<li>If a dimension has a small number of members, Mondrian reads it into a cache on first use. See the <a
href="api/mondrian/olap/MondrianProperties.html#getLargeDimensionThreshold%28%29"
class="included_content_links">mondrian.rolap.LargeDimensionThreshold</a> property.</li>
<li>TODO: describe large dimension support</li>
<li>If a dimension (or, more precisely, the level of the dimension being accessed) is in the fact table, Mondrian does
not perform a join.</li>
<li>If two dimensions access the same table via the same join path, Mondrian only joins them once. For example,
Expand Down
10 changes: 2 additions & 8 deletions mondrian.properties
Expand Up @@ -3,10 +3,9 @@
# Agreement, available at the following URL:
# http://www.opensource.org/licenses/cpl.html.
# Copyright (C) 2001-2002 Kana Software, Inc.
# Copyright (C) 2001-2006 Julian Hyde and others.
# Copyright (C) 2001-2008 Julian Hyde and others.
# All Rights Reserved.
# You must accept the terms of that agreement to use this software.
# jhyde, 31 October, 2001

###############################################################################
# Environment
Expand Down Expand Up @@ -151,12 +150,6 @@ RoleXX='California manager';
# "mondrian.sql" in DEBUG mode. This category can be configured to log in a
# separate file or as part of the debug output.

###############################################################################
# Property which determines when a dimension is considered "large".
# If a dimension has more than this number of members, Mondrian uses a
# smart member reader (mondrian.rolap.SmartMemberReader).
#mondrian.rolap.LargeDimensionThreshold=100

###############################################################################
# Maximum number of simultaneous queries the system will allow.
#
Expand Down Expand Up @@ -442,3 +435,4 @@ mondrian.rolap.iterationLimit=0
#mondrian.rolap.EnableRolapCubeMemberCache=true

# End mondrian.properties

8 changes: 5 additions & 3 deletions src/main/mondrian/calc/IterCalc.java
Expand Up @@ -3,15 +3,14 @@
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2007 Julian Hyde
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc;

import mondrian.olap.Evaluator;

import java.util.List;
import mondrian.olap.Member;

/**
* Expression which evaluates a set of members or tuples to an Iterable.
Expand All @@ -30,6 +29,9 @@ public interface IterCalc extends Calc {
* @return An Iterable of members or tuples, never null.
*/
Iterable evaluateIterable(Evaluator evaluator);
Iterable<Member> evaluateMemberIterable(Evaluator evaluator);
Iterable<Member[]> evaluateTupleIterable(Evaluator evaluator);

}

// End IterCalc.java
6 changes: 3 additions & 3 deletions src/main/mondrian/calc/ListCalc.java
Expand Up @@ -3,16 +3,16 @@
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2007 Julian Hyde
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc;

import mondrian.olap.Evaluator;

import java.util.List;

import mondrian.olap.Evaluator;

/**
* Expression which evaluates a set of members or tuples to a list.
*
Expand Down
40 changes: 40 additions & 0 deletions src/main/mondrian/calc/MemberListCalc.java
@@ -0,0 +1,40 @@
/*
// $Id$
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc;

import java.util.List;

import mondrian.olap.Evaluator;
import mondrian.olap.Member;

// End ListCalc.java

/**
* Expression which evaluates a set of members or tuples to a list.
*
* @author jhyde
* @version $Id$
* @since Sep 27, 2005
*/
public interface MemberListCalc extends ListCalc {
/**
* Evaluates an expression to yield a list of members.
*
* <p>The list is immutable if {@link #getResultStyle()} yields
* {@link mondrian.calc.ResultStyle#MUTABLE_LIST}. Otherwise,
* the caller must not modify the list.
*
* @param evaluator Evaluation context
* @return A list of members, never null.
*/
List<Member> evaluateMemberList(Evaluator evaluator);
}

// End MemberListCalc.java
42 changes: 42 additions & 0 deletions src/main/mondrian/calc/TupleListCalc.java
@@ -0,0 +1,42 @@
/*
// $Id$
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc;

import java.util.List;

import mondrian.olap.Evaluator;
import mondrian.olap.Member;

// End ListCalc.java

/**
* Expression which evaluates a set of members or tuples to a list.
*
* @author jhyde
* @version $Id$
* @since Sep 27, 2005
*/
public interface TupleListCalc extends ListCalc {
/**
* Evaluates an expression to yield a list of tuples.
*
* <p>Each tuple is represented by an array of members.
*
* <p>The list is immutable if {@link #getResultStyle()} yields
* {@link mondrian.calc.ResultStyle#MUTABLE_LIST}. Otherwise,
* the caller must not modify the list.
*
* @param evaluator Evaluation context
* @return A list of tuples, never null.
*/
List<Member[]> evaluateTupleList(Evaluator evaluator);
}

// End TupleListCalc.java
20 changes: 16 additions & 4 deletions src/main/mondrian/calc/impl/AbstractIterCalc.java
Expand Up @@ -3,18 +3,19 @@
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2007 Julian Hyde
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc.impl;

import mondrian.calc.Calc;
import mondrian.calc.IterCalc;
import mondrian.calc.ResultStyle;
import mondrian.olap.Evaluator;
import mondrian.olap.Exp;
import mondrian.olap.Member;
import mondrian.olap.type.SetType;
import mondrian.calc.IterCalc;
import mondrian.calc.Calc;
import mondrian.calc.ResultStyle;

/**
* Abstract implementation of the {@link mondrian.calc.IterCalc} interface.
Expand Down Expand Up @@ -59,6 +60,17 @@ public Calc[] getCalcs() {
public ResultStyle getResultStyle() {
return ResultStyle.ITERABLE;
}

@SuppressWarnings({"unchecked"})
public Iterable<Member> evaluateMemberIterable(Evaluator evaluator) {
return (Iterable<Member>) evaluateIterable(evaluator);
}

@SuppressWarnings({"unchecked"})
public Iterable<Member[]> evaluateTupleIterable(Evaluator evaluator) {
return (Iterable<Member[]>) evaluateIterable(evaluator);
}

}

// End AbstractIterCalc.java
27 changes: 19 additions & 8 deletions src/main/mondrian/calc/impl/AbstractListCalc.java
Expand Up @@ -3,20 +3,19 @@
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2007 Julian Hyde
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc.impl;

import java.util.List;

import mondrian.calc.*;
import mondrian.olap.Evaluator;
import mondrian.olap.Exp;
import mondrian.olap.Member;
import mondrian.olap.type.SetType;
import mondrian.calc.ListCalc;
import mondrian.calc.Calc;
import mondrian.calc.ResultStyle;

import java.util.List;

/**
* Abstract implementation of the {@link mondrian.calc.ListCalc} interface.
Expand All @@ -30,8 +29,9 @@
* @since Sep 27, 2005
*/
public abstract class AbstractListCalc
extends AbstractCalc
implements ListCalc {
extends AbstractCalc
implements ListCalc, MemberListCalc, TupleListCalc
{
private final Calc[] calcs;
private final boolean mutable;

Expand Down Expand Up @@ -78,6 +78,17 @@ public ResultStyle getResultStyle() {
ResultStyle.MUTABLE_LIST :
ResultStyle.LIST;
}

@SuppressWarnings({"unchecked"})
public List<Member> evaluateMemberList(Evaluator evaluator) {
return (List<Member>) evaluateList(evaluator);
}

@SuppressWarnings({"unchecked"})
public List<Member[]> evaluateTupleList(Evaluator evaluator) {
return (List<Member[]>) evaluateList(evaluator);
}

}

// End AbstractListCalc.java
97 changes: 97 additions & 0 deletions src/main/mondrian/calc/impl/AbstractMemberListCalc.java
@@ -0,0 +1,97 @@
/*
// $Id$
// This software is subject to the terms of the Common Public License
// Agreement, available at the following URL:
// http://www.opensource.org/licenses/cpl.html.
// Copyright (C) 2006-2008 Julian Hyde
// All Rights Reserved.
// You must accept the terms of that agreement to use this software.
*/
package mondrian.calc.impl;

import mondrian.olap.*;
import mondrian.olap.type.SetType;
import mondrian.calc.*;

import java.util.List;

/**
* Abstract implementation of the {@link mondrian.calc.ListCalc} interface
* for expressions that return a list of members but never a list of tuples.
*
* <p>The derived class must
* implement the {@link #evaluateMemberList(mondrian.olap.Evaluator)} method,
* and the {@link #evaluate(mondrian.olap.Evaluator)} method will call it.
*
* @see mondrian.calc.impl.AbstractListCalc
*
* @author jhyde
* @version $Id$
* @since Feb 20, 2008
*/
public abstract class AbstractMemberListCalc
extends AbstractCalc
implements MemberListCalc
{
private final Calc[] calcs;
private final boolean mutable;

/**
* Creates an abstract implementation of a compiled expression which
* returns a mutable list of members.
*
* @param exp Expression which was compiled
* @param calcs List of child compiled expressions (for dependency
* analysis)
*/
protected AbstractMemberListCalc(Exp exp, Calc[] calcs) {
this(exp, calcs, true);
}

/**
* Creates an abstract implementation of a compiled expression which
* returns a list.
*
* @param exp Expression which was compiled
* @param calcs List of child compiled expressions (for dependency
* analysis)
* @param mutable Whether the list is mutable
*/
protected AbstractMemberListCalc(Exp exp, Calc[] calcs, boolean mutable) {
super(exp);
this.calcs = calcs;
this.mutable = mutable;
assert getType() instanceof SetType : "expecting a set: " + getType();
assert ((SetType) getType()).getArity() == 1;
}

public Object evaluate(Evaluator evaluator) {
final List<Member> memberList = evaluateMemberList(evaluator);
assert memberList != null : "null as empty memberList is deprecated";
return memberList;
}

public Calc[] getCalcs() {
return calcs;
}

public ResultStyle getResultStyle() {
return mutable ?
ResultStyle.MUTABLE_LIST :
ResultStyle.LIST;
}

public String toString() {
return "AbstractMemberListCalc object";
}

public List<Member> evaluateList(Evaluator evaluator) {
return evaluateMemberList(evaluator);
}

public List<Member[]> evaluateTupleList(Evaluator evaluator) {
throw new UnsupportedOperationException();
}
}

// End AbstractMemberListCalc.java

0 comments on commit 3b50bf1

Please sign in to comment.