Introducing TimeSeriesBuilder, Num, DoubleNum and BigDecimalNum

[team172011]

(permanently updated..)

With this PR ta4j will support using Double as elementary operations
type. It is possible to determine the underlying data type during
time series creation

• all BigDecimal tests passed
• all Double tests passed

new

• Class Num: Abstraction of number operations in the spirit of ta4j
• Class NaN Implementation of Num, represents static Not a Number instance
• Class: DoubleNum: Implementation of Num, uses Double as delegate
• Class: BigDecimalNum: Implementation of Num uses, uses BigDecimal as delegate
• Class: TimeSeriesBuilder: facilitate creation and transport of parameters of BaseTimeSeries

changed

• TimeSeries : addBar(..) functions to store a Bar directly in series
• TimeSeries : numOf(Number) function to get Num implementation of series
• TimeSeries : function() returns the used function to transform Numbers into Nums
• CachedIndicator: numOf(Number), numOf function of the underlying TimeSeries

Changes in all Indicators and test files:

• parameterized AbstractIndicatorTest and AbstractCriterionTest for testing DoubleNum an BigDecimalNum

  • Test classes should extend one of those classes so that they can be called two times with the corresponding Num implementation
    

• changed Decimal to Num

• changed generating a new value Decimal.valueOf(i)) to numOf(i) conditional of class instance

• all indicators needs a reference to its/a time series

Note:

• During the time series creation one can choose the underlying data type.
  Not in the Bar and not in the indicators.Indicators should only store
  the logic and use the Num abstraction to describe this logic.
• Parameters of indicators should be primitive (int, double). There is no
  need for (32bit) precision for parameters. They will be transformed in
  the constructor (via runtime) to the corresponding Num implementation.
• Do not use static Num values in indicator, timeSeries or bars, because
  the underlying Num implementation is defined during runtime. If necessary
  store static int or double values and transform them in the constructor.
• Transform Numbers (int, double,..) with help of numOf function of the
  Indicator or the TimeSeries.
• Add bars directly to the TimeSeries with help of the new addBar(..)
  functions.

Create TimeSeries:

SeriesBuilder seriesBuilder = new SeriesBuilder();
TimeSeries defaultSeries = seriesBuilder.build(); // build a new empty unnamed time series using BigDecimal as delegate
TimeSeries defaultSeriesName = seriesBuilder.withName("default").build(); // build a new empty time series using BigDecimal as delegate
TimeSeries doubleSeries = seriesBuilder.withMaxBarCount(100).withNumTypeOf(DoubleNum::valueOf).withName("useDouble").build(); // use lambda or method reference to define which Num implementation to use
TimeSeries bigDecimalSeries = seriesBuilder.withMaxBarCount(100).withNumTypeOf(BigDecimalNum.class).withName("useBigDecimal").build(); // use class to define which Num implementation to use

for(int i=1000; i>=0;i--){
    defaultSeries.addBar(ZonedDateTime.now().minusSeconds(i),i,i,i,i,i);
    defaultSeriesName.addBar(ZonedDateTime.now().minusSeconds(i),i,i,i,i,i);
    doubleSeries.addBar(ZonedDateTime.now().minusSeconds(i),i,i,i,i,i);
    bigDecimalSeries.addBar(ZonedDateTime.now().minusSeconds(i),i,i,i,i,i);
}

Num and valueOf in indicator:

public class indicator{

    @Override
    protected Num calculate(int index) {
        final Num typicalPrice = typicalPriceInd.getValue(index);
        final Num typicalPriceAvg = smaInd.getValue(index);
        final Num meanDeviation = meanDeviationInd.getValue(index);
        if (meanDeviation.isZero()) {
            return numOf(0);  // Decimal.valueOf
        }
        return (typicalPrice.minus(typicalPriceAvg)).dividedBy(meanDeviation.multipliedBy(FACTOR));
    }
}

Testfile with extending AbstractIndicatorTest (supports excel sheet validation, custom asserts and tests DoubleNum and BigDecimalNum)

public class EMAIndicatorTest extends AbstractIndicatorTest<Indicator<Num>, Num> {

    private ExternalIndicatorTest xls;

    public EMAIndicatorTest(Function<Number, Num> numFunction) throws Exception {
        super((data, params) -> new EMAIndicator(data, (int) params[0]), numFunction);
        xls = new XLSIndicatorTest(this.getClass(), "EMA.xls", 6, numFunction);
    }
    private TimeSeries data;

    @Before
    public void setUp() {
        data = new MockTimeSeries(numFunction,
                64.75, 63.79, 63.73,
                63.73, 63.55, 63.19,
                63.91, 63.85, 62.95,
                63.37, 61.33, 61.51);
    }

    @Test
    public void firstValueShouldBeEqualsToFirstDataValue() throws Exception {
        Indicator<Num> indicator = getIndicator(new ClosePriceIndicator(data), 1);
        assertNumEquals(indicator.getValue(0), 64.75);
    }

    @Test
    public void usingTimeFrame10UsingClosePrice() throws Exception {
        Indicator<Num> indicator = getIndicator(new ClosePriceIndicator(data), 10);
        assertNumEquals(indicator.getValue(9), 63.6948);
        assertNumEquals(indicator.getValue(10), 63.2648);
        assertNumEquals(indicator.getValue(11), 62.9457);
    }

    @Test
    public void stackOverflowError() throws Exception {
        List<Bar> bigListOfBars = new ArrayList<Bar>();
        for (int i = 0; i < 10000; i++) {
            bigListOfBars.add(new MockBar(i,numFunction));
        }
        MockTimeSeries bigSeries = new MockTimeSeries(bigListOfBars);
        Indicator<Num> indicator = getIndicator(new ClosePriceIndicator(bigSeries), 10);
        // if a StackOverflowError is thrown here, then the RecursiveCachedIndicator does not work as intended.
        assertNumEquals(indicator.getValue(9999), 9994.5);
    }

    @Test
    public void externalData() throws Exception {
        TimeSeries xlsSeries = xls.getSeries();
        Indicator<Num> closePrice = new ClosePriceIndicator(xlsSeries);
        Indicator<Num> indicator;

        indicator = getIndicator(closePrice, 1);
        assertIndicatorEquals(xls.getIndicator(1), indicator);
        assertEquals(329.0, indicator.getValue(indicator.getTimeSeries().getEndIndex()).doubleValue(), TestUtils.BIG_DECIMAL_OFFSET);

        indicator = getIndicator(closePrice, 3);
        assertIndicatorEquals(xls.getIndicator(3), indicator);
        assertEquals(327.7748, indicator.getValue(indicator.getTimeSeries().getEndIndex()).doubleValue(), TestUtils.BIG_DECIMAL_OFFSET);

        indicator = getIndicator(closePrice, 13);
        assertIndicatorEquals(xls.getIndicator(13), indicator);
        assertEquals(327.4076, indicator.getValue(indicator.getTimeSeries().getEndIndex()).doubleValue(), TestUtils.BIG_DECIMAL_OFFSET);
    }

}

[edlins]

Good to see this moving along! Excited to get back in sync and do some comparative perf testing.

[damir78]

Hey @team172011
Num -branch looks good :) Like Arithmetics from #88 as main idea))
Another possible improvements:

1. remove "if checks" in the implementations if you know that the data is coming is double. You do not to check again and again ))
2. if 1 done- you can reduce checks ... by placing a condition at call higher.. if something wrong: its wrong as well.. For example don't check every incoming number isNaN...
   It can improve performance ))

[team172011]

@damir78
I see you are right regarding 1. and I can dropp those checks:

        if (!(divisor instanceof DoubleNum)){
            throw new IllegalArgumentException(String.format(
                    "The underlying data types '%s' and '%s' are not the same"
                    ,this.getClass(), divisor.getClass()));
        }

it does not matter if the exception has been thrown because of false if statement or because of failed cast

regarding 2., I think the NaN checks are neccesary. With the external class for the NaN representation (stored in a static field) I could eleminate all the if(this==NaN)checks, but I still have to check if other (subtrahend, multiplicant, etc.) is NaN because ta4j supports the NaN value. That means, for example numOf(1).minus(NaN) should result in NaN and not in a ClassCastException or whatever error the delegate will produce. Furthermore we have to ensure symmetry and transivity of the equals(o) and compareTo(o) functions. For example NaN.equals(numOf(1)) == numOf(1).equals(NaN) must be true

[edlins]

Should TestUtils.BIG_DECIMAL_OFFSET be renamed to TestUtils.NUM_OFFSET? Similarly, the javadoc comments should probably be changed from Decimal to Num. These are inconsequential items but they are quite misleading as-is. Great work here, and sorry to be annoying. :-)

[team172011]

@edlins you are right. Your concern also points out that we need different offsets for DoubleNum and DecimalNum or a special test case for BigDecimalNum that checks its high precision

[edlins]

If you like, sure. I'd just give it a generic name and let people adjust it for their purposes. But I don't see a reason to object unless it makes writing test cases more complicated for newer developers.

[team172011]

merged, resolved and manually pushed to develop