Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated EXPM1() and Tests to New Engine #215

Merged
merged 20 commits into from
Feb 13, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
e2a6901
Updated EXPM1() and tests to new engine
matthewryanwells Feb 2, 2023
e357cdd
Added EXPM1 to PPL Parser
matthewryanwells Feb 2, 2023
c152a95
Added documentation for EXPM1 PPL and fixed small mistake in EXP docu…
matthewryanwells Feb 2, 2023
b5ab93e
Added newline to fix code style issue
matthewryanwells Feb 2, 2023
2958c3e
removed unnecessary integration tests and fixed typo in another test
matthewryanwells Feb 2, 2023
3c856a3
Extracted common code from math functions into external function
matthewryanwells Feb 6, 2023
b1c9fde
Added javadoc comment to new function
matthewryanwells Feb 6, 2023
457a5a6
Improved Javadoc
matthewryanwells Feb 6, 2023
f013e7d
Pulling changes from upstream
matthewryanwells Feb 7, 2023
6d09e77
Added and improved EXPM1 tests, and improved documentation for CLI an…
matthewryanwells Feb 7, 2023
eecd489
Removed EXPM1 from PPL
matthewryanwells Feb 8, 2023
8b88d1c
Updated math base function to take any type and included more functio…
matthewryanwells Feb 8, 2023
7a028d4
Made tiny change to documentation
matthewryanwells Feb 8, 2023
4574275
reverted euler function to previous iteration as the one I made didn'…
matthewryanwells Feb 8, 2023
62cc771
fixed typo in a comment that I made in a previous commit
matthewryanwells Feb 9, 2023
689a1dc
fixed a mistake I made in fixing the typo from my last commit
matthewryanwells Feb 9, 2023
5a3b724
Fixed spacing inconsistencies in math.rst file
matthewryanwells Feb 9, 2023
8042ebe
removed the base math function so that it can be pushed in a seperate…
matthewryanwells Feb 10, 2023
a74ae35
Reverted reverted change that removes the refactoring as this will be…
matthewryanwells Feb 10, 2023
e5a7001
updated base function to have more descriptive variables
matthewryanwells Feb 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions core/src/main/java/org/opensearch/sql/expression/DSL.java
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,10 @@ public static FunctionExpression exp(Expression... expressions) {
return compile(FunctionProperties.None, BuiltinFunctionName.EXP, expressions);
}

public static FunctionExpression expm1(Expression... expressions) {
return compile(FunctionProperties.None, BuiltinFunctionName.EXPM1, expressions);
}

public static FunctionExpression floor(Expression... expressions) {
return compile(FunctionProperties.None, BuiltinFunctionName.FLOOR, expressions);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ public enum BuiltinFunctionName {
CRC32(FunctionName.of("crc32")),
E(FunctionName.of("e")),
EXP(FunctionName.of("exp")),
EXPM1(FunctionName.of("expm1")),
FLOOR(FunctionName.of("floor")),
LN(FunctionName.of("ln")),
LOG(FunctionName.of("log")),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ public static void register(BuiltinFunctionRepository repository) {
repository.register(crc32());
repository.register(euler());
repository.register(exp());
repository.register(expm1());
repository.register(floor());
repository.register(ln());
repository.register(log());
Expand All @@ -85,6 +86,22 @@ public static void register(BuiltinFunctionRepository repository) {
repository.register(tan());
}

/**
* Base function for math functions with similar formats that return DOUBLE.
*
* @param functionName BuiltinFunctionName of math function.
* @param formula lambda function of math formula.
* @return DefaultFunctionResolver for math functions.
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
*/
private static DefaultFunctionResolver doubleMathFunctionBase(
matthewryanwells marked this conversation as resolved.
Show resolved Hide resolved
FunctionName functionName, SerializableFunction<Double, Double> formula) {
return FunctionDSL.define(functionName,
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(formula.apply(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
matthewryanwells marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Definition of abs() function. The supported signature of abs() function are INT -> INT LONG ->
* LONG FLOAT -> FLOAT DOUBLE -> DOUBLE
Expand Down Expand Up @@ -186,15 +203,19 @@ private static DefaultFunctionResolver euler() {
}

/**
* Definition of exp(x) function. Calculate exponent function e to the x The supported signature
* of exp function is INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
* Definition of exp(x) function. Calculate exponent function e to the x
* The supported signature of exp function is INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver exp() {
return FunctionDSL.define(BuiltinFunctionName.EXP.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.exp(v.doubleValue()))),
type, DOUBLE)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.EXP.getName(), v -> Math.exp(v));
}

/**
* Definition of expm1(x) function. Calculate exponent function e to the x, minus 1
* The supported signature of exp function is INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver expm1() {
return doubleMathFunctionBase(BuiltinFunctionName.EXPM1.getName(), v -> Math.expm1(v));
}

/**
Expand All @@ -214,11 +235,7 @@ private static DefaultFunctionResolver floor() {
* ln function is INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver ln() {
return FunctionDSL.define(BuiltinFunctionName.LN.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.log(v.doubleValue()))),
type, DOUBLE)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.LN.getName(), v -> Math.log(v));
}

/**
Expand Down Expand Up @@ -255,23 +272,16 @@ private static DefaultFunctionResolver log() {
* log function is SHORT/INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver log10() {
return FunctionDSL.define(BuiltinFunctionName.LOG10.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.log10(v.doubleValue()))),
type, DOUBLE)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.LOG10.getName(), v -> Math.log10(v));
}

/**
* Definition of log2(x) function. Calculate base-2 logarithm of x The supported signature of log
* function is SHORT/INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver log2() {
return FunctionDSL.define(BuiltinFunctionName.LOG2.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.log(v.doubleValue()) / Math.log(2))), DOUBLE, type))
.collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.LOG2.getName(),
v -> (Math.log(v) / Math.log(2)));
}

/**
Expand Down Expand Up @@ -479,11 +489,7 @@ private static DefaultFunctionResolver sqrt() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver cbrt() {
return FunctionDSL.define(BuiltinFunctionName.CBRT.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.cbrt(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.CBRT.getName(), v -> Math.cbrt(v));
}

/**
Expand Down Expand Up @@ -606,11 +612,7 @@ private static DefaultFunctionResolver atan2() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver cos() {
return FunctionDSL.define(BuiltinFunctionName.COS.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.cos(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.COS.getName(), v -> Math.cos(v));
}

/**
Expand Down Expand Up @@ -641,11 +643,7 @@ private static DefaultFunctionResolver cot() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver degrees() {
return FunctionDSL.define(BuiltinFunctionName.DEGREES.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.toDegrees(v.doubleValue()))),
type, DOUBLE)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.DEGREES.getName(), v -> Math.toDegrees(v));
}

/**
Expand All @@ -655,11 +653,7 @@ private static DefaultFunctionResolver degrees() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver radians() {
return FunctionDSL.define(BuiltinFunctionName.RADIANS.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.toRadians(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.RADIANS.getName(), v -> Math.toRadians(v));
}

/**
Expand All @@ -669,11 +663,7 @@ private static DefaultFunctionResolver radians() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver sin() {
return FunctionDSL.define(BuiltinFunctionName.SIN.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.sin(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.SIN.getName(), v -> Math.sin(v));
}

/**
Expand All @@ -683,10 +673,6 @@ private static DefaultFunctionResolver sin() {
* INTEGER/LONG/FLOAT/DOUBLE -> DOUBLE
*/
private static DefaultFunctionResolver tan() {
return FunctionDSL.define(BuiltinFunctionName.TAN.getName(),
ExprCoreType.numberTypes().stream()
.map(type -> FunctionDSL.impl(FunctionDSL.nullMissingHandling(
v -> new ExprDoubleValue(Math.tan(v.doubleValue()))),
DOUBLE, type)).collect(Collectors.toList()));
return doubleMathFunctionBase(BuiltinFunctionName.TAN.getName(), v -> Math.tan(v));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,58 @@ public void exp_missing_value() {
assertTrue(exp.valueOf(valueEnv()).isMissing());
}

/**
* Test expm1 with integer value.
*/
@ParameterizedTest(name = "expm1({0})")
matthewryanwells marked this conversation as resolved.
Show resolved Hide resolved
@ValueSource(ints = {-2, 2})
public void expm1_int_value(Integer value) {
FunctionExpression expm1 = DSL.expm1(DSL.literal(value));
assertThat(
expm1.valueOf(valueEnv()),
allOf(hasType(DOUBLE), hasValue(Math.expm1(value))));
assertEquals(String.format("expm1(%s)", value.toString()), expm1.toString());
}

/**
* Test expm1 with long value.
*/
@ParameterizedTest(name = "expm1({0})")
@ValueSource(longs = {-2L, 2L})
public void expm1_long_value(Long value) {
FunctionExpression expm1 = DSL.expm1(DSL.literal(value));
assertThat(
expm1.valueOf(valueEnv()),
allOf(hasType(DOUBLE), hasValue(Math.expm1(value))));
assertEquals(String.format("expm1(%s)", value.toString()), expm1.toString());
}

/**
* Test expm1 with float value.
*/
@ParameterizedTest(name = "expm1({0})")
@ValueSource(floats = {-2F, 2F})
matthewryanwells marked this conversation as resolved.
Show resolved Hide resolved
public void expm1_float_value(Float value) {
FunctionExpression expm1 = DSL.expm1(DSL.literal(value));
assertThat(
expm1.valueOf(valueEnv()),
allOf(hasType(DOUBLE), hasValue(Math.expm1(value))));
assertEquals(String.format("expm1(%s)", value.toString()), expm1.toString());
}

/**
* Test expm1 with double value.
*/
@ParameterizedTest(name = "expm1({0})")
@ValueSource(doubles = {-2D, 2D})
public void expm1_double_value(Double value) {
FunctionExpression expm1 = DSL.expm1(DSL.literal(value));
assertThat(
expm1.valueOf(valueEnv()),
allOf(hasType(DOUBLE), hasValue(Math.expm1(value))));
assertEquals(String.format("expm1(%s)", value.toString()), expm1.toString());
}

/**
* Test floor with integer value.
*/
Expand Down
15 changes: 13 additions & 2 deletions docs/user/dql/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -443,10 +443,21 @@ EXPM1
Description
>>>>>>>>>>>

Specifications:
Usage: EXPM1(T) returns the exponential of T, minus 1.

Argument type: INTEGER/LONG/FLOAT/DOUBLE

1. EXPM1(NUMBER T) -> T
Return type: DOUBLE

Example::

os> SELECT EXPM1(-1), EXPM1(0), EXPM1(1);
fetched rows / total rows = 1/1
+---------------------+------------+-------------------+
| EXPM1(-1) | EXPM1(0) | EXPM1(1) |
matthewryanwells marked this conversation as resolved.
Show resolved Hide resolved
|---------------------+------------+-------------------|
| -0.6321205588285577 | 0.0 | 1.718281828459045 |
+---------------------+------------+-------------------+

FLOOR
-----
Expand Down
23 changes: 22 additions & 1 deletion docs/user/ppl/functions/math.rst
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ Usage: exp(x) return e raised to the power of x.

Argument type: INTEGER/LONG/FLOAT/DOUBLE

Return type: INTEGER
Return type: DOUBLE

Example::

Expand All @@ -321,6 +321,27 @@ Example::
| 7.38905609893065 |
+------------------+

EXPM1
-----

Description
>>>>>>>>>>>

Usage: expm1(x) return e raised to the power of x, minus 1.

Argument type: INTEGER/LONG/FLOAT/DOUBLE

Return type: DOUBLE

Example::

os> source=people | eval `EXPM1(2)` = EXPM1(2) | fields `EXPM1(2)`
acarbonetto marked this conversation as resolved.
Show resolved Hide resolved
fetched rows / total rows = 1/1
+------------------+
| EXPM1(2) |
|------------------|
| 6.38905609893065 |
+------------------+

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidental deletion?

FLOOR
-----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,18 @@ result, rows(Math.exp(32)), rows(Math.exp(36)), rows(Math.exp(28)), rows(Math.ex
rows(Math.exp(36)), rows(Math.exp(39)), rows(Math.exp(34)));
}

@Test
public void testExpm1() throws IOException {
JSONObject result =
executeQuery(
String.format(
"source=%s | eval f = expm1(age) | fields f", TEST_INDEX_BANK));
verifySchema(result, schema("f", null, "double"));
verifyDataRows(
result, rows(Math.expm1(32)), rows(Math.expm1(36)), rows(Math.expm1(28)), rows(Math.expm1(33)),
rows(Math.expm1(36)), rows(Math.expm1(39)), rows(Math.expm1(34)));
}

@Test
public void testFloor() throws IOException {
JSONObject result =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,13 @@ public void testE() throws IOException {
verifyDataRows(result, rows(Math.E));
}

@Test
public void testExpm1() throws IOException {
JSONObject result = executeQuery("select expm1(account_number) FROM " + TEST_INDEX_BANK + " LIMIT 2");
verifySchema(result, schema("expm1(account_number)", null, "double"));
verifyDataRows(result, rows(Math.expm1(1)), rows(Math.expm1(6)));
}

@Test
public void testMod() throws IOException {
JSONObject result = executeQuery("select mod(3, 2)");
Expand Down
1 change: 1 addition & 0 deletions ppl/src/main/antlr/OpenSearchPPLLexer.g4
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ CONV: 'CONV';
CRC32: 'CRC32';
E: 'E';
EXP: 'EXP';
EXPM1: 'EXPM1';
FLOOR: 'FLOOR';
LN: 'LN';
LOG: 'LOG';
Expand Down
2 changes: 1 addition & 1 deletion ppl/src/main/antlr/OpenSearchPPLParser.g4
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,7 @@ relevanceArgValue
;

mathematicalFunctionBase
: ABS | CBRT | CEIL | CEILING | CONV | CRC32 | E | EXP | FLOOR | LN | LOG | LOG10 | LOG2 | MOD | PI |POW | POWER
: ABS | CBRT | CEIL | CEILING | CONV | CRC32 | E | EXP | EXPM1 | FLOOR | LN | LOG | LOG10 | LOG2 | MOD | PI |POW | POWER
| RAND | ROUND | SIGN | SQRT | TRUNCATE
| trigonometricFunctionName
;
Expand Down
2 changes: 1 addition & 1 deletion sql/src/main/antlr/OpenSearchSQLParser.g4
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ aggregationFunctionName
;

mathematicalFunctionName
: ABS | CBRT | CEIL | CEILING | CONV | CRC32 | E | EXP | FLOOR | LN | LOG | LOG10 | LOG2 | MOD | PI | POW | POWER
: ABS | CBRT | CEIL | CEILING | CONV | CRC32 | E | EXP | EXPM1 | FLOOR | LN | LOG | LOG10 | LOG2 | MOD | PI | POW | POWER
| RAND | ROUND | SIGN | SQRT | TRUNCATE
| trigonometricFunctionName
;
Expand Down