Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[js-transform] Introduced support for additional parameters #10901

Merged
merged 5 commits into from
Jul 31, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion bundles/org.openhab.transform.javascript/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Transform an input to an output using JavaScript.
It expects the transformation rule to be read from a file which is stored under the `transform` folder.
To organize the various transformations, one should use subfolders.

## Example
## Examples

Let's assume we have received a string containing `foo bar baz` and we're looking for a length of the last word (`baz`).

Expand All @@ -18,6 +18,26 @@ transform/getValue.js:
})(input)
```

JavaScript transformation syntax also support additional parameters which can be passed to the script.
This can prevent redundancy when transformation is needed for several use cases, but with small adaptations.
additional parameters can be passed to the script via [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) query syntax.

As `input` name is reserved for transformed data, it can't be used in query parameters.
Also `?` and `&` characters are reserved, but if they need to passed as additional data, they can be escaped according to URI syntax.


transform/scale.js:
```
(function(data, cf, d) {
return parseFloat(data) * parseFloat(cf) / parseFloat(d);
})(input, correctionFactor, divider)
```

`transform/scale.js?correctionFactor=1.1&divider=10`

Following example will return value `23.54` when `input` data is `214`.


## Test JavaScript

You can use online JavaScript testers to validate your script.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(function(i) {
fwolter marked this conversation as resolved.
Show resolved Hide resolved
var array = i.split(" ");
return array[array.length - 1].length;
})(input)
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(function(i) {
var array = i.split(" ");
return array[array.length - 1].length;
})(input)
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(function(i, a, b) {
return b;
})(input, a, test)
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(function(data, cf, d) {
return parseFloat(data) * parseFloat(cf) / parseFloat(d);
})(input, correctionFactor, divider)
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
(function(i, a, b) {
return parseInt(i) + parseInt(a) + parseInt(b);
})(input, a, b)
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@

import java.io.File;
import java.io.FilenameFilter;
import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URLDecoder;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.stream.Collectors;

import javax.script.Bindings;
Expand Down Expand Up @@ -56,6 +61,8 @@ public class JavaScriptTransformationService implements TransformationService, C
private static final String CONFIG_PARAM_FUNCTION = "function";
private static final String[] FILE_NAME_EXTENSIONS = { "js" };

private static final String SCRIPT_DATA_WORD = "input";

private final JavaScriptEngineManager manager;

@Activate
Expand All @@ -70,25 +77,44 @@ public JavaScriptTransformationService(final @Reference JavaScriptEngineManager
* transformations one should use subfolders.
*
* @param filename the name of the file which contains the Java script
* transformation rule. Transformation service inject input
* (source) to 'input' variable.
* transformation rule. Filename can also include additional
* variables in URI query variable format which will be injected
* to script engine. Transformation service inject input (source)
* to 'input' variable.
* @param source the input to transform
*/
@Override
public @Nullable String transform(String filename, String source) throws TransformationException {
if (filename == null || source == null) {
throw new TransformationException("the given parameters 'filename' and 'source' must not be null");
}

final long startTime = System.currentTimeMillis();
logger.debug("about to transform '{}' by the JavaScript '{}'", source, filename);

Map<String, String> vars = Collections.emptyMap();
String fn = filename;

if (filename.contains("?")) {
String[] parts = filename.split("\\?");
if (parts.length > 2) {
throw new TransformationException("Questionmark should be defined only ones in the filename");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
throw new TransformationException("Questionmark should be defined only ones in the filename");
throw new TransformationException("Questionmark should be defined only once in the filename");

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

}
fn = parts[0];
try {
vars = splitQuery(parts[1]);
} catch (UnsupportedEncodingException e) {
throw new TransformationException("Illegal filename syntax");
}
if (isReservedWordUsed(vars)) {
throw new TransformationException(
"'" + SCRIPT_DATA_WORD + "' word is reserved and can't be used in additional parameters");
}
}

String result = "";

try {
final CompiledScript cScript = manager.getScript(filename);
final CompiledScript cScript = manager.getScript(fn);
final Bindings bindings = cScript.getEngine().createBindings();
bindings.put("input", source);
bindings.put(SCRIPT_DATA_WORD, source);
vars.forEach((k, v) -> bindings.put(k, v));
result = String.valueOf(cScript.eval(bindings));
return result;
} catch (ScriptException e) {
Expand All @@ -99,6 +125,31 @@ public JavaScriptTransformationService(final @Reference JavaScriptEngineManager
}
}

private boolean isReservedWordUsed(Map<String, String> map) {
for (String key : map.keySet()) {
if (SCRIPT_DATA_WORD.equals(key)) {
return true;
}
}
return false;
}

private Map<String, String> splitQuery(@Nullable String query) throws UnsupportedEncodingException {
Map<String, String> result = new LinkedHashMap<>();
if (query != null) {
String[] pairs = query.split("&");
for (String pair : pairs) {
String[] keyval = pair.split("=");
if (keyval.length != 2) {
throw new UnsupportedEncodingException();
} else {
result.put(URLDecoder.decode(keyval[0], "UTF-8"), URLDecoder.decode(keyval[1], "UTF-8"));
}
}
}
return result;
}

@Override
public @Nullable Collection<ParameterOption> getParameterOptions(URI uri, String param, @Nullable String context,
@Nullable Locale locale) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
/**
* Copyright (c) 2010-2021 Contributors to the openHAB project
*
* See the NOTICE file(s) distributed with this work for additional
* information.
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.openhab.transform.javascript.internal;

import static org.junit.jupiter.api.Assertions.*;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Comparator;
import java.util.stream.Stream;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import org.openhab.core.transform.TransformationException;
import org.osgi.framework.BundleContext;

/**
* @author Pauli Anttila - Initial contribution
*/
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.WARN)
public class JavaScriptTransformationServiceTest {

private static final String BASE_FOLDER = "target";
private static final String SRC_FOLDER = "conf";
private static final String CONFIG_FOLDER = BASE_FOLDER + File.separator + SRC_FOLDER;

private @Mock BundleContext bundleContext;

private TestableJavaScriptTransformationService processor;

private class TestableJavaScriptTransformationService extends JavaScriptTransformationService {
public TestableJavaScriptTransformationService(JavaScriptEngineManager manager) {
super(manager);
}
};

@BeforeEach
public void setUp() throws IOException {
JavaScriptEngineManager manager = new JavaScriptEngineManager();
processor = new TestableJavaScriptTransformationService(manager);
copyDirectory(SRC_FOLDER, CONFIG_FOLDER);
}

@AfterEach
public void tearDown() throws IOException {
try (Stream<Path> walk = Files.walk(Path.of(CONFIG_FOLDER))) {
walk.sorted(Comparator.reverseOrder()).map(Path::toFile).forEach(File::delete);
}
}

private void copyDirectory(String from, String to) throws IOException {
Files.walk(Paths.get(from)).forEach(fromPath -> {
Path toPath = Paths.get(to, fromPath.toString().substring(from.length()));
try {
Files.copy(fromPath, toPath);
} catch (IOException e) {
}
});
}

@Test
public void testReadmeExampleWithoutSubFolder() throws Exception {
final String DATA = "foo bar baz";
final String SCRIPT = "readme.js";

String transformedResponse = processor.transform(SCRIPT, DATA);
assertEquals("3", transformedResponse);
}

@Test
public void testReadmeExampleWithSubFolders() throws Exception {
final String DATA = "foo bar baz";
final String SCRIPT = "js/readme/readme.js";

String transformedResponse = processor.transform(SCRIPT, DATA);
assertEquals("3", transformedResponse);
}

@Test
public void testReadmeScaleExample() throws Exception {
final String DATA = "214";
final String SCRIPT = "scale.js?correctionFactor=1.1&divider=10.js";

String transformedResponse = processor.transform(SCRIPT, DATA);
assertEquals("23.54", transformedResponse);
}

@Test
public void testAdditionalVariables() throws Exception {
final String DATA = "100";
final String SCRIPT = "sum.js?a=10&b=1";

String transformedResponse = processor.transform(SCRIPT, DATA);
assertEquals("111", transformedResponse);
}

@Test
public void testIllegalVariableName() throws Exception {
final String DATA = "100";
final String SCRIPT = "sum.js?a=10&input=fail&b=1";

Exception exception = assertThrows(TransformationException.class, () -> processor.transform(SCRIPT, DATA));
assertEquals("'input' word is reserved and can't be used in additional parameters", exception.getMessage());
}

@Test
public void testIllegalQuestionmarkSequence() throws Exception {
final String DATA = "100";
final String SCRIPT = "sum.js?a=1&test=ab?d&b=2";

Exception exception = assertThrows(TransformationException.class, () -> processor.transform(SCRIPT, DATA));
assertEquals("Questionmark should be defined only ones in the filename", exception.getMessage());
}

@Test
public void testIllegalAmbersandSequence() throws Exception {
final String DATA = "foo";
final String SCRIPT = "returntest.js?a=1&test=ab&d&b=2";

Exception exception = assertThrows(TransformationException.class, () -> processor.transform(SCRIPT, DATA));
assertEquals("Illegal filename syntax", exception.getMessage());
}

@Test
public void testEncodedSpecialCharacters() throws Exception {
final String DATA = "100";
final String SCRIPT = "returntest.js?a=1&test=ab%3Fd%26f&b=2";

String transformedResponse = processor.transform(SCRIPT, DATA);
assertEquals("ab?d&f", transformedResponse);
}
}