Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugfix/233 store as binary #234

Merged
merged 4 commits into from
May 29, 2024
Merged

Bugfix/233 store as binary #234

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8adc66f
Check if the output reach the LuceneIndex limit for a node property, …
May 29, 2024
78a91cb
Store the value as binary when the size is too big
May 29, 2024
4456f1c
Fix typo
May 29, 2024
0e5cb32
Fix typo, store binary in a different property
May 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion core/src/main/java/de/valtech/aecu/core/history/HistoryUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
*/
package de.valtech.aecu.core.history;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.GregorianCalendar;
Expand Down Expand Up @@ -50,6 +53,12 @@
import de.valtech.aecu.api.service.HistoryEntry.STATE;
import de.valtech.aecu.core.service.HistoryEntryImpl;

import javax.jcr.Binary;
import javax.jcr.Node;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.jcr.ValueFactory;

/**
* Reads and writes history entries.
*
Expand Down Expand Up @@ -77,6 +86,9 @@ public class HistoryUtil {
protected static final String ATTR_START = "start";
protected static final String ATTR_END = "end";
private static final String NAME_INDEX = "oak:index";
// This size is limited by the LuceneDocumentMaker to be able to read the property and create the new index
// The limit is 102400 but just to be in the safe side, is set to a bit lower number
private static final int MAXIMUN_PROPERTY_SIZE = 100000;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: MAXIMUN_PROPERTY_SIZE -> MAXIMUM_PROPERTY_SIZE

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


private Random random = new Random();

Expand Down Expand Up @@ -358,7 +370,21 @@ private void saveExecutionResultInHistory(ExecutionResult result, String path, R
values.put(ATTR_RUN_STATE, result.getState().name());
values.put(ATTR_PATH, result.getPath());
if (StringUtils.isNotBlank(result.getOutput())) {
values.put(ATTR_RUN_OUTPUT, result.getOutput());
if (result.getOutput().getBytes(StandardCharsets.UTF_8).length < MAXIMUN_PROPERTY_SIZE) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small change suggestion: store full data in seperate property in case it is too big. like that we don't need to check type of the property when reading.

if (result.getOutput().getBytes(StandardCharsets.UTF_8).length > MAXIMUM_PROPERTY_SIZE) {
    values.put(ATTR_RUN_OUTPUT, "Output data too big, full data is stored as a binary in runOutputFull");
    values.put(ATTR_RUN_OUTPUT_FULL, new ByteArrayInputStream(result.getOutput().getBytes(StandardCharsets.UTF_8)));
    LOG.info("Script result is bigger than 100 000 bytes. Full data can be found as a binary in property runOutputFull for path " + path );
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the new property

values.put(ATTR_RUN_OUTPUT, result.getOutput());
} else {
try {
Node node = entry.adaptTo(Node.class);
Session session = node.getSession();
ValueFactory factory = session.getValueFactory();
InputStream is = new ByteArrayInputStream(result.getOutput().getBytes());
Binary binary = factory.createBinary(is);
node.setProperty(ATTR_RUN_OUTPUT, binary);
session.save();
} catch (RepositoryException e) {
LOG.error("Not able to save the output of the script as binary on the History node [{}]", entry.getPath());
}
}
}
if (StringUtils.isNotBlank(result.getResult())) {
values.put(ATTR_RUN_RESULT, result.getResult());
Expand Down
Loading