Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for search on attachments #119

Merged
merged 2 commits into from
Apr 11, 2022
Merged

Support for search on attachments #119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4e4c44c
Support for search on attachments
georgweiss Apr 7, 2022
40af621
Bug fix + updated help content
georgweiss Apr 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions src/main/java/org/phoebus/olog/LogSearchUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

import static org.elasticsearch.index.query.QueryBuilders.boolQuery;
import static org.elasticsearch.index.query.QueryBuilders.disMaxQuery;
import static org.elasticsearch.index.query.QueryBuilders.existsQuery;
import static org.elasticsearch.index.query.QueryBuilders.fuzzyQuery;
import static org.elasticsearch.index.query.QueryBuilders.matchPhraseQuery;
import static org.elasticsearch.index.query.QueryBuilders.nestedQuery;
import static org.elasticsearch.index.query.QueryBuilders.rangeQuery;
import static org.elasticsearch.index.query.QueryBuilders.wildcardQuery;

import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
Expand Down Expand Up @@ -49,12 +49,16 @@ public class LogSearchUtil
final public static DateTimeFormatter MILLI_FORMAT = DateTimeFormatter.ofPattern(MILLI_PATTERN).withZone(ZoneId.systemDefault());

@Value("${elasticsearch.log.index:olog_logs}")
@SuppressWarnings("unused")
private String ES_LOG_INDEX;
@Value("${elasticsearch.log.type:olog_log}")
@SuppressWarnings("unused")
private String ES_LOG_TYPE;
@Value("${elasticsearch.result.size.search.default:100}")
@SuppressWarnings("unused")
private int defaultSearchSize;
@Value("${elasticsearch.result.size.search.max:1000}")
@SuppressWarnings("unused")
private int maxSearchSize;

/**
Expand Down Expand Up @@ -242,6 +246,23 @@ else if(sort.toUpperCase().startsWith("DESC") || sort.toUpperCase().startsWith("
}
}
break;
case "attachments":
DisMaxQueryBuilder attachmentsQuery = disMaxQuery();
List<String> parameterValues = parameter.getValue();
if(parameterValues.isEmpty()){ // user does not specify type -> all attachment types
attachmentsQuery.add(existsQuery("attachments"));
}
else{
for (String value : parameterValues)
{
for (String pattern : value.split("[\\|,;]"))
{
attachmentsQuery.add(wildcardQuery("attachments.fileMetadataDescription", pattern.trim()));
}
}
}
boolQuery.must(attachmentsQuery);
break;
default:
// Unsupported search parameters are ignored
break;
Expand Down Expand Up @@ -327,13 +348,7 @@ else if(sort.toUpperCase().startsWith("DESC") || sort.toUpperCase().startsWith("
searchSourceBuilder.sort("createdDate", sortOrder);
searchSourceBuilder.query(boolQuery);
searchSourceBuilder.size(Math.min(searchResultSize, maxSearchSize));
if (from >= 0)
{
searchSourceBuilder.from(from);
} else
{
searchSourceBuilder.from(0);
}
searchSourceBuilder.from(Math.max(0, from));
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));

searchRequest.source(searchSourceBuilder);
Expand Down
9 changes: 9 additions & 0 deletions src/main/resources/static/SearchHelp_en.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,15 @@ <h3>properties</h3>
Not all elements need to be specified. A query like <code>properties=Shift Info</code> will match all log entries
containing such a property, irrespective of the attribute list it contains.<br>

<h3>attachments</h3>
If this added to the search query, the service will return log entries that contain at least one file attachment.
The value for this key is <b>case sensitive</b>. If a value for this search parameter is omitted, all attachment types are considered. Using
<code>attachments=image</code> will match entries with image attachments, while <code>attachments=plt</code>
will match entries with plot file attachments. To search for log entries that containing attachments any of type but <code>image</code>
or <code>plt</code>, use <code>attachments=file</code>.<br>

As with logbooks and tags, attachment type values can be combined, e.g. <code>attachments=image,plt</code>.

<h3>Combining keys</h3>
If multiple keys are used in a search query, the service will consider all (valid) keys and return log
entries matching all criteria. In other words, search criteria are and:ed. However, as mentioned above,
Expand Down