Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark GeoIpDownloaderTask as completed after cancellation #84028

Merged
merged 107 commits into from Feb 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
8d31dd2
Mark GeoIpDownloaderTask as completed after cancellation
probakowski Feb 16, 2022
1afbc70
Update docs/changelog/84028.yaml
probakowski Feb 16, 2022
003facf
Script: Fields API for Dense Vector (#83550)
stu-elastic Feb 16, 2022
4ad8028
Fix GeoIpDownloader startup during rolling upgrade (#84000)
probakowski Feb 16, 2022
8007fc0
[docs] Mention JDK 17 in the Contributing docs (#84018)
arteam Feb 16, 2022
b004d3c
[ML] fix NER token grouping when special tokens are used (#84042)
benwtrent Feb 16, 2022
cd61ecc
Make intake BWC testing a matrix job for better parallelization
mark-vieira Feb 16, 2022
27d8a98
Rename BWC intake job so that old job is removed
mark-vieira Feb 16, 2022
97a3645
Make pull request BWC testing a matrix job for better parallelization
mark-vieira Feb 16, 2022
f14d8ac
[DOCS] Move ML info and upgrade APIs (#84005)
lcawl Feb 16, 2022
d70bdc1
Unmute BWC rest tests (#84058)
jbaiera Feb 16, 2022
d6b2ba7
Adding a warning if node.attr.data is set (#84050)
masseyke Feb 16, 2022
4ed36bd
Security global privilege for writing profile data of applications (#…
albertzaharovits Feb 16, 2022
f71467e
Remove all "user" related methods from HLRC (#84011)
tvernum Feb 16, 2022
235d018
Correct documentation regarding how to restore no `feature_states` (#…
gwbrown Feb 17, 2022
916defd
Improve BWC for persisted authentication headers (#83913)
ywangd Feb 17, 2022
c01b647
Adjust timeout for responses from SMB fixture (#84037)
jkakavas Feb 17, 2022
ae87a26
Fix GeoHexAggregationBuilderTests (#84049)
iverase Feb 17, 2022
ee36d2d
GeometryNormalizer should not fail if it cannot compute signed area (…
iverase Feb 17, 2022
ddbced2
fix testRestoreLocalHistoryFromTranslogOnPromotion (#84027)
idegtiarenko Feb 17, 2022
7503384
Preserve context in ResultDeduplicator (#84038)
DaveCTurner Feb 17, 2022
2ed206b
[Transform] Improve robustness of checkpointing (#80984)
Feb 17, 2022
d4c1f92
[ML] Add ML memory stats API (#83802)
droberts195 Feb 17, 2022
48da74f
Refactor FilterXContentParser and DelegatingXContentParser (#83457)
javanna Feb 17, 2022
1fa4427
Preserve context in snapshotDeletionListeners (#84089)
DaveCTurner Feb 17, 2022
2241673
Use static empty store files metadata (#84034)
DaveCTurner Feb 17, 2022
7570570
Avoid null threadContext in ResultDeduplicator (#84093)
DaveCTurner Feb 17, 2022
26f46d2
Update Lucene analysis base url (#84094)
amitmbm Feb 17, 2022
0d0b035
Revert "Clean up for superuser role name references (#83627)" (#84096)
ywangd Feb 17, 2022
11edc5e
Add API specs for OpenID Connect APIs
sethmlarson Feb 17, 2022
939715c
Make Metadata extend AbstractCollection (#83791)
arteam Feb 17, 2022
d9fbed5
[DOCS] Fix `ignore_unavailable` parameter definition (#84071)
jrodewig Feb 17, 2022
90e837c
Allow regular data streams to be migrated to tsdb data streams. (#83843)
martijnvg Feb 17, 2022
6e77123
[DOCS] Remove note about partial response from Bulk API docs (#84053)
jrodewig Feb 17, 2022
ec2ecc9
TSDB: Reject the nested object fields that are configured time_series…
weizijun Feb 17, 2022
ce80c1b
Shrink join queries in slow log (#83914)
nik9000 Feb 17, 2022
31ead1f
Group field caps response by index mapping hash (#83494)
dnhatn Feb 17, 2022
d4203b0
[DOCS] Clarify `orientation` usage for WKT and GeoJSON polygons (#84025)
jrodewig Feb 17, 2022
06c8848
Always re-run Feature migrations which have encountered errors (#83918)
gwbrown Feb 17, 2022
3dee8dd
Small formatting clean up (#84144)
nik9000 Feb 17, 2022
2aaa333
Mute GeoGridTilerTestCase#testGeoGridSetValuesBoundingBoxes_Unbounded…
jtibshirani Feb 17, 2022
aa90426
Fix spotless violation from last commit
jtibshirani Feb 17, 2022
b035019
Mute LdapSessionFactoryTests#testSslTrustIsReloaded
jtibshirani Feb 17, 2022
605d94e
Mute FrozenSearchableSnapshotsIntegTests#testCreateAndRestorePartialS…
jtibshirani Feb 18, 2022
a1c0aa0
Adjust auto-configuration related docs (#84080)
jkakavas Feb 20, 2022
1b48d38
Add JWT realm support for JWT validation (#83155)
justincr-elastic Feb 21, 2022
b68e886
Add Maps.newLinkedHashMapWithExpectedSize (#84054)
arteam Feb 21, 2022
dd53866
Do not deserialise the document when not needed in the fields fetch p…
iverase Feb 21, 2022
cbf128a
[DOCS] Fix shrink index API prereqs (#84197)
joegallo Feb 21, 2022
9e3638a
Fix testFollowerCheckerDetectsUnresponsiveNodeAfterMasterReelection (…
DaveCTurner Feb 22, 2022
cd0fb0e
Upgrade to lucene-9.1.0-snapshot-1336263051c (#83667)
mayya-sharipova Feb 22, 2022
0c3c4f6
Change to org.apache.lucene.tests
mayya-sharipova Feb 22, 2022
8067245
Fix spotless format
mayya-sharipova Feb 22, 2022
0b7aa9d
Make allocation explanations more actionable (#83983)
DaveCTurner Feb 22, 2022
1dee151
Only check for metadata input on the root object (#84140)
romseygeek Feb 22, 2022
3c8b48e
Mute test for issue 84212 (#84214)
Feb 22, 2022
266d2b8
Add extra section on doc-value-only fields to documentation (#84209)
ywelsch Feb 22, 2022
ec5e89d
update `readMap` to avoid resizing map during reading (#84045)
idegtiarenko Feb 22, 2022
ea95ac9
Mute test for issue 84218 (#84219)
Feb 22, 2022
7e6c828
Add new Threadpool for Cluster Coordination Activities (#83576)
original-brownbear Feb 22, 2022
dce06da
Shrink slow log for inner_hits (#84143)
nik9000 Feb 22, 2022
78c13c4
Update error message after lucene upgrade (#84228)
iverase Feb 22, 2022
c2146e7
[DOCS] Add read_pipeline privilege to transform setup (#84151)
lcawl Feb 22, 2022
6b5c692
Return result to cluster state ack listener (#84174)
DaveCTurner Feb 22, 2022
fa0cf35
Remove extra repo flag to access archive indices (#84222)
ywelsch Feb 22, 2022
4642f76
Package installation docs minor change (#84177)
jkakavas Feb 22, 2022
0eefaf8
[ML] Text structure finder caps exclude lines pattern at 1000 charact…
droberts195 Feb 22, 2022
4d2817e
Skip product header check in buggy versions (#84210)
DaveCTurner Feb 22, 2022
3ec1885
DOCS Security audit ignore for the `action` event field (#83583) (#8…
albertzaharovits Feb 22, 2022
b524f68
[ML] fix bug where initial scale from 0->1 could scale too high (#84244)
benwtrent Feb 22, 2022
b6cb850
Register Feature migration persistent task state named XContent (#84192)
pgomulka Feb 22, 2022
5609928
Trace Log Upload Timing in BlobStoreRepository (#84221)
original-brownbear Feb 22, 2022
83103e5
Clarify docs about CA fingerprints and Fleet (#84056)
dedemorton Feb 22, 2022
0953528
Fixes Time Series Cancellation Test (#84157)
imotov Feb 22, 2022
3eea30e
Removing io.netty.allocator.numDirectArenas override. (#76593)
nikitiuk0 Feb 23, 2022
3033b71
Introduce lookup runtime fields (#82385)
dnhatn Feb 23, 2022
b829327
Fix GeoGridTilerTestCase (#84164)
iverase Feb 23, 2022
e58cec2
[DOCS] Adds reference of transform reset option to the tutorial (#84194)
szabosteve Feb 23, 2022
eb530ee
[ML] Retry datafeed searches on skipped CCS clusters (#84052)
droberts195 Feb 23, 2022
bd0ae95
Mute DatafeedCcsIT.testDatafeedWithCcsRemoteUnavailable (#84269)
droberts195 Feb 23, 2022
b78ec63
Simplify reading a list and converting it to a map from stream (#84183)
idegtiarenko Feb 23, 2022
4235840
Replace getProject() references with injected services in task implem…
breskeby Feb 23, 2022
50171df
[ML] allow autoscaling to work when vertical scaling is possible (#84…
benwtrent Feb 23, 2022
7944a4d
[TRANSFORM] Remove HLRC from single node tests (#84220)
davidkyle Feb 23, 2022
512705c
SQL: Fix issues with format=txt when paging through result sets and i…
Feb 23, 2022
12322b0
[TRANSFORM] Mute TransformSurvivesUpgradeIT::testTransformRollingUpgr…
davidkyle Feb 23, 2022
23b85ed
SQL: Deprecate index_include_frozen request parameter and `FROZEN` ke…
Feb 23, 2022
030f8b1
[ML] Increase timeout for datafeed CCS tests (#84291)
droberts195 Feb 23, 2022
ba7a0cb
Document partial search results with skip_unavailable (#84057)
dnhatn Feb 23, 2022
c694a0f
[DOCS] Update 8.1.0 release notes (#84185) (#84302)
jrodewig Feb 23, 2022
27f2fd6
[DOCS] Update screenshots for ingest pipeline docs (#83845)
jrodewig Feb 23, 2022
8053f1a
[DOCS] Fix upgrade docs for 8.x (#84076)
jrodewig Feb 23, 2022
9661537
Remove `eof` from expect script (#84287)
pugnascotia Feb 23, 2022
c72c2ee
[DOCS] Add 8.0.1 release notes (#84213) (#84292)
jrodewig Feb 23, 2022
66660d9
Fix SimpleValidateQueryIT#testExplainValidateQueryTwoNodes (#84296)
Feb 23, 2022
7acafa4
Forward-port late deprecation info API changes to 8.x (#83675)
masseyke Feb 23, 2022
9f22340
test
probakowski Feb 23, 2022
7506b58
Merge remote-tracking branch 'origin/master' into complete-geoip
probakowski Feb 23, 2022
b5dbcaa
spotless
probakowski Feb 23, 2022
7e91f48
test rework
probakowski Feb 24, 2022
6dcd9ff
cleanup
probakowski Feb 24, 2022
39fbecc
cleanup
probakowski Feb 24, 2022
82eaaf3
cleanup
probakowski Feb 24, 2022
92bf15d
assertAcked
probakowski Feb 24, 2022
28e9e77
spotless
probakowski Feb 24, 2022
428f0ee
tests rework
probakowski Feb 24, 2022
34b8306
spotless
probakowski Feb 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/84028.yaml
@@ -0,0 +1,5 @@
pr: 84028
summary: Mark `GeoIpDownloaderTask` as completed after cancellation
area: Ingest
type: bug
issues: []
@@ -0,0 +1,95 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

package org.elasticsearch.ingest.geoip;

import org.elasticsearch.action.admin.cluster.node.tasks.list.ListTasksRequest;
import org.elasticsearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.persistent.PersistentTaskParams;
import org.elasticsearch.persistent.PersistentTasksCustomMetadata;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.reindex.ReindexPlugin;
import org.junit.After;

import java.util.Arrays;
import java.util.Collection;

import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;

public class GeoIpDownloaderTaskIT extends AbstractGeoIpIT {

protected static final String ENDPOINT = System.getProperty("geoip_endpoint");

@Override
protected Collection<Class<? extends Plugin>> nodePlugins() {
return Arrays.asList(ReindexPlugin.class, IngestGeoIpPlugin.class, IngestGeoIpSettingsPlugin.class);
}

@Override
protected Settings nodeSettings(int nodeOrdinal, Settings otherSettings) {
return Settings.builder()
.put(super.nodeSettings(nodeOrdinal, otherSettings))
.put(GeoIpDownloader.ENDPOINT_SETTING.getKey(), "http://invalid-endpoint")
.build();
}

@After
public void cleanUp() throws Exception {
assertAcked(
client().admin()
.cluster()
.prepareUpdateSettings()
.setPersistentSettings(
Settings.builder()
.putNull(GeoIpDownloaderTaskExecutor.ENABLED_SETTING.getKey())
.putNull(GeoIpDownloader.POLL_INTERVAL_SETTING.getKey())
.putNull("ingest.geoip.database_validity")
)
.get()
);
}

public void testTaskRemovedAfterCancellation() throws Exception {
assertAcked(
client().admin()
.cluster()
.prepareUpdateSettings()
.setPersistentSettings(Settings.builder().put(GeoIpDownloaderTaskExecutor.ENABLED_SETTING.getKey(), true))
.get()
);
assertBusy(() -> {
PersistentTasksCustomMetadata.PersistentTask<PersistentTaskParams> task = getTask();
assertNotNull(task);
assertTrue(task.isAssigned());
});
ListTasksResponse tasks = client().admin()
.cluster()
.listTasks(new ListTasksRequest().setActions("geoip-downloader[c]"))
.actionGet();
assertEquals(1, tasks.getTasks().size());
assertAcked(
client().admin()
.cluster()
.prepareUpdateSettings()
.setPersistentSettings(Settings.builder().put(GeoIpDownloaderTaskExecutor.ENABLED_SETTING.getKey(), false))
.get()
);
assertBusy(() -> {
ListTasksResponse tasks2 = client().admin()
.cluster()
.listTasks(new ListTasksRequest().setActions("geoip-downloader[c]"))
.actionGet();
assertEquals(0, tasks2.getTasks().size());
});
}

private PersistentTasksCustomMetadata.PersistentTask<PersistentTaskParams> getTask() {
return PersistentTasksCustomMetadata.getTaskWithId(clusterService().state(), GeoIpDownloader.GEOIP_DOWNLOADER);
}
}
Expand Up @@ -299,6 +299,7 @@ protected void onCancelled() {
if (scheduled != null) {
scheduled.cancel();
}
markAsCompleted();
}

@Override
Expand Down