Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SOLR-14702: Remove oppressive language (part1) #1712

Closed
wants to merge 23 commits into from
Closed

SOLR-14702: Remove oppressive language (part1) #1712

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f9c1870
trying this again in clean pr. get this trash out the project. - Harr…
MarcusSorealheis Aug 2, 2020
fb3bdd5
detail the change.
MarcusSorealheis Aug 2, 2020
f26ab19
fix bad replacements. Thanks Kevin.
MarcusSorealheis Aug 2, 2020
8f915e0
cleanup build.
MarcusSorealheis Aug 2, 2020
3303ab5
removing primary from ivy files.
MarcusSorealheis Aug 2, 2020
51dc39c
clean docs a bit more.
MarcusSorealheis Aug 2, 2020
d809913
Harriet is back.
MarcusSorealheis Aug 2, 2020
0d7b063
Harriet changes the name.
MarcusSorealheis Aug 2, 2020
bc94423
Tubman.
MarcusSorealheis Aug 2, 2020
f9ca9ce
Harriet is back again.
MarcusSorealheis Aug 2, 2020
6b1c8dd
fix lucene test.
MarcusSorealheis Aug 2, 2020
670f464
leaving the changes.txt.
MarcusSorealheis Aug 2, 2020
0b41c30
changing file names because they should be free from bondage.
MarcusSorealheis Aug 2, 2020
a040ef1
Toussaint
MarcusSorealheis Aug 2, 2020
987e838
Detailing the change and a big step forward for this project.
MarcusSorealheis Aug 2, 2020
ad25d07
Cleaning up a few more test files and image resources.
MarcusSorealheis Aug 2, 2020
b0f4af7
remove an errant replacement and an applicable replacement I'd previo…
MarcusSorealheis Aug 2, 2020
768c2eb
update the ref guide for more clarity per suggestions.
MarcusSorealheis Aug 2, 2020
8c9aba7
Harriet made 19 trips, ternary to handle deprecated terminology in th…
MarcusSorealheis Aug 3, 2020
fbc9cf6
fix comment.
MarcusSorealheis Aug 3, 2020
ce157e0
change repo to correct one.
MarcusSorealheis Aug 3, 2020
740e516
update another repo url and a couple docs in the ref guide.
MarcusSorealheis Aug 3, 2020
e2e92ba
thanks to Erick Erickson for fixing the test here. Huge. Go Blue..
MarcusSorealheis Aug 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion lucene/suggest/src/test/org/apache/lucene/search/suggest/Top50KWiki.utf8
View file
Open in desktop
Expand Up @@ -6424,7 +6424,7 @@ nationalteam|46926
instruction|46925
chem|46922
tasmania|46922
slave|46918
secondary|46918
violent|46916
flora|46900
manila|46893
Expand Down
8 changes: 5 additions & 3 deletions solr/CHANGES.txt
View file
Open in desktop
Expand Up @@ -35,6 +35,8 @@ Improvements

* SOLR-10814: Add short-name feature to RuleBasedAuthz plugin (Mike Drob, Hrishikesh Gadre)

* SOLR-14702: Pull oppressive language out the project (marcussorealheis)

Other Changes
----------------------
* SOLR-14656: Autoscaling framework removed (Ishan Chattopadhyaya, noble, Ilan Ginzburg)
Expand Down Expand Up @@ -3223,7 +3225,7 @@ Bug Fixes
Optimizations
----------------------

* SOLR-11920: IndexFetcher now fetches only those files (from master/leader) that are different. This
* SOLR-11920: IndexFetcher now fetches only those files (from primary/leader) that are different. This
differential fetching now speeds up recovery times when full index replication is needed, but only
a few segments diverge. (Ishan Chattopadhyaya, Shaun Sabo, John Gallagher)

Expand Down Expand Up @@ -7220,7 +7222,7 @@ Other Changes
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)

* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker)
anymore. The Replication tab is only beneficial to users running Solr in primary-secondary mode. (Varun Thacker)

* SOLR-8985: Added back support for 'includeDynamic' flag to /schema/fields endpoint (noble)

Expand Down Expand Up @@ -7836,7 +7838,7 @@ Other Changes
(Jason Gerlowski, Benoit Vanalderweireldt, shalin)

* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed
anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker)
anymore. The Replication tab is only beneficial to users running Solr in primary-secondary mode. (Varun Thacker)

* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy)

Expand Down
2 changes: 1 addition & 1 deletion ...rthandler-extras/src/test-files/dihextras/solr/collection1/conf/dataimport-solrconfig.xml
View file
Open in desktop
Expand Up @@ -162,7 +162,7 @@

<!-- Maximum number of searchers that may be warming in the background
concurrently. An error is returned if this limit is exceeded. Recommend
1-2 for read-only slaves, higher for masters w/o cache warming. -->
1-2 for read-only secondaries, higher for primaries w/o cache warming. -->
<maxWarmingSearchers>4</maxWarmingSearchers>

</query>
Expand Down
2 changes: 1 addition & 1 deletion ...b/dataimporthandler/src/test-files/dih/solr/collection1/conf/contentstream-solrconfig.xml
View file
Open in desktop
Expand Up @@ -162,7 +162,7 @@

<!-- Maximum number of searchers that may be warming in the background
concurrently. An error is returned if this limit is exceeded. Recommend
1-2 for read-only slaves, higher for masters w/o cache warming. -->
1-2 for read-only secondaries, higher for primaries w/o cache warming. -->
<maxWarmingSearchers>4</maxWarmingSearchers>

</query>
Expand Down
2 changes: 1 addition & 1 deletion ...rthandler/src/test-files/dih/solr/collection1/conf/dataimport-nodatasource-solrconfig.xml
View file
Open in desktop
Expand Up @@ -164,7 +164,7 @@

<!-- Maximum number of searchers that may be warming in the background
concurrently. An error is returned if this limit is exceeded. Recommend
1-2 for read-only slaves, higher for masters w/o cache warming. -->
1-2 for read-only secondaries, higher for primaries w/o cache warming. -->
<maxWarmingSearchers>4</maxWarmingSearchers>

</query>
Expand Down
2 changes: 1 addition & 1 deletion ...trib/dataimporthandler/src/test-files/dih/solr/collection1/conf/dataimport-solrconfig.xml
View file
Open in desktop
Expand Up @@ -162,7 +162,7 @@

<!-- Maximum number of searchers that may be warming in the background
concurrently. An error is returned if this limit is exceeded. Recommend
1-2 for read-only slaves, higher for masters w/o cache warming. -->
1-2 for read-only secondaries, higher for primaries w/o cache warming. -->
<maxWarmingSearchers>4</maxWarmingSearchers>

</query>
Expand Down
2 changes: 1 addition & 1 deletion ...contrib/dataimporthandler/src/test-files/solr/configsets/dihconfigset/conf/solrconfig.xml
View file
Open in desktop
Expand Up @@ -162,7 +162,7 @@

<!-- Maximum number of searchers that may be warming in the background
concurrently. An error is returned if this limit is exceeded. Recommend
1-2 for read-only slaves, higher for masters w/o cache warming. -->
1-2 for read-only secondaries, higher for primaries w/o cache warming. -->
<maxWarmingSearchers>4</maxWarmingSearchers>

</query>
Expand Down
4 changes: 2 additions & 2 deletions solr/contrib/prometheus-exporter/conf/solr-exporter-config.xml
View file
Open in desktop
Expand Up @@ -985,7 +985,7 @@
(if $parent_key_item_len == 5 then $parent_key_items[3] else "" end) as $shard |
(if $parent_key_item_len == 5 then $parent_key_items[4] else "" end) as $replica |
(if $parent_key_item_len == 5 then ($collection + "_" + $shard + "_" + $replica) else $core end) as $core |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isMaster") as $object |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isPrimary") as $object |
$object.key | split(".")[0] as $category |
$object.key | split(".")[1] as $handler |
(if $object.value == true then 1.0 else 0.0 end) as $value |
Expand Down Expand Up @@ -1018,7 +1018,7 @@
(if $parent_key_item_len == 5 then $parent_key_items[3] else "" end) as $shard |
(if $parent_key_item_len == 5 then $parent_key_items[4] else "" end) as $replica |
(if $parent_key_item_len == 5 then ($collection + "_" + $shard + "_" + $replica) else $core end) as $core |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isSlave") as $object |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isSecondary") as $object |
$object.key | split(".")[0] as $category |
$object.key | split(".")[1] as $handler |
(if $object.value == true then 1.0 else 0.0 end) as $value |
Expand Down
8 changes: 4 additions & 4 deletions ...metheus-exporter/src/test-files/conf/prometheus-solr-exporter-integration-test-config.xml
View file
Open in desktop
Expand Up @@ -989,7 +989,7 @@
(if $parent_key_item_len == 5 then $parent_key_items[3] else "" end) as $shard |
(if $parent_key_item_len == 5 then $parent_key_items[4] else "" end) as $replica |
(if $parent_key_item_len == 5 then ($collection + "_" + $shard + "_" + $replica) else $core end) as $core |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isMaster") as $object |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isPrimary") as $object |
$object.key | split(".")[0] as $category |
$object.key | split(".")[1] as $handler |
(if $object.value == true then 1.0 else 0.0 end) as $value |
Expand Down Expand Up @@ -1022,13 +1022,13 @@
(if $parent_key_item_len == 5 then $parent_key_items[3] else "" end) as $shard |
(if $parent_key_item_len == 5 then $parent_key_items[4] else "" end) as $replica |
(if $parent_key_item_len == 5 then ($collection + "_" + $shard + "_" + $replica) else $core end) as $core |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isSlave") as $object |
$parent.value | to_entries | .[] | select(.key == "REPLICATION./replication.isSecondary") as $object |
$object.key | split(".")[0] as $category |
$object.key | split(".")[1] as $handler |
(if $object.value == true then 1.0 else 0.0 end) as $value |
if $parent_key_item_len == 3 then
{
name: "solr_metrics_core_replication_slave",
name: "solr_metrics_core_replication_secondary",
type: "GAUGE",
help: "See following URL: https://lucene.apache.org/solr/guide/metrics-reporting.html",
label_names: ["category", "handler", "core"],
Expand All @@ -1037,7 +1037,7 @@
}
else
{
name: "solr_metrics_core_replication_slave",
name: "solr_metrics_core_replication_secondary",
type: "GAUGE",
help: "See following URL: https://lucene.apache.org/solr/guide/metrics-reporting.html",
label_names: ["category", "handler", "core", "collection", "shard", "replica"],
Expand Down
30 changes: 15 additions & 15 deletions solr/core/src/java/org/apache/solr/handler/IndexFetcher.java
View file
Open in desktop
Expand Up @@ -169,7 +169,7 @@ public class IndexFetcher {

private boolean downloadTlogFiles = false;

private boolean skipCommitOnMasterVersionZero = true;
private boolean skipCommitOnPrimaryVersionZero = true;

private boolean clearLocalIndexFirst = false;

Expand Down Expand Up @@ -236,9 +236,9 @@ public IndexFetcher(@SuppressWarnings({"rawtypes"})final NamedList initArgs, fin
if (fetchFromLeader != null && fetchFromLeader instanceof Boolean) {
this.fetchFromLeader = (boolean) fetchFromLeader;
}
Object skipCommitOnMasterVersionZero = initArgs.get(SKIP_COMMIT_ON_MASTER_VERSION_ZERO);
if (skipCommitOnMasterVersionZero != null && skipCommitOnMasterVersionZero instanceof Boolean) {
this.skipCommitOnMasterVersionZero = (boolean) skipCommitOnMasterVersionZero;
Object skipCommitOnPrimaryVersionZero = initArgs.get(SKIP_COMMIT_ON_MASTER_VERSION_ZERO);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an update here? Also, since you changed the parameter used, this needs to handle back compatibility

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only one that needs to change, possibly. Still stitching it all together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't have to handle back-compat in this PR. I can commit to it prior to 9.0's release and even prioritize the work before opening PRs on the other work I am doing.

if (skipCommitOnPrimaryVersionZero != null && skipCommitOnPrimaryVersionZero instanceof Boolean) {
this.skipCommitOnPrimaryVersionZero = (boolean) skipCommitOnPrimaryVersionZero;
}
String masterUrl = (String) initArgs.get(MASTER_URL);
if (masterUrl == null && !this.fetchFromLeader)
Expand Down Expand Up @@ -420,19 +420,19 @@ IndexFetchResult fetchLatestIndex(boolean forceReplication, boolean forceCoreRel
} catch (Exception e) {
final String errorMsg = e.toString();
if (!Strings.isNullOrEmpty(errorMsg) && errorMsg.contains(INTERRUPT_RESPONSE_MESSAGE)) {
log.warn("Master at: {} is not available. Index fetch failed by interrupt. Exception: {}", masterUrl, errorMsg);
log.warn("Primary at: {} is not available. Index fetch failed by interrupt. Exception: {}", masterUrl, errorMsg);
return new IndexFetchResult(IndexFetchResult.FAILED_BY_INTERRUPT_MESSAGE, false, e);
} else {
log.warn("Master at: {} is not available. Index fetch failed by exception: {}", masterUrl, errorMsg);
log.warn("Primary at: {} is not available. Index fetch failed by exception: {}", masterUrl, errorMsg);
return new IndexFetchResult(IndexFetchResult.FAILED_BY_EXCEPTION_MESSAGE, false, e);
}
}

long latestVersion = (Long) response.get(CMD_INDEX_VERSION);
long latestGeneration = (Long) response.get(GENERATION);

log.info("Master's generation: {}", latestGeneration);
log.info("Master's version: {}", latestVersion);
log.info("Primary's generation: {}", latestGeneration);
log.info("Primary's version: {}", latestVersion);
Comment on lines +434 to +435
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my point, all these log in IndexFetcher talk about Master/Primary even when running in SolrCloud mode. This is likely adding to the confusion that Shawn talked about in the Jira. If these were "leader"/"Follower", we'd have solve it :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gothca

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shawn suggested that we shouldn't use the same terminology for the legacy architecture. I've updated the first PR in any event. At this time, keeping the same terminology seems to be the right way to go unless I rewrite the replication handler. There's an argument for that because I'm sure there are areas where we could improve, but it won't happen in this PR.


// TODO: make sure that getLatestCommit only returns commit points for the main index (i.e. no side-car indexes)
IndexCommit commit = solrCore.getDeletionPolicy().getLatestCommit();
Expand All @@ -453,23 +453,23 @@ IndexFetchResult fetchLatestIndex(boolean forceReplication, boolean forceCoreRel
}

if (log.isInfoEnabled()) {
log.info("Slave's generation: {}", commit.getGeneration());
log.info("Slave's version: {}", IndexDeletionPolicyWrapper.getCommitTimestamp(commit)); // logOK
log.info("Secondary's generation: {}", commit.getGeneration());
log.info("Secondary's version: {}", IndexDeletionPolicyWrapper.getCommitTimestamp(commit)); // logOK
}

if (latestVersion == 0L) {
if (commit.getGeneration() != 0) {
// since we won't get the files for an empty index,
// we just clear ours and commit
log.info("New index in Master. Deleting mine...");
log.info("New index in Primary. Deleting mine...");
RefCounted<IndexWriter> iw = solrCore.getUpdateHandler().getSolrCoreState().getIndexWriter(solrCore);
try {
iw.get().deleteAll();
} finally {
iw.decref();
}
assert TestInjection.injectDelayBeforeSlaveCommitRefresh();
if (skipCommitOnMasterVersionZero) {
assert TestInjection.injectDelayBeforeSecondaryCommitRefresh();
if (skipCommitOnPrimaryVersionZero) {
openNewSearcherAndUpdateCommitPoint();
} else {
SolrQueryRequest req = new LocalSolrQueryRequest(solrCore, new ModifiableSolrParams());
Expand All @@ -486,7 +486,7 @@ IndexFetchResult fetchLatestIndex(boolean forceReplication, boolean forceCoreRel
// TODO: Should we be comparing timestamps (across machines) here?
if (!forceReplication && IndexDeletionPolicyWrapper.getCommitTimestamp(commit) == latestVersion) {
//master and slave are already in sync just return
log.info("Slave in sync with master.");
log.info("Secondary in sync with master.");
successfulInstall = true;
return IndexFetchResult.ALREADY_IN_SYNC;
}
Expand Down Expand Up @@ -1998,7 +1998,7 @@ public void destroy() {
HttpClientUtil.close(myHttpClient);
}

String getMasterUrl() {
String getPrimaryUrl() {
return masterUrl;
}

Expand Down