Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-3249: renew ContainerCache.INSTANCE in order to test it in a fresh state #705

Closed
wants to merge 39 commits into from
Closed

HDDS-3249: renew ContainerCache.INSTANCE in order to test it in a fresh state #705

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
f8e8084
HDDS-3249: renew ContainerCache.INSTANCE in order to test it in a fre…
Mar 23, 2020
49599c7
correct style
Mar 23, 2020
c64d86f
HDDS-3234. Fix retry interval default in Ozone client. (#698)
bharatviswa504 Mar 23, 2020
f2e259b
HDDS-3235.Change to default of max retry count for Ozone client (#699)
sadanand48 Mar 23, 2020
db89958
cleanup ContainerCache after TestBlockDeletingService and TestContain…
Mar 23, 2020
f259a64
remove unused import
Mar 23, 2020
081b933
remove setting default instance to null in shutdown method
Mar 23, 2020
ad7cd6a
HDDS-2878. Refactor MiniOzoneLoadGenerator to add more load generator…
mukul1987 Mar 24, 2020
cee9c6c
remove cache.isFull assertion from TestContainerCache
Mar 24, 2020
512d607
Revert "HDDS-3142. Create isolated enviornment for OM to test it with…
elek Mar 24, 2020
8249385
HDDS-3250. Create a separate log file for Warnings and Errors in Mini…
mukul1987 Mar 25, 2020
f6be766
HDDS-3243. Recon should not have the ability to send Create/Close Con…
avijayanhwx Mar 25, 2020
4682bab
HDDS-3164. Add Recon endpoint to serve missing containers and its met…
vivekratnavel Mar 25, 2020
07fcb79
HDDS-3284. ozonesecure-mr test fails due to lack of disk space (#725)
adoroszlai Mar 26, 2020
3d28568
HDDS-3074. Make the configuration of container scrub consistent. (#722)
cku328 Mar 27, 2020
7d132ce
HDDS-3179. Pipeline placement based on Topology does not have fallbac…
timmylicheng Mar 27, 2020
eece604
HDDS-3281. Add timeouts to all robot tests (#723)
hanishakoneru Mar 27, 2020
a661a75
HDDS-3288: Update default RPC handler SCM/OM count to 100 (#729)
rakeshadr Mar 28, 2020
099ab62
HDDS-3273. getConf does not return all OM addresses. (#727)
swagle Mar 28, 2020
9c829fb
HDDS-3234. Fix retry interval default in Ozone client. (#698)
bharatviswa504 Mar 23, 2020
021bed6
HDDS-3235.Change to default of max retry count for Ozone client (#699)
sadanand48 Mar 23, 2020
da528e0
HDDS-2878. Refactor MiniOzoneLoadGenerator to add more load generator…
mukul1987 Mar 24, 2020
725ceed
Revert "HDDS-3142. Create isolated enviornment for OM to test it with…
elek Mar 24, 2020
d4e4be7
HDDS-3250. Create a separate log file for Warnings and Errors in Mini…
mukul1987 Mar 25, 2020
28d5d6a
HDDS-3243. Recon should not have the ability to send Create/Close Con…
avijayanhwx Mar 25, 2020
2f27380
HDDS-3164. Add Recon endpoint to serve missing containers and its met…
vivekratnavel Mar 25, 2020
3da3258
HDDS-3284. ozonesecure-mr test fails due to lack of disk space (#725)
adoroszlai Mar 26, 2020
c19a3a5
HDDS-3074. Make the configuration of container scrub consistent. (#722)
cku328 Mar 27, 2020
18a4b05
HDDS-3179. Pipeline placement based on Topology does not have fallbac…
timmylicheng Mar 27, 2020
42d8d25
HDDS-3281. Add timeouts to all robot tests (#723)
hanishakoneru Mar 27, 2020
37281ac
HDDS-3288: Update default RPC handler SCM/OM count to 100 (#729)
rakeshadr Mar 28, 2020
562ac8b
HDDS-3273. getConf does not return all OM addresses. (#727)
swagle Mar 28, 2020
c2a0d48
HDDS-3249: renew ContainerCache.INSTANCE in order to test it in a fre…
Mar 23, 2020
aa7cfac
correct style
Mar 23, 2020
bbdd30d
cleanup ContainerCache after TestBlockDeletingService and TestContain…
Mar 23, 2020
74e5804
remove unused import
Mar 23, 2020
af28759
remove setting default instance to null in shutdown method
Mar 23, 2020
744d4ea
remove cache.isFull assertion from TestContainerCache
Mar 24, 2020
3421e2b
Merge remote-tracking branch 'origin/HDDS-3249' into HDDS-3249
Mar 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 0 additions & 34 deletions dev-support/byteman/mock-scm.btm
View file
Open in desktop

This file was deleted.

92 changes: 0 additions & 92 deletions hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/freon/FakeClusterTopology.java
View file
Open in desktop

This file was deleted.

100 changes: 0 additions & 100 deletions ...common/src/main/java/org/apache/hadoop/hdds/freon/FakeScmBlockLocationProtocolClient.java
View file
Open in desktop

This file was deleted.

76 changes: 0 additions & 76 deletions ...on/src/main/java/org/apache/hadoop/hdds/freon/FakeScmContainerLocationProtocolClient.java
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/DatanodeDetails.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ private DatanodeDetails(String uuid, String ipAddress, String hostName,
this.certSerialId = certSerialId;
}

protected DatanodeDetails(DatanodeDetails datanodeDetails) {
public DatanodeDetails(DatanodeDetails datanodeDetails) {
super(datanodeDetails.getHostName(), datanodeDetails.getNetworkLocation(),
datanodeDetails.getCost());
this.uuid = datanodeDetails.uuid;
Expand Down
4 changes: 2 additions & 2 deletions hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/ScmConfigKeys.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ public final class ScmConfigKeys {
"dfs.ratis.client.request.retry.interval";
public static final TimeDuration
DFS_RATIS_CLIENT_REQUEST_RETRY_INTERVAL_DEFAULT =
TimeDuration.valueOf(1000, TimeUnit.MILLISECONDS);
TimeDuration.valueOf(15000, TimeUnit.MILLISECONDS);
public static final String DFS_RATIS_SERVER_RETRY_CACHE_TIMEOUT_DURATION_KEY =
"dfs.ratis.server.retry-cache.timeout.duration";
public static final TimeDuration
Expand Down Expand Up @@ -221,7 +221,7 @@ public final class ScmConfigKeys {

public static final String OZONE_SCM_HANDLER_COUNT_KEY =
"ozone.scm.handler.count.key";
public static final int OZONE_SCM_HANDLER_COUNT_DEFAULT = 10;
public static final int OZONE_SCM_HANDLER_COUNT_DEFAULT = 100;

public static final String OZONE_SCM_SECURITY_HANDLER_COUNT_KEY =
"ozone.scm.security.handler.count.key";
Expand Down
7 changes: 1 addition & 6 deletions ...he/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,18 +111,13 @@ private ScmContainerLocationResponse submitRequest(
builderConsumer.accept(builder);
ScmContainerLocationRequest wrapper = builder.build();

response = submitRpcRequest(wrapper);
response = rpcProxy.submitRequest(NULL_RPC_CONTROLLER, wrapper);
} catch (ServiceException ex) {
throw ProtobufHelper.getRemoteException(ex);
}
return response;
}

private ScmContainerLocationResponse submitRpcRequest(
ScmContainerLocationRequest wrapper) throws ServiceException {
return rpcProxy.submitRequest(NULL_RPC_CONTROLLER, wrapper);
}

/**
* Asks SCM where a container should be allocated. SCM responds with the set
* of datanodes that should be used creating this container. Ozone/SCM only
Expand Down
2 changes: 1 addition & 1 deletion hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ public final class OzoneConfigKeys {

public static final String OZONE_CLIENT_MAX_RETRIES =
"ozone.client.max.retries";
public static final int OZONE_CLIENT_MAX_RETRIES_DEFAULT = 100;
public static final int OZONE_CLIENT_MAX_RETRIES_DEFAULT = 5;
public static final String OZONE_CLIENT_RETRY_INTERVAL =
"ozone.client.retry.interval";
public static final TimeDuration OZONE_CLIENT_RETRY_INTERVAL_DEFAULT =
Expand Down
8 changes: 4 additions & 4 deletions hadoop-hdds/common/src/main/resources/ozone-default.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@
</property>
<property>
<name>dfs.ratis.client.request.retry.interval</name>
<value>1000ms</value>
<value>15000ms</value>
<tag>OZONE, RATIS, MANAGEMENT</tag>
<description>Interval between successive retries for a ratis client request.
</description>
Expand Down Expand Up @@ -412,7 +412,7 @@
</property>
<property>
<name>ozone.client.max.retries</name>
<value>100</value>
<value>5</value>
<tag>OZONE, CLIENT</tag>
<description>Maximum number of retries by Ozone Client on encountering
exception while writing a key.
Expand Down Expand Up @@ -505,7 +505,7 @@
</property>
<property>
<name>ozone.om.handler.count.key</name>
<value>20</value>
<value>100</value>
<tag>OM, PERFORMANCE</tag>
<description>
The number of RPC handler threads for OM service endpoints.
Expand Down Expand Up @@ -918,7 +918,7 @@
</property>
<property>
<name>ozone.scm.handler.count.key</name>
<value>10</value>
<value>100</value>
<tag>OZONE, MANAGEMENT, PERFORMANCE</tag>
<description>
The number of RPC handler threads for each SCM service
Expand Down
Loading