Improvements in m0crate utility (KV test run).

[somnathbghule]

There are improvement areas such as,

1. Correctness in output logs (for all log levels) of m0crate KV test run.

	a. Following parameters are not being used by m0crate kv test but still displayed in the output logs.
		Before Fix:
			info: average size:          65536
			info: maximal size:          1048576
			info: block size:            0
		After Fix:
			No need to display this if KV test run is running.
	b. Following parameter shows wrong output.
		Before Fix:
			info: number of operations:  1000 (OP_COUT: 32 was passed. always shows the same value.). 
		After Fix:
			info: number of operations:  32
	c. If we read following output it looks like percentage of ops but its actually 
            count of number of operations remaining. 
		Before Fix:
			trace: dix: prcnt: [5, 4, 0, 0]		
		After Fix: 
			trace: dix: ops remaining: [5, 4, 0, 0]
	d.  Result should shows details about the test run,
            Before Fix:
                            result: all, 0.133912
            After Fix:
                          result: Total, 0.133912 s
	                          Avg time per ops.
	                          PUT, <time taken by put ops> s
	                          GET, <time taken by get ops> s and so on.

2. Implementation of specifying the specific KEY and VALUE sizes in kv test run, sizes can be number (int) or random.

    key_size=sizeof(struct m0_fid)
    value_size=RECORD_SIZE-key_size;
    here RECORD_SIZE is tunable parameter available in .yaml file.
    There is no way/option available to set specific size of key and value.

[welcome]

Thanks for opening this issue. A contributor should be by to give feedback soon. In the meantime, please check out the contributing guidelines and explore other ways you can get involved.

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @huanghua78 @mukundkanekar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[madhavemuri]

@imvenkip : Can you check this PR and verify it ?

[imvenkip]

@imvenkip : Can you check this PR and verify it ?
Sure @madhavemuri . Will check it.

[imvenkip]

@somnathbghule Could you add exactly used m0crate commands?

[somnathbghule]

@somnathbghule``` Could you add exactly used m0crate commands?

I used following command to run m0crate kv test.
cmd#
m0crate -S <kv_test .yaml file>

For example:
motr/m0crate/tests/test6.yaml template can be used in m0crate kv test run.
Note: you would require to update few fields in .yaml file like MOTR_LOCAL_ADDR, MOTR_HA_ADDR, PROF,  PROCESS_FID etc

[imvenkip]

@somnathbghule``` Could you add exactly used m0crate commands?

I used following command to run m0crate kv test.
cmd#
m0crate -S <kv_test .yaml file>

For example:
motr/m0crate/tests/test6.yaml template can be used in m0crate kv test run.
Note: you would require to update few fields in .yaml file like MOTR_LOCAL_ADDR, MOTR_HA_ADDR, PROF,  PROCESS_FID etc

@somnathbghule Could you share contents of kv_test .yaml file?

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @huanghua78 @mukundkanekar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[nkommuri]

@imvenkip @truptiatseagate Any improvements happened to m0crate as part of this ticket?

[imvenkip]

@nkommuri Already fix is available in main line.
Commit id: 7a734e7
Merged PR: #461

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @nkommuri @mehjoshi @huanghua78 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @nkommuri @mehjoshi @huanghua78 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[cortx-admin]

Zoheb Khan commented in Jira Server:

{noformat}
[root@cortx-data-headless-svc-ssc-vm-rhev4-1739 tmp]# m0crate -S m0crate_sample.yaml
info: starting workload 0
info: workload type index/3
info: random seed: 0
info: number of threads: 1
info: dix: End of operations.
result: total_s, 156.043289, avg_time_per_op_ns, 876352816.0, key_size_bytes, 16, value_size_bytes, -1, ops, 32
result: PUT, total_time_s, 75.440199, avg_time_per_op_ns, 357506212.0, ops, 32
result: GET, total_time_s, 0.000000, avg_time_per_op_ns, 0.0, ops, 0
result: NEXT, total_time_s, 0.000000, avg_time_per_op_ns, 0.0, ops, 0
result: DEL, total_time_s, 0.000000, avg_time_per_op_ns, 0.0, ops, 0Total: time=[156:043290138] ([4:876352816] per op) ops=32
PUT: [75:440198800] ([2:357506212] per op) ops=32
info: done workload 0
info: ---------------------------------------
{noformat}

[cortx-admin]

Zoheb Khan commented in Jira Server:

This is observed that when VALUE_SIZE is given as random, the value size is shown as -1 in output which is wrong or invalid since size(in bytes) can't be a negative number.

Workload file used:

 
{noformat}
CrateConfig_Sections:

• MOTR_CONFIG
• WORKLOAD_SPECMOTR_CONFIG:
  MOTR_LOCAL_ADDR: inet:tcp:cortx-data-headless-svc-ssc-vm-rhev4-1739@5001
  MOTR_HA_ADDR: inet:tcp:cortx-data-headless-svc-ssc-vm-rhev4-1739@2001
  PROF: 0x7000000000000001:0xde
  PROCESS_FID: 0x7200000000000001:0x8d
  LAYOUT_ID: 1 # Defines the UNIT_SIZE (9: 1MB)
  IS_OOSTORE: 1 # Is oostore-mode?
  IS_READ_VERIFY: 0 # Enable read-verify?
  TM_RECV_QUEUE_MIN_LEN: 16 # Minimum length of the receive queue
  M0_MAX_RPC_MSG_SIZE: 262144 # Maximum rpc message size
  IDX_SERVICE_ID: 1WORKLOAD_SPEC: # Workload specification section
  WORKLOAD: # First Workload
  WORKLOAD_TYPE: 0 # Index(0), IO(1)
  WORKLOAD_SEED: tstamp # SEED to the random number generator
  NUM_KVP: 2 # int
  NXRECORDS: 2 # int or default
  OP_COUNT: 64 # int [units] or unlimited, total operations count.
  EXEC_TIME: unlimited # int (seconds) or unlimited
  WARMUP_PUT_CNT: 0 # int (ops) or all
  WARMUP_DEL_RATIO: 0 # int (ops / ratio)
  KEY_PREFIX: random # int or random
  KEY_ORDER: ordered # ordered or random
  KEY_SIZE: random # int or random
  VALUE_SIZE: random # int or random
  MAX_KEY_SIZE: 16384 # int
  MAX_VALUE_SIZE: 32768 # int
  INDEX_FID: <7800000000000001:3> # fid
  PUT: 60 # int (in percentage)
  DEL: 10 # int (in percentage)
  GET: 20 # int (in percentage)
  NEXT: 10 # int (in percentage)
  LOG_LEVEL: 2 # err(0), warn(1), info(2), trace(3), debug(4)
  [root@cortx-data-headless-svc-ssc-vm-rhev4-1739 tmp]#
  [zoheb] 0:root@ssc-vm-rhev4-1739:~* {noformat}
  Result: {color:#de350b}Look for  "value_size_bytes, -1" in the below section{color}

 
{noformat}
[root@cortx-data-headless-svc-ssc-vm-rhev4-1739 tmp]# m0crate -S m0crate_sample_new.yaml
info: starting workload 0
info: workload type index/3
info: random seed: 0
info: number of threads: 1
info: dix: End of operations.
result: total_s, 4.577239, avg_time_per_op_ns, 73826440.0, key_size_bytes, 1809, value_size_bytes, -1, ops, 62
result: PUT, total_time_s, 2.848022, avg_time_per_op_ns, 74947955.0, ops, 38
result: GET, total_time_s, 0.773699, avg_time_per_op_ns, 64474937.0, ops, 12
result: NEXT, total_time_s, 0.600257, avg_time_per_op_ns, 100042859.0, ops, 6
result: DEL, total_time_s, 0.258094, avg_time_per_op_ns, 43015598.0, ops, 6Total: time=[4:577239329] ([0:073826440] per op) ops=62
PUT: [2:848022296] ([0:074947955] per op) ops=38
GET: [0:773699252] ([0:064474937] per op) ops=12
NEXT: [0:600257159] ([0:100042859] per op) ops=6
DEL: [0:258093589] ([0:043015598] per op) ops=6
info: done workload 0
info: ---------------------------------------
{noformat}
This needs to be fixed.

[stale]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @nkommuri @mehjoshi @huanghua78 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[cortx-admin]

Chandradhar Raval commented in Jira Server:

Marking this issue Closed as the same is not targeted for alpha release. If required the same can be reopened or tracked via story ticket in future