Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Reduce unittest time cost #268

Closed
yhmo opened this issue Nov 9, 2019 · 1 comment
Closed

[FEATURE] Reduce unittest time cost #268

yhmo opened this issue Nov 9, 2019 · 1 comment
Assignees
Labels
kind/enhancement Issues or changes related to enhancement
Projects

Comments

@yhmo
Copy link
Contributor

yhmo commented Nov 9, 2019

Is your feature request related to a problem? Please describe.
Now the milvus unittest totally cost about 16 minutes. I wish milvus unittest time cost can be reduce to less than 10 minutes

Describe the solution you'd like
Find out bottleneck and fix it.
A list of test case time cost:
[ OK ] DBTest.CONFIG_TEST (267 ms)
====[ OK ] DBTest.DB_TEST (13208 ms)
[ OK ] DBTest.SEARCH_TEST (3154 ms)
[ OK ] DBTest.PRELOADTABLE_TEST (2344 ms)
[ OK ] DBTest.SHUTDOWN_TEST (1080 ms)
==[ OK ] DBTest.INDEX_TEST (5266 ms)
[ OK ] DBTest2.ARHIVE_DISK_CHECK (2110 ms)
[ OK ] DBTest2.DELETE_TEST (4137 ms)
[ OK ] DBTest2.DELETE_BY_RANGE_TEST (2142 ms)
====[ OK ] MySqlDBTest.DB_TEST (36311 ms)
[ OK ] MySqlDBTest.SEARCH_TEST (4250 ms)
[ OK ] MySqlDBTest.ARHIVE_DISK_CHECK (2193 ms)
[ OK ] MySqlDBTest.DELETE_TEST (1160 ms)
[ OK ] EngineTest.FACTORY_TEST (1075 ms)
[ OK ] EngineTest.ENGINE_IMPL_TEST (1183 ms)
[ OK ] MemManagerTest.VECTOR_SOURCE_TEST (73 ms)
[ OK ] MemManagerTest.MEM_TABLE_FILE_TEST (1263 ms)
====[ OK ] MemManagerTest.MEM_TABLE_TEST (12986 ms)
==[ OK ] MemManagerTest2.SERIAL_INSERT_SEARCH_TEST (5296 ms)
==[ OK ] MemManagerTest2.INSERT_TEST (8906 ms)
====[ OK ] MemManagerTest2.CONCURRENT_INSERT_SEARCH_TEST (18072 ms)
==[ OK ] MemManagerTest2.VECTOR_IDS_TEST (6058 ms)
[ OK ] MetaTest.TABLE_TEST (75 ms)
[ OK ] MetaTest.TABLE_FILE_TEST (97 ms)
[ OK ] MetaTest.ARCHIVE_TEST_DAYS (1315 ms)
[ OK ] MetaTest.ARCHIVE_TEST_DISK (190 ms)
[ OK ] MetaTest.TABLE_FILES_TEST (420 ms)
[ OK ] MetaTest.INDEX_TEST (84 ms)
[ OK ] MySqlMetaTest.TABLE_TEST (162 ms)
[ OK ] MySqlMetaTest.TABLE_FILE_TEST (162 ms)
[ OK ] MySqlMetaTest.ARCHIVE_TEST_DAYS (1587 ms)
[ OK ] MySqlMetaTest.ARCHIVE_TEST_DISK (305 ms)
[ OK ] MySqlMetaTest.TABLE_FILES_TEST (514 ms)
[ OK ] MySqlMetaTest.INDEX_TEST (155 ms)
[ OK ] DBMiscTest.EXCEPTION_TEST (0 ms)
[ OK ] DBMiscTest.OPTIONS_TEST (0 ms)
[ OK ] DBMiscTest.META_TEST (40 ms)
[ OK ] DBMiscTest.UTILS_TEST (1 ms)
[ OK ] DBSearchTest.MERGE_RESULT_SET_TEST (4 ms)
[ OK ] DBSearchTest.REDUCE_PERF_TEST (4057 ms)
[ OK ] GPURESTEST.copyandsearch (1572 ms)
[ OK ] GPURESTEST.trainandsearch (1649 ms)
[ OK ] IDMAPTest.idmap_basic (228 ms)
[ OK ] IDMAPTest.idmap_serialize (48 ms)
[ OK ] IDMAPTest.copy_test (1269 ms)
==[ OK ] IVFParameters/IVFTest.ivf_basic/0 (5382 ms)
[ OK ] IVFParameters/IVFTest.ivf_basic/1 (1636 ms)
======[ OK ] IVFParameters/IVFTest.ivf_basic/2 (58760 ms)
======[ OK ] IVFParameters/IVFTest.ivf_basic/3 (53260 ms)
==[ OK ] IVFParameters/IVFTest.ivf_basic/4 (5855 ms)
[ OK ] IVFParameters/IVFTest.ivf_basic/5 (951 ms)
====[ OK ] IVFParameters/IVFTest.ivf_serialize/0 (11694 ms)
[ OK ] IVFParameters/IVFTest.ivf_serialize/1 (1288 ms)
======[ OK ] IVFParameters/IVFTest.ivf_serialize/2 (119152 ms)
======[ OK ] IVFParameters/IVFTest.ivf_serialize/3 (106097 ms)
====[ OK ] IVFParameters/IVFTest.ivf_serialize/4 (11010 ms)
[ OK ] IVFParameters/IVFTest.ivf_serialize/5 (1002 ms)
==[ OK ] IVFParameters/IVFTest.clone_test/0 (6460 ms)
[ OK ] IVFParameters/IVFTest.clone_test/1 (547 ms)
======[ OK ] IVFParameters/IVFTest.clone_test/2 (60657 ms)
======[ OK ] IVFParameters/IVFTest.clone_test/3 (54374 ms)
==[ OK ] IVFParameters/IVFTest.clone_test/4 (6714 ms)
[ OK ] IVFParameters/IVFTest.clone_test/5 (777 ms)
[ OK ] KDTTest.kdt_basic (3153 ms)
[ OK ] MetricbaseTest.METRICBASE_TEST (23 ms)
====[ OK ] MetricTest.METRIC_TEST (23858 ms)
[ OK ] MetricTest.COLLECTOR_METRICS_TEST (44 ms)
[ OK ] PrometheusTest.PROMETHEUS_TEST (211 ms)
======[ OK ] NSGInterfaceTest.basic_test (35344 ms)
[ OK ] AlgorithmTest.SHORTESTPATH_TEST (117 ms)
[ OK ] NodeTest.ADD_NEIGHBOUR (0 ms)
[ OK ] NodeTest.REPEAT_ADD_NEIGHBOUR (0 ms)
[ OK ] NodeTest.GET_NEIGHBOURS (0 ms)
[ OK ] NormalTest.INST_TEST (7 ms)
[ OK ] ResourceBaseTest.NAME (16 ms)
[ OK ] ResourceBaseTest.TYPE (16 ms)
[ OK ] ResourceBaseTest.DEVICE_ID (17 ms)
[ OK ] ResourceBaseTest.HAS_LOADER (16 ms)
[ OK ] ResourceBaseTest.HAS_EXECUTOR (17 ms)
[ OK ] ResourceAdvanceTest.DISK_RESOURCE_TEST (16 ms)
[ OK ] ResourceAdvanceTest.CPU_RESOURCE_TEST (17 ms)
[ OK ] ResourceAdvanceTest.GPU_RESOURCE_TEST (17 ms)
[ OK ] ResourceAdvanceTest.TEST_RESOURCE_TEST (16 ms)
[ OK ] ResourceFactoryTest.CREATE (11 ms)
[ OK ] ResourceMgrBaseTest.ADD (15 ms)
[ OK ] ResourceMgrBaseTest.ADD_DISK (15 ms)
[ OK ] ResourceMgrBaseTest.CONNECT (18 ms)
[ OK ] ResourceMgrBaseTest.INVALID_CONNECT (16 ms)
[ OK ] ResourceMgrBaseTest.CLEAR (8 ms)
[ OK ] ResourceMgrBaseTest.GET_DISK_RESOURCES (9 ms)
[ OK ] ResourceMgrBaseTest.GET_ALL_RESOURCES (9 ms)
[ OK ] ResourceMgrBaseTest.GET_COMPUTE_RESOURCES (9 ms)
[ OK ] ResourceMgrBaseTest.GET_RESOURCE_BY_TYPE_AND_DEVICEID (9 ms)
[ OK ] ResourceMgrBaseTest.GET_RESOURCE_BY_NAME (9 ms)
[ OK ] ResourceMgrBaseTest.GET_NUM_OF_RESOURCE (8 ms)
[ OK ] ResourceMgrBaseTest.GET_NUM_OF_COMPUTE_RESOURCE (9 ms)
[ OK ] ResourceMgrBaseTest.GET_NUM_OF_GPU_RESOURCE (9 ms)
[ OK ] ResourceMgrBaseTest.DUMP (8 ms)
[ OK ] ResourceMgrBaseTest.DUMP_TASKTABLES (9 ms)
[ OK ] ResourceMgrAdvanceTest.REGISTER_SUBSCRIBER (1010 ms)
[ OK ] SchedulerTest.ON_LOAD_COMPLETED (3073 ms)
[ OK ] SchedulerTest.PUSH_TASK_TO_NEIGHBOUR_RANDOMLY_TEST (3025 ms)
[ OK ] TaskTest.INVALID_INDEX (0 ms)
[ OK ] TaskTableItemTest.CONSTRUCT (0 ms)
[ OK ] TaskTableItemTest.DESTRUCT (0 ms)
[ OK ] TaskTableItemTest.IS_FINISH (0 ms)
[ OK ] TaskTableItemTest.DUMP (0 ms)
[ OK ] TaskTableItemTest.LOAD (0 ms)
[ OK ] TaskTableItemTest.LOADED (0 ms)
[ OK ] TaskTableItemTest.EXECUTE (0 ms)
[ OK ] TaskTableItemTest.EXECUTED (0 ms)
[ OK ] TaskTableItemTest.MOVE (0 ms)
[ OK ] TaskTableItemTest.MOVED (0 ms)
[ OK ] TaskTableBaseTest.SUBSCRIBER (4 ms)
[ OK ] TaskTableBaseTest.PUT_TASK (4 ms)
[ OK ] TaskTableBaseTest.PUT_INVALID_TEST (4 ms)
[ OK ] TaskTableBaseTest.PUT_BATCH (3 ms)
[ OK ] TaskTableBaseTest.SIZE (4 ms)
[ OK ] TaskTableBaseTest.OPERATOR (4 ms)
[ OK ] TaskTableBaseTest.PICK_TO_LOAD (4 ms)
[ OK ] TaskTableBaseTest.PICK_TO_LOAD_LIMIT (3 ms)
[ OK ] TaskTableBaseTest.PICK_TO_LOAD_CACHE (4 ms)
[ OK ] TaskTableBaseTest.PICK_TO_EXECUTE (4 ms)
[ OK ] TaskTableBaseTest.PICK_TO_EXECUTE_LIMIT (4 ms)
[ OK ] TaskTableBaseTest.PICK_TO_EXECUTE_CACHE (4 ms)
[ OK ] TaskTableAdvanceTest.LOAD (3 ms)
[ OK ] TaskTableAdvanceTest.LOADED (4 ms)
[ OK ] TaskTableAdvanceTest.EXECUTE (4 ms)
[ OK ] TaskTableAdvanceTest.EXECUTED (4 ms)
[ OK ] TaskTableAdvanceTest.MOVE (3 ms)
[ OK ] TaskTableAdvanceTest.MOVED (4 ms)
[ OK ] CacheTest.DUMMY_TEST (0 ms)
[ OK ] CacheTest.CPU_CACHE_TEST (4 ms)
[ OK ] CacheTest.GPU_CACHE_TEST (136 ms)
[ OK ] CacheTest.INVALID_TEST (2 ms)
[ OK ] ConfigTest.CONFIG_TEST (1 ms)
[ OK ] ConfigTest.SERVER_CONFIG_VALID_TEST (4 ms)
[ OK ] ConfigTest.SERVER_CONFIG_INVALID_TEST (2 ms)
[ OK ] ConfigTest.SERVER_CONFIG_TEST (4 ms)
[ OK ] RpcHandlerTest.HAS_TABLE_TEST (1061 ms)
[ OK ] RpcHandlerTest.INDEX_TEST (1060 ms)
[ OK ] RpcHandlerTest.INSERT_TEST (1063 ms)
[ OK ] RpcHandlerTest.SEARCH_TEST (1063 ms)
[ OK ] RpcHandlerTest.TABLES_TEST (1068 ms)
[ OK ] RpcHandlerTest.CMD_TEST (1061 ms)
[ OK ] RpcHandlerTest.DELETE_BY_RANGE_TEST (1060 ms)
[ OK ] RpcSchedulerTest.BASE_TASK_TEST (1 ms)
[ OK ] UtilTest.EXCEPTION_TEST (0 ms)
[ OK ] UtilTest.SIGNAL_TEST (0 ms)
[ OK ] UtilTest.COMMON_TEST (1 ms)
[ OK ] UtilTest.STRINGFUNCTIONS_TEST (0 ms)
[ OK ] UtilTest.BLOCKINGQUEUE_TEST (0 ms)
[ OK ] UtilTest.LOG_TEST (1 ms)
[ OK ] UtilTest.TIMERECORDER_TEST (0 ms)
[ OK ] UtilTest.STATUS_TEST (0 ms)
[ OK ] UtilTest.ROLLOUTHANDLER_TEST (1 ms)
[ OK ] ValidationUtilTest.VALIDATE_TABLENAME_TEST (1 ms)
[ OK ] ValidationUtilTest.VALIDATE_DIMENSION_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_INDEX_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_TOPK_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_NPROBE_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_GPU_TEST (2 ms)
[ OK ] ValidationUtilTest.VALIDATE_IPADDRESS_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_NUMBER_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_BOOL_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_DOUBLE_TEST (0 ms)
[ OK ] ValidationUtilTest.VALIDATE_DBURI_TEST (0 ms)
[ OK ] KnowhereTest.KNOWHERE_RESOURCE_TEST (141 ms)
[ OK ] whatever.test_config (0 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/0 (13825 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/1 (1969 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/2 (1141 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/3 (16538 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/4 (1449 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/5 (1404 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.BASE_TEST/6 (238 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/0 (14907 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/1 (1859 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/2 (1479 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/3 (18195 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/4 (1554 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/5 (1503 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.TO_GPU_TEST/6 (888 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/0 (13620 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/1 (2005 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/2 (1349 ms)
====[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/3 (17714 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/4 (1513 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/5 (1491 ms)
[ OK ] WrapperParam/KnowhereWrapperTest.SERIALIZE_TEST/6 (442 ms)

'==' more than 5 seconds
'====' more than 10 seconds
'======' more than 30 seconds

@yhmo yhmo added the kind/enhancement Issues or changes related to enhancement label Nov 9, 2019
@yhmo yhmo added this to To do in v0.6.0 via automation Nov 9, 2019
@cydrain
Copy link
Contributor

cydrain commented Dec 31, 2019

done

@cydrain cydrain closed this as completed Dec 31, 2019
v0.6.0 automation moved this from To do to Done Dec 31, 2019
yah01 pushed a commit to yah01/milvus that referenced this issue Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement Issues or changes related to enhancement
Projects
No open projects
v0.6.0
  
Done
Development

No branches or pull requests

5 participants