[SUPPORT] write to a mor table 12times, but no compaction instance with default configuration

[xuzifu666]

Tips before filing an issue

• Have you gone through our FAQs?

• Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

• If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced

A clear and concise description of the problem.

To Reproduce

Steps to reproduce the behavior:

1. create a mor table with sparksql
2. write a batch data to the mor table 12times
3. query .hoodie dir
4. only deltacommit instance, and no compaction instance

like as follow:
Found 42 items
0 2023-04-18 11:39 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/.aux
0 2023-04-18 11:39 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/.schema
0 2023-04-20 19:13 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/.temp
248563 2023-04-18 12:54 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418114018435.deltacommit
829 2023-04-18 11:54 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418114018435.deltacommit.inflight
0 2023-04-18 11:40 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418114018435.deltacommit.requested
328631 2023-04-18 14:24 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418141519273.deltacommit
187188 2023-04-18 14:19 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418141519273.deltacommit.inflight
0 2023-04-18 14:15 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230418141519273.deltacommit.requested
330770 2023-04-20 10:58 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420103907224.deltacommit
187188 2023-04-20 10:51 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420103907224.deltacommit.inflight
0 2023-04-20 10:39 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420103907224.deltacommit.requested
674579 2023-04-20 11:21 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420110730118.deltacommit
187188 2023-04-20 11:14 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420110730118.deltacommit.inflight
0 2023-04-20 11:07 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420110730118.deltacommit.requested
349763 2023-04-20 11:44 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420113116519.deltacommit
187188 2023-04-20 11:39 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420113116519.deltacommit.inflight
0 2023-04-20 11:31 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420113116519.deltacommit.requested
429551 2023-04-20 11:56 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420114628833.deltacommit
187188 2023-04-20 11:52 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420114628833.deltacommit.inflight
0 2023-04-20 11:46 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420114628833.deltacommit.requested
638438 2023-04-20 12:56 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420124536149.deltacommit
187188 2023-04-20 12:51 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420124536149.deltacommit.inflight
0 2023-04-20 12:45 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420124536149.deltacommit.requested
369101 2023-04-20 14:16 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420141200756.deltacommit
186932 2023-04-20 14:14 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420141200756.deltacommit.inflight
0 2023-04-20 14:12 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420141200756.deltacommit.requested
369148 2023-04-20 17:07 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420170143657.deltacommit
186932 2023-04-20 17:05 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420170143657.deltacommit.inflight
0 2023-04-20 17:01 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420170143657.deltacommit.requested
453372 2023-04-20 17:28 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420172000974.deltacommit
186932 2023-04-20 17:24 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420172000974.deltacommit.inflight
0 2023-04-20 17:20 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420172000974.deltacommit.requested
674192 2023-04-20 18:48 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420183455512.deltacommit
187188 2023-04-20 18:42 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420183455512.deltacommit.inflight
0 2023-04-20 18:34 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420183455512.deltacommit.requested
477985 2023-04-20 19:13 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420190317010.deltacommit
187188 2023-04-20 19:08 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420190317010.deltacommit.inflight
0 2023-04-20 19:03 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/20230420190317010.deltacommit.requested
0 2023-04-18 11:39 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/archived
3131 2023-04-18 11:40 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/hoodie.properties
0 2023-04-18 11:40 /region/85/warehouse/bdsp_test/deduplicate_hudi_mor_bucket_index_test10/.hoodie/metadata
Expected behavior

A clear and concise description of what you expected to happen.

Environment Description

• Hudi version : 0.14

• Spark version : 3.2.0

• Hive version :

• Hadoop version : 3.1

• Storage (HDFS/S3/GCS..) :hdfs

• Running on Docker? (yes/no) :no

Additional context

Add any other context about the problem here.

Stacktrace

Add the stacktrace of the error.

[danny0405]

The spark batch sql does not include any compaction service at all when writing, what is exact the behavior you want for the use case?

[xuzifu666]

at is exact the behavior you want for th

ok，no problem，we would set inline compaction from false to true，thanks

[xuzifu666]

at is exact the behavior you want for th

ok，no problem，we would set inline compaction from false to true，thanks @danny0405

[nsivabalan]

with 0.14.0, we have added automatic compaction with spark batch writes if user is not explicitly overriding it. just incase it helps.