Fix delete dead lock.

[ColinLeeo]

The current system has a deadlock problem when the deletion is concurrent with query and insertion:

1. Query worker 's num is limit as core num.
2. Deletion will acquire Dataregion's write lock and Datanode Schemacache's write lock.
3. Insert or query will request additional queries to obtain metadata in the case of acquiring a read lock.

DeadLock cycle like this:

graph BT
A[#QueryWorker# get one core to run query]
B[#QueryWorker# hold read lock on dataregion]
A-->B
C[#DeleteData# hold write lock on dataregion]
D[#DeleteData# hold write lock on datanode schema]
C-->D
E[#InsertData# holdread lock on datanode schema]
F[#InsertData# get one core to run fetch schema]
E-->F
B-->C
F-->A
D-->E

Loading

In this pr:

1. Optimize the lock order to prevent deadlocks
2. Add invalidate last cache operation. The deletion operation does not need to clean up all caches, but should clean the corresponding last cache.
3. You can use the following patch to reproduce the deadlock problem.
   0001-to-cause-dead-lock-when-delete.patch
   In this patch, datanodeCache will always be empty, so query or insert will fetch remote schema. Besides the query worker's num is limit to 1 and the program will wait about 20s/30s after acquiring the critical lock.
   To reproduce the problem, run these three queries in three sessions within 30s:
   session1: insert into root.sg1.t1(t) values(1);
   session2: delete from root.sg1.t1.**;
   session3: select * from root.sg1.**;

[JackieTien97]

why don't usesynchronized (entry) here?

[JackieTien97]

it should return the diff memory size instead of 0.

[JackieTien97]

add some comments here to explain why this field doesn't need to be serialized (because it will always be false except for that in delete timesereis, and in which case we will set this to true in DataNodeInternalRPCServiceImpl.deleteDataForDeleteSchema.

Suggested change

	private final boolean isDeleteTimeseries;
	private final transient boolean isDeleteTimeseries;

[JackieTien97]

just call DataNodeSchemaCache.getInstance().unsetLastCacheInDataRegion(getDatabaseName())?

[JackieTien97]

Suggested change

	deviceFilter = d -> ((PartialPath) d).equals(devicePath.getFullPath());
	deviceFilter = d -> ((PartialPath) d).equals(devicePath);