<dependency>
<groupId>com.github.ddphin</groupId>
<artifactId>ddphin-spring-boot-starter</artifactId>
<version>1.2.3</version>
</dependency>
ddphin collect the data base on mybatis, by default, you can use the DefaultCollect if you config the elasticsearch.sync.output, otherwise, you can implement Collector to collect the data as you want.
public interface Collector {
void collect(Object var1, String var2, Executor var3, MappedStatement var4) throws SQLException;
}
ddphin will collect data and convert the whole data to bulk request body by default, your can implement RequestBodyBuilder to build it from the collect data ContextHolder (which is thread accesssable) as you want.
public interface RequestBodyBuilder {
String build();
void setOutputMap(Map<String, ESSyncItemOutputItem> var1);
}
ddphin will collect data and convert the whole data to bulk request body, you can implement BulkRequestBodyTransmitor to handler the request body, such as transmit it to message queue.
by default, ddphin transmit the request body to elasticsearch with bulk processor directly if you have config the elasticsearch.repo.
BulkRequestBodyTransmitor:
public interface RequestBodyTransmitor {
void transmit(String var1) throws IOException;
}
if you use the default BulkRequestBodyBuilder and BulkRequestBodyTransmitor, and you have config the elasticsearch.repo, you can implement the interface ESVersionService to handle the elasticsearch sync version and log.
version: just for success case
public class ESVersionBean {
private String index;// elasearch index
private String id;// document id
private Long seqNo;// elasearch seqNo
private Long primaryTerm;// elasearch primaryTerm
}
version log: for every case
public class ESVersionLogBean {
private String index;// elasearch index
private String id;// document id
private Long seqNo;// elasearch seqNo
private Long primaryTerm;// elasearch primaryTerm
private Long lid;// execute id
private String context;//request data
private String message;// failed message
private Integer status;// 0:success; 1:failed; -1:not executed
esVersion Service:
public interface ESVersionService {
Integer replaceList(List<ESVersionBean> var1);// save/update version
Integer insertLogList(List<ESVersionLogBean> var1);// save log
}
add the configuration to your application.yml
elasticsearch.repo: is the elasticsearch server configuration, just ignore it if you do not transmit the data to elasticsearch directly.elasticsearch.sync: is the elasticsearch sync configuration,ddphinneed it to collect the data and then convert to elasticsearch request body,ddphinhas a defaulttransmitor, with which, the request body will transmit to the elasticsearch server directly (if you have config theelasticsearch.repo).api: a list, tellddphinwhich apis will be listened to collect the data.point:INSERT/UPDATE/DELETEis the supported mybatis command,AFTERtellddphincollect dataAFTERthe mybatis command, BEFORE tellddphincollect data before teh mybatis command.input: is a K-V map, the key tellsddphinwhich mybatis will be listened, and the value is the entity corresponds to the mapper.(usually, the mapper is the handler of a single table, and the entity is the data of the table).output: is a K-V map, is the relation definition for the whole entity, finally, all the data will be convert to the elasticsearch index mapping base on theoutputconfiguration.key: the entity key, will use to be cache(every mapper has a cache), usually, it is the id.map: map entity column to elasticsearch simple property, for elasearch suggestion field( which has input and o optional weight field), the value can be like this:field.inputandfield.weight.has: is a K-V map, value indicate elasticsearch array property or nested property.with-type:primitiveadd the field as the simple property directly.nestedis the elasticsearch nested property array,arrayis the array of simple property array.as: just fornestedandarray, is the array nameunique-by: just fornested, is the unique property in the array.association: is the association relation to parentoutput.belongs.to: the parentoutputbelongs.with: the association key will used to associate the parentoutputbelongs.as: the name in parentoutputon-missing.query: the method used to query the parentoutputwhenbelongs.withis missing. it can be the method of the mapper, or a sql command.on-missing.with: the param of the currentoutputused to query the parentoutput.
reload: will be ignored ifon-missingis triggered.query: the method used to query the currentoutputand then reload the datawith: the param of the currentoutputused to reload data.
elasticsearch:
repo:
host: 127.0.0.1
port: 9200
scheme: http
sync:
api:
- '/**'
point:
INSERT: AFTER
UPDATE: AFTER
DELETE: BEFORE
input:
com.ddphin.ddphin.business.commodity.mapper.MSpuMapper: spu
com.ddphin.ddphin.business.commodity.mapper.MSpuStockMapper: spu_stock
com.ddphin.ddphin.business.commodity.mapper.MSpuMarkMapper: spu_mark
com.ddphin.ddphin.business.commodity.mapper.MSpuLabelMapper: spu_label
com.ddphin.ddphin.business.commodity.mapper.MSpuPlatformMapper: spu_platform
com.ddphin.ddphin.business.commodity.mapper.MSpuServiceMapper: spu_service
com.ddphin.ddphin.business.commodity.mapper.MSkuMapper: sku
com.ddphin.ddphin.business.commodity.mapper.MSkuParaMapper: sku_para
com.ddphin.ddphin.business.commodity.mapper.MSkuStockMapper: sku_stock
output:
spu:
key: id
map:
id: id
cid: cid
bid: bid
status: status
summary: summary
pic: pic
shows: shows
marketDate: marketDate
has:
sku:
with-type: nested
as: sku
unique-by: id
spu_label:
with-type: array
as: label
spu_platform:
with-type: array
as: platform
spu_service:
with-type: array
as: service
spu_mark:
with-type: primitive
spu_stock:
with-type: primitive
spu_mark:
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
key: spuid
map:
mark: mark
spu_stock:
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
reload:
query: query
with: spuid
key: spuid
map:
sales: sales
praise: praise
comment: comment
poor: poor
spu_label:
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
key: id
map:
lid: lid
spu_platform:
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
key: id
map:
type: type
spu_service:
key: id
map:
sid: sid
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
sku:
key: id
map:
id: id
title: title
mprice: mprice
pprice: pprice
vprice: vprice
has:
sku_stock:
with-type: primitive
sku_para:
with-type: nested
as: para
unique-by: pid
association:
belongs:
to: spu
with: spuid
as: id
on-missing:
query: query
with: id
sku_stock:
key: skuid
map:
inventory: inventory
association:
belongs:
to: sku
with: skuid
as: id
on-missing:
query: query
with: id
reload:
query: query
with: skuid
sku_para:
key: id
map:
pid: pid
vid: vid
association:
belongs:
to: sku
with: sid
as: id
on-missing:
query: query
with: id
{
"spu": {
"mappings": {
"properties": {
"bid": {
"type": "keyword"
},
"cid": {
"type": "keyword"
},
"comment": {
"type": "long"
},
"evals": {
"type": "integer",
"index": false,
"null_value": 0
},
"id": {
"type": "keyword"
},
"label": {
"type": "keyword"
},
"mark": {
"type": "float",
"index": false,
"null_value": 1
},
"marketDate": {
"type": "date"
},
"pic": {
"type": "text",
"index": false
},
"platform": {
"type": "keyword"
},
"poor": {
"type": "long"
},
"praise": {
"type": "integer",
"index": false,
"null_value": 0
},
"price": {
"type": "integer"
},
"sales": {
"type": "integer",
"index": false,
"null_value": 0
},
"service": {
"type": "keyword"
},
"shows": {
"type": "text",
"index": false
},
"sku": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
},
"inventory": {
"type": "integer",
"null_value": 0
},
"mprice": {
"type": "integer",
"index": false
},
"para": {
"type": "nested",
"properties": {
"pid": {
"type": "keyword"
},
"vid": {
"type": "keyword"
}
}
},
"pprice": {
"type": "integer",
"index": false
},
"title": {
"type": "text",
"index": false
}
}
},
"status": {
"type": "keyword"
},
"summary": {
"type": "text",
"analyzer": "ikpyIndexAnalyzer",
"search_analyzer": "ikpySearchAnalyzer"
}
}
}
}
}