Skip to content

Commit

Permalink
[ISSUE apache#2472] Add producer best practice for english
Browse files Browse the repository at this point in the history
  • Loading branch information
coder-zzzz authored and tianliuliu committed Dec 21, 2021
1 parent ddd09ff commit a2c8503
Showing 1 changed file with 66 additions and 0 deletions.
66 changes: 66 additions & 0 deletions docs/en/best_practice.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,72 @@
# Best practices

## 1 Producer
### 1.1 Attention of send message

#### 1 Uses of tags
An application should use one topic as far as possible, but identify the message's subtype with tags.Tags can be set freely by the application.
Only when producers set tags while sending messages, can consumers to filter messages through broker with tags when subscribing messages: message.setTags("TagA").

#### 2 Uses of keys
The unique identifier for each message at the business level set to the Keys field to help locate message loss problems in the future.
The server creates an index(hash index) for each message, and the application can query the message content via Topic,key,and who consumed the message.
Since it is a hash index, make sure that the key is as unique as possible to avoid potential hash conflicts.

```java
// order id
String orderId = "20034568923546";
message.setKeys(orderId);
```
#### 3 Log print
Print the message log when send success or failed, make sure to print the SendResult and key fields.
Send messages is successful as long as it does not throw exception. Send successful will have multiple states defined in sendResult.
Each state is describing below:

- **SEND_OK**

Message send successfully.Note that even though message send successfully, but it doesn't mean than it is reliable.
To make sure nothing lost, you should also enable the SYNC_MASTER or SYNC_FLUSH.

- **FLUSH_DISK_TIMEOUT**

Message send successfully, but the server flush messages to disk timeout.At this point, the message has entered the server's memory, and the message will be lost only when the server is down.
Flush mode and sync flush time interval can be set in the configuration parameters. It will return FLUSH_DISK_TIMEOUT when Broker server doesn't finish flush message to disk in timout(default is 5s
) when sets FlushDiskType=SYNC_FLUSH(default is async flush).

- **FLUSH_SLAVE_TIMEOUT**

Message send successfully, but sync to slave timeout.At this point, the message has entered the server's memory, and the message will be lost only when the server is down.
It will return FLUSH_SLAVE_TIMEOUT when Broker server role is SYNC_MASTER(default is ASYNC_MASTER),and it doesn't sync message to slave successfully in the timeout(default 5s).

- **SLAVE_NOT_AVAILABLE**

Message send successfully, but slave is not available.It will return SLAVE_NOT_AVAILABLE when Broker role is SYNC_MASTER(default is ASYNC_MASTER), and it doesn't have a slave server available.

### 1.2 Handling of message send failure
Send method of producer itself supports internal retry. The logic of retry is as follows:
- At most twice.
- Try next broker when sync send mode, try current broker when async mode. The total elapsed time of this method does not exceed the value of sendMsgTimeout(default is 10s).
- It will not be retried when the message is sent to the Broker with a timeout exception.

The strategy above ensures the success of message sending to some extent.If the business has a high requirement for message reliability, it is recommended to add the corresponding retry logic:
for example, if the sync send method fails, try to store the message in DB, and then retry periodically by the bg thread to ensure the message must send to broker successfully.

Why the above DB retry strategy is not integrated into the MQ client, but requires the application to complete it by itself is mainly based on the following considerations:
First, the MQ client is designed to be stateless mode, convenient for arbitrary horizontal expansion, and only consumes CPU, memory and network resources.
Second, if the MQ client internal integration a KV storage module, the data can only be relatively reliable when sync flush to disk, but the sync flush will cause performance lose, so it's usually
use async flush.Because the application shutdown is not controlled by the MQ operators, A violent shutdown like kill -9 may often occur, resulting in data not flushed to disk and being lost.
Thirdly, the producer is a virtual machine with low reliability, which is not suitable for storing important data.
In conclusion, it is recommended that the retry process must be controlled by the application.

### 1.3 Send message by oneway
Typically, this is the process by which messages are sent:

- Client send request to server
- Server process request
- Server response to client
So, the time taken to send a message is the sum of the three steps above.Some scenarios require very little time, but not much reliability, such as log collect application.
This type application can use oneway to send messages. Oneway only send request without waiting for a reply, and send a request at the client implementation level is simply the overhead of an
operating system call that writes data to the client's socket buffer, this process that typically takes microseconds.

## 2 Consumer

Expand Down

0 comments on commit a2c8503

Please sign in to comment.