New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIP004 - Support for AEADs implemented by large libraries #30

Closed
Mygod opened this Issue Jan 12, 2017 · 280 comments

Comments

Projects
None yet
@Mygod
Contributor

Mygod commented Jan 12, 2017

Use AEADs to replace stream cipher + OTA. Previous discussion: #29.

Proposed AEAD algorithms:

  • ChaCha20-Poly1305 (see also: xSocks)
  • XChaCha20-Poly1305
  • Salsa20-Poly1305
  • AES-256-GCM (faster but not low-end-box-friendly)

Update: The following shows an example of TCP stream in chacha20-ietf-poly1305 mode (original idea by @breakwa11 and @Noisyfox). Other AEAD should follow the similar format.

Cipher: chacha20-ietf-poly1305

TCP request (after encryption, *ciphertext*)
+--------+----------------+--------------+--------------+---------------+
| NONCE  | PayloadLen_TAG | *PayloadLen* | Payload_TAG  |   *Payload*   |
+--------+----------------+--------------+--------------+---------------+
|  12    |       16       |       2      |     16       |    Variable   |
+--------+----------------+--------------+--------------+---------------+

TCP Chunk (after encryption, *ciphertext*)
+--------------+------------+-----------+----------+
| DATA_LEN_TAG | *DATA_LEN* |  DATA_TAG |  *DATA*  |
+--------------+------------+-----------+----------+
|      16      |     2      |     16    | Variable |
+--------------+------------+-----------+----------+

@Mygod Mygod added the enhancement label Jan 12, 2017

@Mygod Mygod referenced this issue Jan 12, 2017

Closed

Refine doc for OTA #29

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

My plan is forking ourselves, dropping all the support of stream ciphers and OTA.

After that the we can provide a new version of the protocol (v2?).

Contributor

madeye commented Jan 12, 2017

My plan is forking ourselves, dropping all the support of stream ciphers and OTA.

After that the we can provide a new version of the protocol (v2?).

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 12, 2017

I want to clarify AES-GCM is the fastest on machines with particular instruction sets, for example, server and client are both running on modern Intel CPU.

wongsyrone commented Jan 12, 2017

I want to clarify AES-GCM is the fastest on machines with particular instruction sets, for example, server and client are both running on modern Intel CPU.

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone commented Jan 12, 2017

I agree with #30 (comment)

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 12, 2017

Contributor

I don't see the reason to drop compatibility for stream ciphers for now as @madeye has said:

However, I don't think any known attacks to shadowsocks protocol is realistic. And please don't forget that most of the traffic over shadowsocks is TLS...

People who has higher security standards could migrate to using these algorithms. And if benchmark shows its influence to performance is negligible we should start to encourage everyone to migrate to AEADs.

And I don't think adding these additional algorithms would be a lot of work (we need at most 1 more abstraction layer).

Contributor

Mygod commented Jan 12, 2017

I don't see the reason to drop compatibility for stream ciphers for now as @madeye has said:

However, I don't think any known attacks to shadowsocks protocol is realistic. And please don't forget that most of the traffic over shadowsocks is TLS...

People who has higher security standards could migrate to using these algorithms. And if benchmark shows its influence to performance is negligible we should start to encourage everyone to migrate to AEADs.

And I don't think adding these additional algorithms would be a lot of work (we need at most 1 more abstraction layer).

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 12, 2017

They can coexist as different projects, it is easy to maintain IMO.

wongsyrone commented Jan 12, 2017

They can coexist as different projects, it is easy to maintain IMO.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 12, 2017

Contributor

Considering SIP003, I think it's easier to maintain if implemented in one single project.

Contributor

Mygod commented Jan 12, 2017

Considering SIP003, I think it's easier to maintain if implemented in one single project.

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

To support AEADs, first we need to define chunks for the TCP stream. Here is a proposal:

+----------+----------+----------+----------+
| DATA.LEN |   NONCE  |    TAG   |   DATA   | ...
+----------+----------+----------+----------+
|     2    |    8     |    16    | Variable | ...
+----------+----------+----------+----------| 

Assuming average data length equals to a typical MTU - 26 (1492 - 26 = 1466), then the maximum transfer efficiency is 1466 / 1492 = 98%.

Contributor

madeye commented Jan 12, 2017

To support AEADs, first we need to define chunks for the TCP stream. Here is a proposal:

+----------+----------+----------+----------+
| DATA.LEN |   NONCE  |    TAG   |   DATA   | ...
+----------+----------+----------+----------+
|     2    |    8     |    16    | Variable | ...
+----------+----------+----------+----------| 

Assuming average data length equals to a typical MTU - 26 (1492 - 26 = 1466), then the maximum transfer efficiency is 1466 / 1492 = 98%.

@Noisyfox

This comment has been minimized.

Show comment
Hide comment
@Noisyfox

Noisyfox Jan 12, 2017

So which part of the chunk is encrypted? DATA? Or the whole chunk? Or both of them?

Noisyfox commented Jan 12, 2017

So which part of the chunk is encrypted? DATA? Or the whole chunk? Or both of them?

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

@Noisyfox DATA.LEN, NONCE and TAG should not be encrypted. Only DATA is encrypted. TAG is generated from encrypted DATA. For more details: https://github.com/lparam/xSocks/blob/master/src/crypto.c#L25

Contributor

madeye commented Jan 12, 2017

@Noisyfox DATA.LEN, NONCE and TAG should not be encrypted. Only DATA is encrypted. TAG is generated from encrypted DATA. For more details: https://github.com/lparam/xSocks/blob/master/src/crypto.c#L25

@Noisyfox

This comment has been minimized.

Show comment
Hide comment
@Noisyfox

Noisyfox Jan 12, 2017

That's basically the pattern of TLS record if I did understand correctly.

Noisyfox commented Jan 12, 2017

That's basically the pattern of TLS record if I did understand correctly.

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

@Noisyfox Yes, exactly.

Contributor

madeye commented Jan 12, 2017

@Noisyfox Yes, exactly.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 12, 2017

Contributor

Is DATA.LEN necessary?

@wongsyrone I don't think we need AEAD actually?

Contributor

Mygod commented Jan 12, 2017

Is DATA.LEN necessary?

@wongsyrone I don't think we need AEAD actually?

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

@Mygod Yes, I think so. It's required to split TCP stream to chunks.

Contributor

madeye commented Jan 12, 2017

@Mygod Yes, I think so. It's required to split TCP stream to chunks.

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 12, 2017

AE:

  • Encrypts a message with a key and a nonce to keep it confidential
  • Computes an authentication tag. This tag is used to make sure that the message hasn't been tampered with before decrypting it.

AEAD:

  • Encrypts a message with a key and a nonce to keep it confidential
  • Computes an authentication tag. This tag is used to make sure that the message, as well as optional, non-confidential (non-encrypted) data, haven't been tampered with.

AEAD can protect plain text as well.

wongsyrone commented Jan 12, 2017

AE:

  • Encrypts a message with a key and a nonce to keep it confidential
  • Computes an authentication tag. This tag is used to make sure that the message hasn't been tampered with before decrypting it.

AEAD:

  • Encrypts a message with a key and a nonce to keep it confidential
  • Computes an authentication tag. This tag is used to make sure that the message, as well as optional, non-confidential (non-encrypted) data, haven't been tampered with.

AEAD can protect plain text as well.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 12, 2017

Contributor

@madeye What about letting it equal to packet length - 24?

@wongsyrone I don't think we have non-confidential data to protect however.

Contributor

Mygod commented Jan 12, 2017

@madeye What about letting it equal to packet length - 24?

@wongsyrone I don't think we have non-confidential data to protect however.

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

@Mygod In a TCP stream, there is not packet length...

Contributor

madeye commented Jan 12, 2017

@Mygod In a TCP stream, there is not packet length...

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 12, 2017

Contributor

Oh sorry I got confused because you mentioned MTU. 😅

Contributor

Mygod commented Jan 12, 2017

Oh sorry I got confused because you mentioned MTU. 😅

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

MTU here is only to demonstrate how large a chunk could be in the real world... In our implementation, we should also limit the DATA.LEN, make it smaller than the socket buffer size, e.g. 2048 in shadowsocks-libev.

Contributor

madeye commented Jan 12, 2017

MTU here is only to demonstrate how large a chunk could be in the real world... In our implementation, we should also limit the DATA.LEN, make it smaller than the socket buffer size, e.g. 2048 in shadowsocks-libev.

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 12, 2017

NONCE is too short.

Note: the currently included AEAD cipher based on the draft only uses an eight byte nonce, which is too short to be safely generated randomly. A later version of libsodium will include the updated version with a longer nonce.

https://crypto.stackexchange.com/questions/29311/how-do-i-tack-on-additional-data-to-libsodiums-public-key-authenticated-encry

wongsyrone commented Jan 12, 2017

NONCE is too short.

Note: the currently included AEAD cipher based on the draft only uses an eight byte nonce, which is too short to be safely generated randomly. A later version of libsodium will include the updated version with a longer nonce.

https://crypto.stackexchange.com/questions/29311/how-do-i-tack-on-additional-data-to-libsodiums-public-key-authenticated-encry

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 12, 2017

Contributor

Then what about this:

+----------+----------+----------+----------+----------|
| DATA.LEN |   SEQ    |   NONCE  |    TAG   |   DATA   | ...
+----------+----------+----------+----------+----------+
|     2    |    8     |    8     |    16    | Variable | ...
+----------+----------+----------+----------+----------+

SEQ is a 64-bit sequence number. The real NONCE is SEQ+ NONCE. DATA.LEN + SEQ is also the AD of the AEAD.

(Why shouldn't I use TLS directly? 😅 )

Contributor

madeye commented Jan 12, 2017

Then what about this:

+----------+----------+----------+----------+----------|
| DATA.LEN |   SEQ    |   NONCE  |    TAG   |   DATA   | ...
+----------+----------+----------+----------+----------+
|     2    |    8     |    8     |    16    | Variable | ...
+----------+----------+----------+----------+----------+

SEQ is a 64-bit sequence number. The real NONCE is SEQ+ NONCE. DATA.LEN + SEQ is also the AD of the AEAD.

(Why shouldn't I use TLS directly? 😅 )

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 18, 2017

[draft] main idea from @breakwa11 and @Noisyfox
The purpose of Length tag is to keep length as-is and not tempered by the third party since this is the key to separate general ciphertext and auth tag.

Please note that this is not compatible with current OTA and original protocol.

/*
 * The main difference between OTA designed by madeye and this one is MtE vs EtM.
 *
 * The first nonce is either from client or server side, it is generated via randombytes_buf()
 * in libsodium, and the sequent nonces are incremented via sodium_increment() in libsodium.
 * Please note that sodium_increment() considers the number to be encoded in little-endian format.
 * IMPORTANT: nonce should be used only once, let us running on the right track.
 *
 * Data.Len is used to separate general ciphertext and Auth tag. We can start decryption
 * if and only if the verification is passed. 
 * Firstly, we do length check, then decrypt it, separate ciphertext and attached data tag
 * based on the verified length, verify data tag and decrypt the corresponding data.
 * Finally, do what you supposed to do, e.g. forward user data.
 *
 * For UDP, nonces are generated randomly without the incrementation.
 *
 * TCP request (before encryption)
 * +------+---------------------+------------------+
 * | ATYP | Destination Address | Destination Port |
 * +------+---------------------+------------------+
 * |  1   |       Variable      |         2        |
 * +------+---------------------+------------------+
 *
 * TCP request (after encryption)
 * +--------+-----------+---------------+----------+---------------+
 * | NONCE  |PayloadLen |PayloadLen_TAG | Payload  |  Payload_TAG  |
 * +--------+-----------+---------------+----------+---------------+
 * | Fixed  |     2     |     Fixed     | Variable |     Fixed     |
 * +--------+-----------+---------------+----------+---------------+
 * 
 * Payload input: atyp + dst.addr + dst.port
 * PayloadLen is length(atyp + dst.addr + dst.port)
 * Payload_TAG and PayloadLen_TAG are in plaintext
 *
 * TCP Chunk (before encryption)
 * +----------+
 * |  DATA    |
 * +----------+
 * | Variable |
 * +----------+
 *
 * Data.Len is a 16-bit big-endian integer indicating the length of DATA.
 *
 * TCP Chunk (after encryption)
 * +------------+---------+-----------+----------+
 * | Data.Len*  | Len_TAG |  DATA_TAG |  DATA*   |
 * +------------+---------+-----------+----------+
 * |      2     | Fixed   |  Fixed    | Variable |
 * +------------+---------+-----------+----------+
 *
 * Len_TAG and DATA_TAG have the same length, they are in plaintext.
 * After encryption, DATA -> DATA*
 *
 * UDP (before encryption)
 * +------+---------------------+------------------+----------+
 * | ATYP | Destination Address | Destination Port |   DATA   |
 * +------+---------------------+------------------+----------+
 * |  1   |       Variable      |         2        | Variable |
 * +------+---------------------+------------------+----------+
 *
 * UDP (after encryption)
 * +--------+----------+-----------+
 * | NONCE  |  DATA*   |  DATA_TAG |
 * +--------+----------+-----------+
 * | Fixed  | Variable |  Fixed    |
 * +--------+----------+-----------+
 *
 * DATA* is Encrypt(atyp + dst.addr + dst.port + DATA)
 * RSV and FRAG are dropped
 * Since UDP packet is either received completely or missed,
 * we don't have to keep a field to track its length.
 *
 */

wongsyrone commented Jan 18, 2017

[draft] main idea from @breakwa11 and @Noisyfox
The purpose of Length tag is to keep length as-is and not tempered by the third party since this is the key to separate general ciphertext and auth tag.

Please note that this is not compatible with current OTA and original protocol.

/*
 * The main difference between OTA designed by madeye and this one is MtE vs EtM.
 *
 * The first nonce is either from client or server side, it is generated via randombytes_buf()
 * in libsodium, and the sequent nonces are incremented via sodium_increment() in libsodium.
 * Please note that sodium_increment() considers the number to be encoded in little-endian format.
 * IMPORTANT: nonce should be used only once, let us running on the right track.
 *
 * Data.Len is used to separate general ciphertext and Auth tag. We can start decryption
 * if and only if the verification is passed. 
 * Firstly, we do length check, then decrypt it, separate ciphertext and attached data tag
 * based on the verified length, verify data tag and decrypt the corresponding data.
 * Finally, do what you supposed to do, e.g. forward user data.
 *
 * For UDP, nonces are generated randomly without the incrementation.
 *
 * TCP request (before encryption)
 * +------+---------------------+------------------+
 * | ATYP | Destination Address | Destination Port |
 * +------+---------------------+------------------+
 * |  1   |       Variable      |         2        |
 * +------+---------------------+------------------+
 *
 * TCP request (after encryption)
 * +--------+-----------+---------------+----------+---------------+
 * | NONCE  |PayloadLen |PayloadLen_TAG | Payload  |  Payload_TAG  |
 * +--------+-----------+---------------+----------+---------------+
 * | Fixed  |     2     |     Fixed     | Variable |     Fixed     |
 * +--------+-----------+---------------+----------+---------------+
 * 
 * Payload input: atyp + dst.addr + dst.port
 * PayloadLen is length(atyp + dst.addr + dst.port)
 * Payload_TAG and PayloadLen_TAG are in plaintext
 *
 * TCP Chunk (before encryption)
 * +----------+
 * |  DATA    |
 * +----------+
 * | Variable |
 * +----------+
 *
 * Data.Len is a 16-bit big-endian integer indicating the length of DATA.
 *
 * TCP Chunk (after encryption)
 * +------------+---------+-----------+----------+
 * | Data.Len*  | Len_TAG |  DATA_TAG |  DATA*   |
 * +------------+---------+-----------+----------+
 * |      2     | Fixed   |  Fixed    | Variable |
 * +------------+---------+-----------+----------+
 *
 * Len_TAG and DATA_TAG have the same length, they are in plaintext.
 * After encryption, DATA -> DATA*
 *
 * UDP (before encryption)
 * +------+---------------------+------------------+----------+
 * | ATYP | Destination Address | Destination Port |   DATA   |
 * +------+---------------------+------------------+----------+
 * |  1   |       Variable      |         2        | Variable |
 * +------+---------------------+------------------+----------+
 *
 * UDP (after encryption)
 * +--------+----------+-----------+
 * | NONCE  |  DATA*   |  DATA_TAG |
 * +--------+----------+-----------+
 * | Fixed  | Variable |  Fixed    |
 * +--------+----------+-----------+
 *
 * DATA* is Encrypt(atyp + dst.addr + dst.port + DATA)
 * RSV and FRAG are dropped
 * Since UDP packet is either received completely or missed,
 * we don't have to keep a field to track its length.
 *
 */
@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jan 18, 2017

I don't see the point of "length-tag". AEAD support additional data to be authenticated as plaintext. You can use length as the "additional data" for temper proofing.

ghost commented Jan 18, 2017

I don't see the point of "length-tag". AEAD support additional data to be authenticated as plaintext. You can use length as the "additional data" for temper proofing.

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 18, 2017

Contributor

@v2ray Agreed. The DATA.LEN can be protected by AEAD as plaintext.

Contributor

madeye commented Jan 18, 2017

@v2ray Agreed. The DATA.LEN can be protected by AEAD as plaintext.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

+1. But this reminds me that we need random noise indistinguishability more than authentication (i.e. IND-CCA2) since the original purpose of this repo is to circumvent GFW and it's much harder to perform a realistic CCA2 distinction. Let's review our additions against random noise indistinguishability:

  1. TAG: Check. This should be guaranteed by the security of the hash algorithm we used.
  2. DATA.LEN: Oh shit this is not random at all. Adversary can distinguish them very easily.
  3. NONCE: No problem as long as they are randomly generated. (which isn't the case in TCP)
  4. SEQ: Terrible idea. It makes distinction much more easier.

OTA also has this problem. So what do you think?

Contributor

Mygod commented Jan 18, 2017

+1. But this reminds me that we need random noise indistinguishability more than authentication (i.e. IND-CCA2) since the original purpose of this repo is to circumvent GFW and it's much harder to perform a realistic CCA2 distinction. Let's review our additions against random noise indistinguishability:

  1. TAG: Check. This should be guaranteed by the security of the hash algorithm we used.
  2. DATA.LEN: Oh shit this is not random at all. Adversary can distinguish them very easily.
  3. NONCE: No problem as long as they are randomly generated. (which isn't the case in TCP)
  4. SEQ: Terrible idea. It makes distinction much more easier.

OTA also has this problem. So what do you think?

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

removes some of my useless comments

Okay I think I got this sorted out. What we need to do is just to put data.len in the encryption data; send nonce ONLY in the first data and it auto increments for EVERY data. And voilà!

Contributor

Mygod commented Jan 18, 2017

removes some of my useless comments

Okay I think I got this sorted out. What we need to do is just to put data.len in the encryption data; send nonce ONLY in the first data and it auto increments for EVERY data. And voilà!

@v3aqb

This comment has been minimized.

Show comment
Hide comment
@v3aqb

v3aqb Jan 18, 2017

v3aqb commented Jan 18, 2017

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

@v3aqb

better encrypted

I beg to differ. I took a quick glance and you implemented what shadowsocks was designed to prevent.

Contributor

Mygod commented Jan 18, 2017

@v3aqb

better encrypted

I beg to differ. I took a quick glance and you implemented what shadowsocks was designed to prevent.

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Jan 18, 2017

Contributor

@Mygod Then the only difference from our current OTA is that the new proposal is Encrypt-Then-MAC... 😄

Contributor

madeye commented Jan 18, 2017

@Mygod Then the only difference from our current OTA is that the new proposal is Encrypt-Then-MAC... 😄

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 18, 2017

This is just an example of improving content security (aka CCA-proof), while it undermines protocol security (aka censorship-bypassing-detection-proof).

Before any progress, we should examine it carefully and make the right choice.

wongsyrone commented Jan 18, 2017

This is just an example of improving content security (aka CCA-proof), while it undermines protocol security (aka censorship-bypassing-detection-proof).

Before any progress, we should examine it carefully and make the right choice.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

@madeye No. There are two significant differences:

  1. Data.len is transmitted in ciphertext instead of plain text. This implies that it's easy for GFW to detect shadowsocks connections with OTA enabled.
  2. AE algorithms are chosen and implemented by large library and tested/attempted attack by more people.

@wongsyrone Yes. We need to know which one is more important in this context. I think my approach should be safest afaic.

Contributor

Mygod commented Jan 18, 2017

@madeye No. There are two significant differences:

  1. Data.len is transmitted in ciphertext instead of plain text. This implies that it's easy for GFW to detect shadowsocks connections with OTA enabled.
  2. AE algorithms are chosen and implemented by large library and tested/attempted attack by more people.

@wongsyrone Yes. We need to know which one is more important in this context. I think my approach should be safest afaic.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jan 18, 2017

@Mygod if data.len is encrypted, how to figure out the length of chunk for decryption (in the context of AEAD)?

ghost commented Jan 18, 2017

@Mygod if data.len is encrypted, how to figure out the length of chunk for decryption (in the context of AEAD)?

@wongsyrone

This comment has been minimized.

Show comment
Hide comment
@wongsyrone

wongsyrone Jan 18, 2017

The noise data is another approach to obfuscating packet length pattern, which may help.

Without data.len authentication, you cannot assume it hasn't been touched. That's why we decide to add a tag for it.

Encryption never implies authentication.

In fact, the AD part is never used in the proposal; I planned to make it work like another pre-shared info and verify it before decryption. It looks like a kind of overkill.

wongsyrone commented Jan 18, 2017

The noise data is another approach to obfuscating packet length pattern, which may help.

Without data.len authentication, you cannot assume it hasn't been touched. That's why we decide to add a tag for it.

Encryption never implies authentication.

In fact, the AD part is never used in the proposal; I planned to make it work like another pre-shared info and verify it before decryption. It looks like a kind of overkill.

@Noisyfox

This comment has been minimized.

Show comment
Hide comment
@Noisyfox

Noisyfox Jan 18, 2017

@v2ray You get the point man.
That's why we encrypt length as a 'chunk' with it's own tag, which means we could first verify and decrypt the length which has a fixed size and then read the remain data according to the verified length.

Noisyfox commented Jan 18, 2017

@v2ray You get the point man.
That's why we encrypt length as a 'chunk' with it's own tag, which means we could first verify and decrypt the length which has a fixed size and then read the remain data according to the verified length.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

@v2ray Just decrypt the first 2 bytes for stream cipher or first block for block cipher. I don't see a problem.

@wongsyrone All encryption I mentioned are authenticated encryption. data.len will be included when calculating hash as well.

Contributor

Mygod commented Jan 18, 2017

@v2ray Just decrypt the first 2 bytes for stream cipher or first block for block cipher. I don't see a problem.

@wongsyrone All encryption I mentioned are authenticated encryption. data.len will be included when calculating hash as well.

@Noisyfox

This comment has been minimized.

Show comment
Hide comment
@Noisyfox

Noisyfox Jan 18, 2017

@Mygod If some one increase the length by one byte, then the server will wait for one more byte instead of close the connection immediately.

Noisyfox commented Jan 18, 2017

@Mygod If some one increase the length by one byte, then the server will wait for one more byte instead of close the connection immediately.

@Noisyfox

This comment has been minimized.

Show comment
Hide comment
@Noisyfox

Noisyfox Jan 18, 2017

Well you see, AEAD's main point is never decrypt any data before the cipher text is been verified.

Noisyfox commented Jan 18, 2017

Well you see, AEAD's main point is never decrypt any data before the cipher text is been verified.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Jan 18, 2017

Contributor

@Noisyfox Good point. So maybe we should authenticate data.len and data separately instead? Like TAG, encrypted_data.len, TAG, encrypted_data?

Contributor

Mygod commented Jan 18, 2017

@Noisyfox Good point. So maybe we should authenticate data.len and data separately instead? Like TAG, encrypted_data.len, TAG, encrypted_data?

madeye added a commit to shadowsocks/shadowsocks-android that referenced this issue Feb 8, 2017

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Feb 8, 2017

Contributor

@madeye @riobard Oh okay.

Contributor

Mygod commented Feb 8, 2017

@madeye @riobard Oh okay.

@shinku721

This comment has been minimized.

Show comment
Hide comment
@shinku721

shinku721 Feb 9, 2017

I firmly support this enhancement personally, but I must point out that it cannot help shadowsocks to get rid of the risk completely.

In the new proposal, as I assumed, length with its tag is checked firstly, then the payload and its tag.

Now we can do a detection:

  1. record the stream of a certain connection
  2. change some byte and detect the length of further bytes we send that causes the server to close the connection
  3. now change the next byte beyond the length in step 2, and repeat step 2.
  4. finally we break up the stream into chunks

Since the length of (Length_Tag + Length) is fixed, we can see this pattern appear (maybe repeatedly if more than one chunk in the uploading stream) in those chunks. Then we are almost sure that the server is a shadowsocks.

It is possible to delay the time of disconnection, but maybe the attacker can wait until timeout, so that's is not the ultimate solution.

Actually I don't think the security of OTA is compromised, though it is MtE. HMAC without padded block cipher mode (CBC-like) has no obvious vulnerabilities currently. This is somehow a replay attack (the same request header passes the check and the data is performed).

shinku721 commented Feb 9, 2017

I firmly support this enhancement personally, but I must point out that it cannot help shadowsocks to get rid of the risk completely.

In the new proposal, as I assumed, length with its tag is checked firstly, then the payload and its tag.

Now we can do a detection:

  1. record the stream of a certain connection
  2. change some byte and detect the length of further bytes we send that causes the server to close the connection
  3. now change the next byte beyond the length in step 2, and repeat step 2.
  4. finally we break up the stream into chunks

Since the length of (Length_Tag + Length) is fixed, we can see this pattern appear (maybe repeatedly if more than one chunk in the uploading stream) in those chunks. Then we are almost sure that the server is a shadowsocks.

It is possible to delay the time of disconnection, but maybe the attacker can wait until timeout, so that's is not the ultimate solution.

Actually I don't think the security of OTA is compromised, though it is MtE. HMAC without padded block cipher mode (CBC-like) has no obvious vulnerabilities currently. This is somehow a replay attack (the same request header passes the check and the data is performed).

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Feb 9, 2017

Contributor

To defend against this kind of attack, we should probably drop malicious requests instead of closing the connection.

The biggest problem with OTA is that it's length in transmitted in plain text, not MtE.

Contributor

Mygod commented Feb 9, 2017

To defend against this kind of attack, we should probably drop malicious requests instead of closing the connection.

The biggest problem with OTA is that it's length in transmitted in plain text, not MtE.

@shinku721

This comment has been minimized.

Show comment
Hide comment
@shinku721

shinku721 Feb 9, 2017

@Mygod Really? I have implemented shadowsocks myself for serval times and I always encrypted the length field together with MAC and payload, and it seems to be compatible with libev versions.

shinku721 commented Feb 9, 2017

@Mygod Really? I have implemented shadowsocks myself for serval times and I always encrypted the length field together with MAC and payload, and it seems to be compatible with libev versions.

@Mygod

This comment has been minimized.

Show comment
Hide comment
@Mygod

Mygod Feb 9, 2017

Contributor

Hmmmm I guess I misunderstood the documentation...

Contributor

Mygod commented Feb 9, 2017

Hmmmm I guess I misunderstood the documentation...

@printempw

This comment has been minimized.

Show comment
Hide comment
@printempw

printempw Feb 9, 2017

The DATA.LEN is transmitted as ciphertext, actually.

But the biggest problem with OTA is that adversaries can build evil packets and do server behavior based attacks since DATA.LEN is not authenticated in OTA.

Some details about the attack: https://blessing.studio/why-do-shadowsocks-deprecate-ota/

printempw commented Feb 9, 2017

The DATA.LEN is transmitted as ciphertext, actually.

But the biggest problem with OTA is that adversaries can build evil packets and do server behavior based attacks since DATA.LEN is not authenticated in OTA.

Some details about the attack: https://blessing.studio/why-do-shadowsocks-deprecate-ota/

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Feb 10, 2017

Contributor

@shinku721 One interesting thing is most of inner traffic of shadowsocks is TLS. It means even with stream ciphers, we would see chunks in shadowsocks traffic.

Is it a characteristic? Yes, it is. But is it really useful for attacking? No, I don't think so.

Contributor

madeye commented Feb 10, 2017

@shinku721 One interesting thing is most of inner traffic of shadowsocks is TLS. It means even with stream ciphers, we would see chunks in shadowsocks traffic.

Is it a characteristic? Yes, it is. But is it really useful for attacking? No, I don't think so.

@shinku721

This comment has been minimized.

Show comment
Hide comment
@shinku721

shinku721 Feb 11, 2017

@madeye Chunks are not the pattern, but the length of the chunks is.
However, I agree with you that this cannot be a practical attack. I mean, it has some characters similar to the weakness of OTA mentioned above, that it has to replay the request header. Both attacks are currently in theory and no evidence can prove that the GFW has already adopted any of them.

The reason why I support AEAD ciphers is just that AEAD ciphers are widely used in current practice.
IMO there is no urgent need to deprecate OTA, but we must stop using the original protocol without forcing OTA.

shinku721 commented Feb 11, 2017

@madeye Chunks are not the pattern, but the length of the chunks is.
However, I agree with you that this cannot be a practical attack. I mean, it has some characters similar to the weakness of OTA mentioned above, that it has to replay the request header. Both attacks are currently in theory and no evidence can prove that the GFW has already adopted any of them.

The reason why I support AEAD ciphers is just that AEAD ciphers are widely used in current practice.
IMO there is no urgent need to deprecate OTA, but we must stop using the original protocol without forcing OTA.

@riobard

This comment has been minimized.

Show comment
Hide comment
@riobard

riobard Feb 11, 2017

Contributor

Please note that SIP004 is superseded by SIP007 (#42).

Contributor

riobard commented Feb 11, 2017

Please note that SIP004 is superseded by SIP007 (#42).

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

从梅林被老司机领到这里,我就想问问大家,有什么决定性的代码或者证据证明OTA是先天不足必须给钦定的AEAD让路么?如果有,请联系我,我提供个C段地址,求扫描

tigaly commented Feb 11, 2017

从梅林被老司机领到这里,我就想问问大家,有什么决定性的代码或者证据证明OTA是先天不足必须给钦定的AEAD让路么?如果有,请联系我,我提供个C段地址,求扫描

@hellofwy

This comment has been minimized.

Show comment
Hide comment
@hellofwy

hellofwy Feb 11, 2017

@tigaly
可参考:https://blessing.studio/why-do-shadowsocks-deprecate-ota/,当然我并不是完全认同文章的内容。
就我这段时间对他们讨论的观察,决定用 AEAD 主要对安全的的考虑,并不是防识别。
因为现在也引入了插件功能,可以通过插件实现协议混淆防检测,或实现协议伪装欺骗运营商 QoS 获取加速,或者其他功能。

hellofwy commented Feb 11, 2017

@tigaly
可参考:https://blessing.studio/why-do-shadowsocks-deprecate-ota/,当然我并不是完全认同文章的内容。
就我这段时间对他们讨论的观察,决定用 AEAD 主要对安全的的考虑,并不是防识别。
因为现在也引入了插件功能,可以通过插件实现协议混淆防检测,或实现协议伪装欺骗运营商 QoS 获取加速,或者其他功能。

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@hellofwy 这篇文章针对OTA不就是那个很容易fix的漏洞么(通篇吹破哇)?还有什么直接证据呢?那么,出于安全级别的考虑有没有什么别的漏洞无解呢?

tigaly commented Feb 11, 2017

@hellofwy 这篇文章针对OTA不就是那个很容易fix的漏洞么(通篇吹破哇)?还有什么直接证据呢?那么,出于安全级别的考虑有没有什么别的漏洞无解呢?

@hellofwy

This comment has been minimized.

Show comment
Hide comment
@hellofwy

hellofwy Feb 11, 2017

@tigaly
就我目前知道的,并没什么直接证据。
安全这东西无止境,建议关注 TLS 协议最新相关研究,毕竟这个是世界范围内广泛使用和关注的。
作为普通使用者,在意安全的时候,使用 HTTPS 就好。不要太依赖底层 ss 协议。

hellofwy commented Feb 11, 2017

@tigaly
就我目前知道的,并没什么直接证据。
安全这东西无止境,建议关注 TLS 协议最新相关研究,毕竟这个是世界范围内广泛使用和关注的。
作为普通使用者,在意安全的时候,使用 HTTPS 就好。不要太依赖底层 ss 协议。

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@hellofwy 我并不是在意安全问题,我只是用梅林的时候发现了OTA更新被直接干掉了,才想来学习下,不过这种漏洞解决的方式真奇怪哈,要是哪天证明AEAD也有问题?是不是要搞个DEAD协议出来?

tigaly commented Feb 11, 2017

@hellofwy 我并不是在意安全问题,我只是用梅林的时候发现了OTA更新被直接干掉了,才想来学习下,不过这种漏洞解决的方式真奇怪哈,要是哪天证明AEAD也有问题?是不是要搞个DEAD协议出来?

@v3aqb

This comment has been minimized.

Show comment
Hide comment
@v3aqb

v3aqb Feb 11, 2017

v3aqb commented Feb 11, 2017

@breakwa11

This comment has been minimized.

Show comment
Hide comment
@breakwa11

breakwa11 Feb 11, 2017

@tigaly 既然不在意安全问题,那你直接用最早的版本的协议就好了,用OTA干什么?OTA本质就是增强安全为目的设计的

breakwa11 commented Feb 11, 2017

@tigaly 既然不在意安全问题,那你直接用最早的版本的协议就好了,用OTA干什么?OTA本质就是增强安全为目的设计的

@hellofwy

This comment has been minimized.

Show comment
Hide comment
@hellofwy

hellofwy Feb 11, 2017

@tigaly
想太多。
OTA 一直也只是标为实验性功能。
AEAD 是信息安全领域的已有名词,并不是这里的人自己创造的。

hellofwy commented Feb 11, 2017

@tigaly
想太多。
OTA 一直也只是标为实验性功能。
AEAD 是信息安全领域的已有名词,并不是这里的人自己创造的。

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@breakwa11 那你去掉OTA就是为了安全?有没有证据可以证明呢?我并不在意安全问题,并不是我心大到完全不在意安全,只要gfw探测不到就行了

@hellofwy 我当然知道,只是开玩笑的说法,对于大家这种严谨的做法,我真是大开眼界

tigaly commented Feb 11, 2017

@breakwa11 那你去掉OTA就是为了安全?有没有证据可以证明呢?我并不在意安全问题,并不是我心大到完全不在意安全,只要gfw探测不到就行了

@hellofwy 我当然知道,只是开玩笑的说法,对于大家这种严谨的做法,我真是大开眼界

@breakwa11

This comment has been minimized.

Show comment
Hide comment
@breakwa11

breakwa11 Feb 11, 2017

@tigaly 什么叫做“我去掉OTA”??????????

breakwa11 commented Feb 11, 2017

@tigaly 什么叫做“我去掉OTA”??????????

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@breakwa11 除了你还有谁一直在鼓吹OTA有不可修复的漏洞?

tigaly commented Feb 11, 2017

@breakwa11 除了你还有谁一直在鼓吹OTA有不可修复的漏洞?

@breakwa11

This comment has been minimized.

Show comment
Hide comment
@breakwa11

breakwa11 Feb 11, 2017

所以我也可以说:windows和linux有一大堆漏洞,然后修复漏洞的人就是我了,是这种逻辑吧,太好了,成百上千的都是我弄的

breakwa11 commented Feb 11, 2017

所以我也可以说:windows和linux有一大堆漏洞,然后修复漏洞的人就是我了,是这种逻辑吧,太好了,成百上千的都是我弄的

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@breakwa11 你这手驴打滚偷换概念玩的6,我也懒得跟你说,OTA和AEAD这两个功能的更替,有多少你在里面带节奏,明眼人一看就知道,不是你想推脱就推脱的

tigaly commented Feb 11, 2017

@breakwa11 你这手驴打滚偷换概念玩的6,我也懒得跟你说,OTA和AEAD这两个功能的更替,有多少你在里面带节奏,明眼人一看就知道,不是你想推脱就推脱的

@breakwa11

This comment has been minimized.

Show comment
Hide comment
@breakwa11

breakwa11 Feb 11, 2017

那不然呢?SS的代码我一行没动,在一行没动的情况下就成了是我改的?我不动一根手指但几千代码都是我改的,哇,难不成你说madeye的号被盗了?

breakwa11 commented Feb 11, 2017

那不然呢?SS的代码我一行没动,在一行没动的情况下就成了是我改的?我不动一根手指但几千代码都是我改的,哇,难不成你说madeye的号被盗了?

@RobertYan

This comment has been minimized.

Show comment
Hide comment
@RobertYan

RobertYan Feb 11, 2017

@tigaly @breakwa11 Please stop posting any irrelevant comments here.

RobertYan commented Feb 11, 2017

@tigaly @breakwa11 Please stop posting any irrelevant comments here.

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@breakwa11 有种叫教唆犯你不会没听过吧

tigaly commented Feb 11, 2017

@breakwa11 有种叫教唆犯你不会没听过吧

@hellofwy

This comment has been minimized.

Show comment
Hide comment
@hellofwy

hellofwy Feb 11, 2017

@tigaly
这里讨论关于人的问题不好。
毕竟这里说最好都是软件的问题。
针对人的在其他地方讨论就好。

hellofwy commented Feb 11, 2017

@tigaly
这里讨论关于人的问题不好。
毕竟这里说最好都是软件的问题。
针对人的在其他地方讨论就好。

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@hellofwy 是啊,本来说的好好的,不知道哪来的人跳出来一阵节奏和歪楼,
另外我翻了下好多关于OTA的讨论都完全被人带节奏,TLS和SS的用途被混为一谈,我就不说了,反正我也不怎么用ss,你们大家慢慢加油吧

tigaly commented Feb 11, 2017

@hellofwy 是啊,本来说的好好的,不知道哪来的人跳出来一阵节奏和歪楼,
另外我翻了下好多关于OTA的讨论都完全被人带节奏,TLS和SS的用途被混为一谈,我就不说了,反正我也不怎么用ss,你们大家慢慢加油吧

@qinyuhang

This comment has been minimized.

Show comment
Hide comment
@qinyuhang

qinyuhang Feb 11, 2017

qinyuhang commented Feb 11, 2017

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@qiuyuzhou 我进这个讨论组第一句话就说了,谁能针对OTA探测,求扫描,我提供一个C段地址,只是其他人一直歪楼带节奏才变成这样

tigaly commented Feb 11, 2017

@qiuyuzhou 我进这个讨论组第一句话就说了,谁能针对OTA探测,求扫描,我提供一个C段地址,只是其他人一直歪楼带节奏才变成这样

@qinyuhang

This comment has been minimized.

Show comment
Hide comment
@qinyuhang

qinyuhang Feb 11, 2017

qinyuhang commented Feb 11, 2017

@tigaly

This comment has been minimized.

Show comment
Hide comment
@tigaly

tigaly Feb 11, 2017

@qiuyuzhou 如果她敢接就不会一直打嘴仗了,从sadoneli一直扯到这里来

tigaly commented Feb 11, 2017

@qiuyuzhou 如果她敢接就不会一直打嘴仗了,从sadoneli一直扯到这里来

@qinyuhang

This comment has been minimized.

Show comment
Hide comment
@qinyuhang

qinyuhang Feb 11, 2017

qinyuhang commented Feb 11, 2017

@shadowsocks shadowsocks locked and limited conversation to collaborators Feb 11, 2017

@madeye

This comment has been minimized.

Show comment
Hide comment
@madeye

madeye Feb 11, 2017

Contributor

Locked as off-topic.

Contributor

madeye commented Feb 11, 2017

Locked as off-topic.

@qiuyuzhou

This comment has been minimized.

Show comment
Hide comment
@qiuyuzhou

qiuyuzhou Mar 19, 2017

@tigaly 你@错人了。。。

qiuyuzhou commented Mar 19, 2017

@tigaly 你@错人了。。。

dszhengyu referenced this issue in shadowsocks/shadowsocks-go Sep 4, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.