Ticket20700 035 02 #301

dgoulet-tor · 2018-08-30T14:12:24Z

No description provided.

This commit loads all client public keys from every file in `authorized_clients/` directory. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

We need to generate all the related keys when building the descriptor, so that we can encrypt the descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

This commit tests that the descriptor building result, when the client authorization is enabled, includes everything that is needed. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Previously, we encrypted the descriptor without the descriptor cookie. This commit, when the client auth is enabled, the descriptor cookie is always used. I also removed the code that is used to generate fake auth clients because it will not be used anymore. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

The new ClientOnionAuthDir option is introduced which is where tor looks to find the HS v3 client authorization files containing the client private key material. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Because this secret data building logic is not only used by the descriptor encoding process but also by the descriptor decoding, refactor the function to take both steps into account. Signed-off-by: David Goulet <dgoulet@torproject.org>

This commit refactors the existing decryption code to make it compatible with a new logic for when the client authorization is enabled. Signed-off-by: David Goulet <dgoulet@torproject.org>

Parse the client authorization section from the descriptor, use the client private key to decrypt the auth clients, and then use the descriptor cookie to decrypt the descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Previously, the validation by decoding a created descriptor was disabled because the interface had to be entirely changed and not implemented at the time. This commit re-enabled it because it is now implemented. Signed-off-by: David Goulet <dgoulet@torproject.org>

Series of functions that we now need in hs_service.c. Signed-off-by: David Goulet <dgoulet@torproject.org>

When reloading tor, check if our the configured client authorization have changed from what we previously had. If so, republish the updated descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

Signed-off-by: David Goulet <dgoulet@torproject.org>

Part of #20700. Signed-off-by: David Goulet <dgoulet@torproject.org>

coveralls · 2018-08-30T14:44:35Z

Coverage increased (+0.3%) to 59.832% when pulling a2b821c on dgoulet-tor:ticket20700_035_02 into 94605f0 on torproject:master.

In function get_fname_suffix, previously it uses /, but in fact it should use PATH_SEPARATOR.

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+static int
+client_filename_is_valid(const char *filename)
+{
+  int ret = 1;


This function should just use "strcmpend()."

Fixup commit: 67fa226

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+static hs_service_authorized_client_t *
+parse_authorized_client(const char *client_key_str)
+{
+  char *auth_type = NULL;


Maybe it would be a good idea to describe the format in a comment here too.

Fixup commit: de70396

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+  }
+
+  client = tor_malloc_zero(sizeof(hs_service_authorized_client_t));
+  if (base32_decode((char *) client->client_pk.public_key,


Will this detect truncated public keys?

Fixup commit: 7396175

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+   * cannot create a client pubkey key directory, we consider it as a bug. */
+  client_keys_dir_path = hs_path_from_filename(config->directory_path,
+                                               dname_client_pubkeys);
+  if (BUG(hs_check_service_private_dir(get_options()->User,


Are we calling this from inside the options code, or from somewhere else? If it's inside the options code, we can't call get_options() -- we should be taking the or_options_t as an argument instead. But if it's happening from somewhere else, this is okay.

Soooo we might have a bigger problem here. It is ultimately called from options_act() and for v3 it is easy to fix but we'll need to do it for v2 as well.

get_options()->User is what is accessed during key loading since we double check if the directory permissions are valid.

This will touch many parts of HS and Rend subsystem so I propose we do this after with its own set of commits?

hm, ok. Actually, it might be safe to leave this alone, since the User option is not allowed to change while Tor is running.

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+
+  file_list = tor_listdir(client_keys_dir_path);
+
+  SMARTLIST_FOREACH_BEGIN(file_list, char *, filename) {


Could this be const char *?

Fixup commit: 75a261c

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_service.c

+
+  config->clients = smartlist_new();
+
+  file_list = tor_listdir(client_keys_dir_path);


We need to check for whether this value is NULL, I think.

Fixup commit: 5505486

nmathewson · 2018-09-05T13:17:27Z

src/test/test_hs_service.c

+    "descriptor:x:dz4q5xqlb4ldnbs72iarrml4ephk3du4i7o2cgiva5lwr6wkquja"));
+  /* Some malformed string. */
+  tt_assert(!parse_authorized_client("descriptor:x25519:aa=="));
+


I'd suggest adding checks for 1-field, 2-field, and 0-field strings as well: they're a corner case that we're likely to mess up in the future if we don't have tests.

Fixup commit: 629c188

nmathewson · 2018-09-05T13:17:27Z

src/test/test_hs_service.c

+
+#undef conf_fmt
+
+  MOCK(read_file_to_str, mock_read_file_to_str);


It's okay to do this kind of mocking -- but in the future, it might be just as good (or better) to just put the files into a temporary directory on disk. See uses of get_fname() for other places in the tests where we generate temporary files and directories.

nmathewson · 2018-09-05T13:17:27Z

src/test/test_hs_service.c

+
+    SMARTLIST_FOREACH_BEGIN(service->config.clients,
+                            hs_service_authorized_client_t *, client) {
+      if (!tor_memcmp(&pubkey, &client->client_pk, sizeof(pubkey))) {


See documentation for tor_memcmp: it's better to use tor_memeq() in this case.

err, ug. This is not actually documented enough. Will document now.

Fixup commit: f8e98f8

nmathewson · 2018-09-05T13:17:27Z

src/feature/hs/hs_descriptor.c

+
+  /* Calculate KEYS = KDF(SECRET_SEED, 40) */
+  xof = crypto_xof_new();
+  crypto_xof_add_bytes(xof, secret_seed, sizeof(secret_seed));


Now that I'm looking at this implementation, I'm a little concerned that we're only putting the curve25519 output into the KDF. I think we should also put something about the hidden service itself (its identity? its blinded identity?) into the KDF, to prevent tricky attacks where the ephemeral public key is copied from one descriptor to another. What do you think?

Hm. I'm not sure exactly what kind of attack that could enable, but it does seem like a sane KDF use choice so we can do it.

I made a torspec PR with a design that adds the HS subcredential to the KDF computation:
torproject/torspec#34

(Using the subcred there is like using both the blinded and the identity key and it was by far the easiest solution in terms of engineering.)

Please also see my branch ticket20700_035_02 for two commits that implement the above.
https://github.com/asn-d6/tor/tree/ticket20700_035_02

Some new test vectors broke because of adding the subcred but I don't have the script that generated them so I just updated them... Can you please rerun the script if you have it?

nmathewson · 2018-09-05T13:20:41Z

src/feature/hs/hs_descriptor.c

@@ -2475,6 +2504,84 @@ hs_desc_intro_point_free_(hs_desc_intro_point_t *ip)
  tor_free(ip);
 }

+/* Build a fake client info for the descriptor */
+void


Should this function (and the one below) just allocate and return a new hs_desc_authorized_client_t ?

Commit c6403e6

nmathewson

See comments

nmathewson

see comments

nmathewson · 2018-09-05T13:40:01Z

src/feature/hs/hs_descriptor.c

+  /* If the client id of auth client is not the same as the calculcated
+   * client id, it means that this auth client is invaild according to the
+   * client secret key client_sk. */
+  if (!tor_memeq(client->client_id, keystream, HS_DESC_CLIENT_ID_LEN)) {


Instead of !tor_memeq, you can use tor_memneq

Fixup commit 54d9da1

nmathewson · 2018-09-05T13:40:46Z

src/feature/hs/hs_descriptor.c

+        tor_malloc_zero(sizeof(hs_desc_authorized_client_t));
+
+      if (decode_auth_client(token, client) < 0) {
+        log_warn(LD_REND, "Auth client is not valid");


Is this message actually going to tell the user what they need to know?

What will happen to the connection? Will it be retried?

I've improved a bit the message in fixup commit 1f98b67.

We could make it that the log is much more verbose but at this stage this is decoding of a descriptor so it will ultimately result in the SOCKS connection being closed client side because we can't properly decode the descriptor.

nmathewson · 2018-09-05T13:42:55Z

src/feature/hs/hs_service.c

+  idx_matched =
+    tor_malloc_zero(sizeof(*idx_matched) * smartlist_len(config2->clients));
+
+  SMARTLIST_FOREACH_BEGIN(config1->clients,


I don't like doing this quadratic loop here -- do we really need to do this?

One option would be to sort the lists, then compare them in lockstep.

Come to think of it, we should probably also scramble the lists before we actually use them, so a client doesn't learn anything from its position in the list.

This is unfortunately needed... It will happen rarely that is when the service is HUP.

The reason we need this is to detect if there are new client authorization configured for the service and if so, we need to build a new descriptor for the service and republish.

So for that, we have to go through all clients from both services object (old and new) and figure out if any of them changed (removed or added).

Come to think of it, we should probably also scramble the lists before we actually use them, so a client doesn't learn anything from its position in the list.

Oh hmmmm, yeah I think that is not a stupid idea. We should scramble that list after loading the key file. I propose we do this after, noting down.

I agree to scramble the list. It's safer to not let the client know the position.
Yes it's better to improve the running time. Do we need to do it in this branch?

A separate branch is fine, but IMO it should happen in 0.3.5 (especially since it is easy to do)

nmathewson · 2018-09-05T13:44:57Z

src/feature/hs/hs_service.c

@@ -1136,23 +1136,31 @@ parse_authorized_client(const char *client_key_str)
                         SPLIT_SKIP_SPACE, 0);
  /* Wrong number of fields. */
  if (smartlist_len(fields) != 3) {
+    log_warn(LD_REND, "The file is in a wrong format.");


Let's say which file -- it's not going to help the user otherwise.

This has been fixed with a future commit in this branch.

nmathewson · 2018-09-05T13:45:27Z

src/feature/hs/hs_service.c

    goto err;
  }

  client = tor_malloc_zero(sizeof(hs_service_authorized_client_t));
  if (base32_decode((char *) client->client_pk.public_key,
                    sizeof(client->client_pk.public_key),
                    pubkey_b32, strlen(pubkey_b32)) < 0) {
+    log_warn(LD_REND, "The public key cannot be decoded.");


Again, we should really be saying what file is busted, or the user won't know what to do about it.

Same as above.

Validate the length of the base32 encoded key taken from the filename.

Return a newly allocated fake client authorization object instead of taking the object as a parameter. Signed-off-by: David Goulet <dgoulet@torproject.org>

nmathewson · 2018-09-06T14:47:24Z

asn is right here.This makes me worried that this code is not tested.

asn-d6 · 2018-09-06T13:57:28Z

I think this is not right. base64_decode() returns the number of bytes written, not read. Hence we should check that the retval is what we expect to be getting out of base64, in this case I think sizeof(client->client_id) if I'm not mistaken.

e.g.:

digest_from_base64(char *digest, const char *d64)
{
  if (base64_decode(digest, DIGEST_LEN, d64, strlen(d64)) == DIGEST_LEN)
    return 0;
  else
    return -1;
}

asn-d6 · 2018-09-06T14:00:35Z

src/feature/hs/hs_descriptor.c

+  /* Try to decode what we just encoded. Symmetry is nice!, but it is
+   * symmetric only if the client auth is disabled. That is, the descriptor
+   * cookie will be NULL. */
+  if (!descriptor_cookie) {


Can't we also test-decode even if we have a descriptor cookie?

The reason is that hs_desc_decode_descriptor parameter requires the client private key not the descriptor cookie and we can't find the private key using descriptor cookie.

And I think using a private key as a parameter of hs_desc_decode_descriptor is more reasonable because we use a private key to decode not a descriptor cookie

Hm, I see what you mean. IIf I'm not mistaken, I think with some medium-level refactoring we could make that smooth and also have it test-decode in the client auth case as well (so that this code path gets used).

Perhaps we can open a ticket about this.

This reverts commit 97404e3.

Also update some client auth test vectors that broke...

nmathewson · 2018-09-30T21:31:57Z

A later version of this was merged

ppopth and others added 22 commits Aug 30, 2018

hs-v3: Load all client auth keys to the service

0f28284

This commit loads all client public keys from every file in `authorized_clients/` directory. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 loading client auth keys service side

629c188

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Generate all descriptor related keys

0a243a6

We need to generate all the related keys when building the descriptor, so that we can encrypt the descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 building a descriptor with client auth

2b2a303

This commit tests that the descriptor building result, when the client authorization is enabled, includes everything that is needed. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: Build an HSv3 descriptor with authorized client

d25e381

Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 descriptor encoding with client authorization

718f39d

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Load client authorization secret key from file

47d62f8

The new ClientOnionAuthDir option is introduced which is where tor looks to find the HS v3 client authorization files containing the client private key material. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 client authorization loading secret key

33071c8

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Refactor secret data building logic

a6790e9

Because this secret data building logic is not only used by the descriptor encoding process but also by the descriptor decoding, refactor the function to take both steps into account. Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Refactor the descriptor decryption/decoding

a7a0545

This commit refactors the existing decryption code to make it compatible with a new logic for when the client authorization is enabled. Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Decrypt the descriptor with client private key

09c3d4b

Parse the client authorization section from the descriptor, use the client private key to decrypt the auth clients, and then use the descriptor cookie to decrypt the descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 descriptor decoding with client authorization

af0b5b8

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Make all descriptor content free functions public

bbefbd8

Series of functions that we now need in hs_service.c. Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Republish descriptors if client auth changes

85115a2

When reloading tor, check if our the configured client authorization have changed from what we previously had. If so, republish the updated descriptor. Signed-off-by: David Goulet <dgoulet@torproject.org>

test: HS v3 client auth is config equal function

cd01788

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Rename client_sk to client_auth_sk

051d6ab

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Rename client_pk to client_auth_pk

7a18b4d

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Log client auth load activities service side

b49df5a

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Log client auth load activities client side

4eea079

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs-v3: Improve v3 client authorization logging

270bfc9

Part of #20700. Signed-off-by: David Goulet <dgoulet@torproject.org>

ppopth added 3 commits Sep 4, 2018

bug: Use PATH_SEPARATOR instead of slash

e7d6c0e

In function get_fname_suffix, previously it uses /, but in fact it should use PATH_SEPARATOR.

fixup! test: HS v3 client authorization loading secret key

f1bc43a

fixup! test: HS v3 loading client auth keys service side

9f715d5

nmathewson reviewed Sep 5, 2018

View changes

dgoulet-tor added 15 commits Sep 5, 2018

fixup! hs-v3: Load all client auth keys to the service

67fa226

fixup! hs-v3: Load all client auth keys to the service

de70396

fixup! hs-v3: Load all client auth keys to the service

7396175

Validate the length of the base32 encoded key taken from the filename.

fixup! hs-v3: Load all client auth keys to the service

75a261c

fixup! hs-v3: Log client auth load activities service side

7eaa650

fixup! hs-v3: Load all client auth keys to the service

5505486

fixup! test: HS v3 loading client auth keys service side

c87bf6a

fixup! test: HS v3 loading client auth keys service side

f8e98f8

hs-v3: Make hs_desc_build_fake_authorized_client() return an object

c6403e6

Return a newly allocated fake client authorization object instead of taking the object as a parameter. Signed-off-by: David Goulet <dgoulet@torproject.org>

fixup! hs-v3: Encrypt the descriptor using a cookie

fe50c14

fixup! hs-v3: Load client authorization secret key from file

9d1fe48

fixup! hs-v3: Load client authorization secret key from file

4ba612a

fixup! hs-v3: Decrypt the descriptor with client private key

97404e3

fixup! hs-v3: Decrypt the descriptor with client private key

54d9da1

fixup! hs-v3: Decrypt the descriptor with client private key

1f98b67

asn-d6 reviewed Sep 6, 2018

View changes

dgoulet-tor and others added 6 commits Sep 6, 2018

Revert "fixup! hs-v3: Decrypt the descriptor with client private key"

6e919ff

This reverts commit 97404e3.

fixup! hs-v3: Decrypt the descriptor with client private key

0510edf

HSv3: Add subcredential in client auth KDF on the service-side.

bcd2b2d

Also update some client auth test vectors that broke...

HSv3: Add subcredential in client auth KDF on the client-side.

0b60cc1

fixup! hs-v3: Republish descriptors if client auth changes

3bcfe21

HSv3: Don't assert when reading bad client-side privkeys.

a2b821c

nmathewson closed this Sep 30, 2018


		file_list = tor_listdir(client_keys_dir_path);

		SMARTLIST_FOREACH_BEGIN(file_list, char *, filename) {


		config->clients = smartlist_new();

		file_list = tor_listdir(client_keys_dir_path);


		#undef conf_fmt

		MOCK(read_file_to_str, mock_read_file_to_str);

torproject / tor Public

Ticket20700 035 02 #301

Ticket20700 035 02 #301

Conversation

dgoulet-tor commented Aug 30, 2018

coveralls commented Aug 30, 2018 • edited

asn-d6 Sep 6, 2018 • edited

asn-d6 Sep 6, 2018 • edited

nmathewson left a comment

nmathewson left a comment

nmathewson commented on 97404e3 Sep 6, 2018

asn-d6 commented on 97404e3 Sep 6, 2018

nmathewson commented Sep 30, 2018

coveralls commented Aug 30, 2018 •

edited

asn-d6 Sep 6, 2018 •

edited

asn-d6 Sep 6, 2018 •

edited

nmathewson commented on `97404e3` Sep 6, 2018

asn-d6 commented on `97404e3` Sep 6, 2018