Testing masque server proxy and client #4

deshmukhrajvardhan · 2021-09-24T19:21:51Z

Do you have instructions to test the masque server proxy and client?

I tried the below:

Before i did tested the masque client/server, i tested the simple_client(toy) and simple_server(toy)
and was able to connect and transfer data file.

Then i tested with masque

started the server (proxy)

./build/masque_quic_server --cache_dir=/data/quic-root/  --certificate_file=/data/quic-cert/leaf_cert.pem   --key_file=/data/quic-cert/leaf_cert.pkcs8  --cache_dir=/data/quic-root/ --port=9661

Started Open MASQUE server

started the client

./build/masque_quic_client   --disable_certificate_verification=true   127.0.0.1:9661   "https://www.example.org/index.html"

MASQUE is connected 5e56f37d2d6ce2e7 in Open mode

but the data file wasn't sent over from the masque server to the client and i saw dns requests to try to resolve the url.

Failed to connect with client f7f7d2ac0ceccd3d server bebdd806c8411e8f to www.example.org. Error: QUIC_NETWORK_IDLE_TIMEOUT

Do i have to configure the masque server proxy to a quic server? How do i do that (don't see options/examples)?

Should the topology look like below? And how do i configure the proxy?

masque_client<->masque_server_proxy<->quic_server

Thanks,
Raj

The text was updated successfully, but these errors were encountered:

DavidSchinazi · 2021-09-24T20:12:09Z

Hi Raj, yes currently the masque_client only speaks QUIC over MASQUE. www.example.org does not speak QUIC so this won't work. If you replace www.example.org with www.google.com (or any other QUIC server) it should work.

deshmukhrajvardhan · 2021-09-24T20:40:34Z

Hi @DavidSchinazi,

Thanks for the quick response. I tried this but it didn't work.

./build/masque_quic_client   --disable_certificate_verification=true   127.0.0.1:9661   "https://www.google.com/index.html"

I am supplying the masque_server (proxy's) ip:port, i assume that's correct?

i was planning to use the toy server in quiche. Would i have to configure the masque_server proxy?

Thanks again!

DavidSchinazi · 2021-09-27T16:00:55Z

That should work. What exactly are you trying to do?

deshmukhrajvardhan · 2021-09-27T16:54:45Z

Aim is to see MASQUE in action and also capture and look at the pcaps.

To do that, wanted to use quiche's quic toy server, masque server (proxy) and masque client to request (GET) and transfer a file.

quic-server<->masque-server<->masque-client

./build/simple_quic_server   --quic_response_cache_dir=/data/quic-root/   --certificate_file=/data/quic-cert/leaf_cert.pem   --key_file=/data/quic-cert/leaf_cert.pkcs8

# $ ls /data/quic-root/www.example.org/
#    index.html

./build/masque_quic_server --cache_dir=/data/quic-root/  --certificate_file=/data/quic-cert/leaf_cert.pem   --key_file=/data/quic-cert/leaf_cert.pkcs8

./build/masque_quic_client   --disable_certificate_verification=true   127.0.0.1:9661   "https://www.example.org/index.html"

But i get this error from the masque-client:

Failed to connect with client 0e5820cb29363791 server 5e025d66d4f957af to www.example.org. Error: QUIC_NETWORK_IDLE_TIMEOUT

DavidSchinazi · 2021-09-27T17:39:15Z

As discussed in #4 (comment), you can't use www.example.org because that doesn't support QUIC.

deshmukhrajvardhan · 2021-09-27T18:08:54Z

Sorry about not stating all the steps, I will state them below:

I Download the html file and generate certs

mkdir -p /data/quic-root && wget -p --save-headers https://www.google.com -P /data/quic-root && cd -

and host it with the quic server below.

I tried this (quiche's quic toy server<->client):

./build/simple_quic_server   --quic_response_cache_dir=/data/quic-root/   --certificate_file=/data/quic-cert/leaf_cert.pem   --key_file=/data/quic-cert/leaf_cert.pkcs8

./build/simple_quic_client   --disable_certificate_verification=true   --host=127.0.0.1 --port=6121 "https://www.google.com/index.html"

And that works. (file is transferred to the client)

Then to test masque server(proxy) and masque client:
quic-server<->masque-server<->masque-client
a. keep the earlier quiche's quic server on.
b. start masque-server (proxy)

./build/masque_quic_server --cache_dir=/data/quic-root/  --certificate_file=/data/quic-cert/leaf_cert.pem   --key_file=/data/quic-cert/leaf_cert.pkcs8

c. start the masque client

./build/masque_quic_client   --disable_certificate_verification=true   127.0.0.1:9661 "https://www.google.com/index.html"

the masque client is connected to the masque server (proxy)
MASQUE is connected 69e0263f0d59bb4d in Open mode

But get this error:

Failed to connect with client 8005ed888fdc4487 server 1395f38f2bf16752 to www.google.com. Error: QUIC_NETWORK_IDLE_TIMEOUT

Do i have to configure the masque_server proxy?

DavidSchinazi · 2021-09-27T19:12:24Z

Oh I see. masque_server will attempt to connect to real hostnames so it'll try to connect to the real www.example.org or the real www.google.com - so it won't talk to your quic_server if you tell it to go to those. If you want to do masque_client -> masque_server -> quic_server, you'll need to run masque_client like this:

./build/masque_quic_client --disable_certificate_verification  <IP_AND_PORT_OF_MASQUE_SERVER> "https://<IP_AND_PORT_OF_QUIC_SERVER>/index.html"

so in your example I think you'll want:

./build/masque_quic_client --disable_certificate_verification 127.0.0.1:9661 "https://127.0.0.1:6121/index.html"

deshmukhrajvardhan · 2021-09-27T20:34:53Z

Thanks @DavidSchinazi for the clarification and the solutions, that worked!
I had to create a folder /data/quic-root/127.0.0.1:6121 and place the index.html in it.
Another observation, it only works for --masque_mode=legacy (doesn't work for open mode, not sure why).

The difference between legacy and open is:

quiche/quic/masque/masque_server_backend.cc

Line 31 in 2682b51

if (masque_mode_ == MasqueMode::kLegacy) {

and

quiche/quic/masque/masque_epoll_client.cc

Line 89 in 2682b51

if (masque_client->masque_mode() == MasqueMode::kLegacy) {

DavidSchinazi · 2021-09-27T20:53:50Z

Note that you need to specify the same --masque_mode on both the masque_client and the masque_server, did you do that? If you did, can I ask you to run both commands with --v=2 and send me the logs?

deshmukhrajvardhan · 2021-09-27T21:25:51Z

I used the same --masque_mode on the masque_client and masque_server.
the masque_server and masque_client binaries don't recognize --v command (Invalid flag of v)

I have been using this platform (it has minimal modules) to build quiche,
https://github.com/bilibili/quiche/blob/main/CMakeLists.txt
and added this to the CMakefile

### masque quic client
SET(MASQUE_QUIC_CLIENT_SRCS
    gquiche/quic/masque/masque_client_bin.cc
    gquiche/quic/tools/quic_epoll_client_factory.cc
    # gquiche/quic/tools/quic_toy_client.cc
    gquiche/quic/tools/quic_url.cc
    gquiche/quic/tools/quic_spdy_client_base.cc
    gquiche/quic/tools/quic_client_base.cc
    gquiche/quic/tools/quic_client.cc
    gquiche/quic/tools/quic_spdy_client_base.cc
    gquiche/quic/tools/quic_simple_client_stream.cc
    gquiche/quic/tools/quic_simple_client_session.cc
    gquiche/quic/tools/quic_client_epoll_network_helper.cc
)

# Build bvc quic server binaries.
ADD_EXECUTABLE(masque_quic_client ${MASQUE_QUIC_CLIENT_SRCS} ${EPOLL_SERVER_SRC})
TARGET_LINK_LIBRARIES(masque_quic_client -static-libstdc++
    quiche
)

### masque quic server
SET(MASQUE_QUIC_SERVER_SRCS
    gquiche/quic/masque/masque_server_bin.cc
    gquiche/quic/tools/quic_backend_response.cc
    gquiche/quic/tools/quic_epoll_server_factory.cc
    gquiche/quic/tools/quic_memory_cache_backend.cc
    gquiche/quic/tools/quic_server.cc
    gquiche/quic/tools/quic_simple_crypto_server_stream_helper.cc
    gquiche/quic/tools/quic_simple_dispatcher.cc
    gquiche/quic/tools/quic_simple_server_session.cc
    gquiche/quic/tools/quic_simple_server_stream.cc
    # gquiche/quic/tools/quic_toy_server.cc
    gquiche/quic/tools/quic_url.cc
)

# Build bvc quic server binaries.
ADD_EXECUTABLE(masque_quic_server ${MASQUE_QUIC_SERVER_SRCS} ${EPOLL_SERVER_SRC})
TARGET_LINK_LIBRARIES(masque_quic_server -static-libstdc++
    quiche
)

which files would i have to build along with the client and server to have the --v command

OR

do you have instructions about how to build it with chromium or other suggestions?

DavidSchinazi · 2021-09-27T23:02:22Z

I've never used that platform so I don't know if it works. I'd suggest building from Chromium source:

cd chromium/src
autoninja -C out/Default masque_server
out/Default/masque_server --certificate_file=leaf_cert.pem --key_file=leaf_cert.pkcs8 --v=2
autoninja -C out/Default masque_client
out/Default/masque_client --disable_certificate_verification <IP_AND_PORT_OF_MASQUE_SERVER> "https://<IP_AND_PORT_OF_QUIC_SERVER>/index.html" --v=2

deshmukhrajvardhan · 2021-09-28T19:06:55Z

Hi David,

It works (quic-server<->masque-server<->masque-client) fine with Chromium.
Thanks for the instructions and your help.
We can close this issue.
Before that, can you point me to the explanation about the difference between open mode and legacy mode.

Thanks,
Raj

DavidSchinazi · 2021-09-28T22:47:52Z

I'm happy to hear that!

The modes are discussed here.

*** Reason for rollback *** This causes test failures in Chromium. I would normally fix it, but currently QUICHE roll to Chromium is blocked on another CL and there are two other CLs that are complicated to roll. Please allow me to roll this one back for now, I'll be happy to help with debugging after I am able to roll the latest QUICHE into Chromium. Error is: [ RUN ] HeaderValidatorTest.NameHasInvalidChar ../../buildtools/third_party/libc++/trunk/include/array:205: _LIBCPP_ASSERT '__n < _Size' failed. out-of-bounds access in std::array<T, N> Received signal 6 #0 0x7fe693f3799f base::debug::CollectStackTrace() #1 0x7fe693c8fd3a base::debug::StackTrace::StackTrace() #2 0x7fe693c8fcf5 base::debug::StackTrace::StackTrace() #3 0x7fe693f3746c base::debug::(anonymous namespace)::StackDumpSignalHandler() #4 0x7fe6911f1200 (/lib/x86_64-linux-gnu/libpthread-2.33.so+0x131ff) #5 0x7fe690db3891 gsignal #6 0x7fe690d9d536 abort #7 0x7fe6912c941c std::__Cr::__libcpp_abort_debug_function() #8 0x7fe6953c9996 std::__Cr::array<>::operator[]() #9 0x7fe6953c95c9 http2::adapter::(anonymous namespace)::AllCharsInMap() #10 0x7fe6953c87ec http2::adapter::(anonymous namespace)::IsValidHeaderName() #11 0x7fe6953c7e15 http2::adapter::HeaderValidator::ValidateSingleHeader() #12 0x56309a95f81d http2::adapter::test::HeaderValidatorTest_NameHasInvalidChar_Test::TestBody() #13 0x56309bf0b83b testing::internal::HandleSehExceptionsInMethodIfSupported<>() #14 0x56309befc767 testing::internal::HandleExceptionsInMethodIfSupported<>() #15 0x56309beeaac1 testing::Test::Run() #16 0x56309beeb1d7 testing::TestInfo::Run() #17 0x56309beeb83d testing::TestSuite::Run() #18 0x56309bef5e7a testing::internal::UnitTestImpl::RunAllTests() #19 0x56309bf0fa3b testing::internal::HandleSehExceptionsInMethodIfSupported<>() #20 0x56309befe337 testing::internal::HandleExceptionsInMethodIfSupported<>() #21 0x56309bef5a11 testing::UnitTest::Run() #22 0x56309c167aa1 RUN_ALL_TESTS() *** Original change description *** Use static char maps in HeaderValidator::ValidateSingleHeader(). When validating header names/values against allowed characters, switching to static char maps may help with performance. This CL is otherwise not a functional change. *** PiperOrigin-RevId: 425943183

deshmukhrajvardhan closed this as completed Sep 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Testing masque server proxy and client #4

Testing masque server proxy and client #4

deshmukhrajvardhan commented Sep 24, 2021 •

edited

Loading

DavidSchinazi commented Sep 24, 2021

deshmukhrajvardhan commented Sep 24, 2021 •

edited

Loading

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 28, 2021

DavidSchinazi commented Sep 28, 2021

Testing masque server proxy and client #4

Testing masque server proxy and client #4

Comments

deshmukhrajvardhan commented Sep 24, 2021 • edited Loading

DavidSchinazi commented Sep 24, 2021

deshmukhrajvardhan commented Sep 24, 2021 • edited Loading

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 27, 2021

OR

DavidSchinazi commented Sep 27, 2021

deshmukhrajvardhan commented Sep 28, 2021

DavidSchinazi commented Sep 28, 2021

deshmukhrajvardhan commented Sep 24, 2021 •

edited

Loading

deshmukhrajvardhan commented Sep 24, 2021 •

edited

Loading