Skip to content

Commit

Permalink
Merge branch 'master' into quic-latest
Browse files Browse the repository at this point in the history
* master:
  Make tls_conn_timeout test more reliable in CI (#7018)
  Remove deprecated verify.server for 9.0 (#7040)
  Updated GitHub description and homepage URL to be https (#7019)
  Add virtual destructor to QUICTPConfig. (#7036)
  Fix code to eliminate warning and enable feature (#7031)
  add a null check to avoid crashing (#7035)
  Squashed commit of the following: (#7000)
  Fixed problem with all "forced" volumes cache (#7028)
  Spacing tweaks to acl_filter_rule::print (#7026)
  Removes dead code from iocore/dns (#7025)
  Removes TODO (#7027)
  Add logic to resolve content-length transfer-encoding conflicts on response (#6992)
  Add memory_profile plugin (#7014)
  Fix typos relating to tls_bridge (#7011)
  slice: clean up of created 502 response header (#6919)
  Add new API / TSPluginDSOReloadEnable that overrides the configuration variable `proxy.config.plugin.dynamic_reload_mode` for a particular plugin. (#6880)
  Remove incorrect assert in inactivity timeout handling (#7012)
  Removes use of SPLIT_DNS macro (#7010)
  Fixed core when sending back a redirect and having an invalid server response (#7004)
  slice: fix throttle not work (#7008)
  Updates to thread scale factor (#7007)
  Added tasks and launch files for vscode, to configure, build and debug (#7005)
  NextHop Strategy Refactor and Fixes (#6782)
  Make the setting of the continuation handler safer. (#6996)
  ProtocolStack n -> count (#7006)
  Fix volume/stripe calcs when using forced volumes (#6995)
  Cleanup: Write error message on diags output instead of stderr (#6997)

 Conflicts:
	iocore/net/P_QUICNetVConnection.h
	iocore/net/P_QUICPacketHandler.h
	iocore/net/QUICNetProcessor.cc
	iocore/net/QUICNetVConnection.cc
	iocore/net/QUICPacketHandler.cc
	iocore/net/quic/Mock.h
	iocore/net/quic/QUICCongestionController.h
	iocore/net/quic/QUICContext.cc
	iocore/net/quic/QUICContext.h
	iocore/net/quic/QUICDebugNames.cc
	iocore/net/quic/QUICFrame.cc
	iocore/net/quic/QUICHandshake.cc
	iocore/net/quic/QUICKeyGenerator.h
	iocore/net/quic/QUICLossDetector.cc
	iocore/net/quic/QUICLossDetector.h
	iocore/net/quic/QUICNewRenoCongestionController.cc
	iocore/net/quic/QUICPacket.cc
	iocore/net/quic/QUICPacket.h
	iocore/net/quic/QUICPacketFactory.cc
	iocore/net/quic/QUICPacketFactory.h
	iocore/net/quic/QUICRetryIntegrityTag.cc
	iocore/net/quic/QUICRetryIntegrityTag.h
	iocore/net/quic/QUICTLS.h
	iocore/net/quic/QUICTLS_boringssl.cc
	iocore/net/quic/QUICTLS_openssl.cc
	iocore/net/quic/QUICTransportParameters.cc
	iocore/net/quic/QUICTransportParameters.h
	iocore/net/quic/QUICTypes.cc
	iocore/net/quic/QUICTypes.h
	iocore/net/quic/qlog/QLogListener.h
	iocore/net/quic/test/test_QUICHandshakeProtocol.cc
	iocore/net/quic/test/test_QUICLossDetector.cc
	iocore/net/quic/test/test_QUICPacket.cc
	iocore/net/quic/test/test_QUICPacketFactory.cc
	iocore/net/quic/test/test_QUICPacketHeaderProtector.cc
	iocore/net/quic/test/test_QUICStreamManager.cc
	iocore/net/quic/test/test_QUICVersionNegotiator.cc
	proxy/http/HttpProxyServerMain.cc
	src/traffic_quic/quic_client.cc
	src/tscore/ink_inet.cc
  • Loading branch information
maskit committed Jul 27, 2020
2 parents 83e1da7 + 549c626 commit f6e1744
Show file tree
Hide file tree
Showing 94 changed files with 2,428 additions and 1,322 deletions.
22 changes: 20 additions & 2 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,24 @@ notifications:
issues: issues@trafficserver.apache.org
pullrequests: github@trafficserver.apache.org
github:
description: "Apache Traffic Server™ is a fast, scalable and extensible HTTP/1.1 and HTTP/2 compliant caching proxy server."
homepage: https://trafficserver.apache.org/
labels:
- proxy
- cdn
- cache
- apache
features:
# Enable wiki for documentation
wiki: false
# Enable issue management
issues: true
# Enable projects for project management boards
projects: true
enabled_merge_buttons:
# enable squash button:
squash: true
# Enable squash button:
squash: true
# Disable rebase button:
rebase: false
# Disable merge button:
merge: false
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ rc/trafficserver.service

.svn/
.vscode/
target

tsxs

Expand Down
43 changes: 43 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "(gdb) traffic_server",
"type": "cppdbg",
"request": "launch",
"program": "${workspaceFolder}/src/traffic_server/.libs/traffic_server",
"args": [],
"stopAtEntry": false,
"cwd": "${workspaceFolder}",
"environment": [],
"externalConsole": false,
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
}
]
},
{
"name": "(gdb) traffic_manager",
"type": "cppdbg",
"request": "launch",
"program": "${workspaceFolder}/src/traffic_manager/.libs/traffic_manager",
"args": [],
"stopAtEntry": false,
"cwd": "${workspaceFolder}",
"environment": [],
"externalConsole": false,
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
}
]
}
]
}
64 changes: 64 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Autoreconf",
"type": "shell",
"command": "autoreconf -if",
"problemMatcher": []
},
{
"label": "Configure",
"type": "shell",
"command": "./configure --prefix=${workspaceFolder}/target --enable-ccache --enable-experimental-plugins --enable-example-plugins --enable-test-tools --enable-debug --enable-werror ${env:ATS_VSCODE_CONFIGURE}",
"dependsOrder": "sequence",
"dependsOn": ["Autoreconf"],
"problemMatcher": []
},
{
"label": "Build",
"type": "shell",
"command": "make -j 16",
"problemMatcher": [
"$gcc"
],
"group": {
"kind": "build",
"isDefault": true
}
},
{
"label": "Install",
"type": "shell",
"command": "make -j 16 install",
"problemMatcher": [
"$gcc"
]
},
{
"label": "Test",
"type": "shell",
"command": "make -j 8 test",
"problemMatcher": [
"$gcc"
],
"group": {
"kind": "test",
"isDefault": true
}
},
{
"label": "Full Build",
"dependsOrder": "sequence",
"dependsOn": ["Configure", "Build", "Install"],
"problemMatcher": [
"$gcc"
]
}
{
"label": "Dump Enviroment Variables",
"command": "env",
"problemMatcher": []
}
]
}
2 changes: 1 addition & 1 deletion configs/records.config.default.in
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# https://docs.trafficserver.apache.org/records.config#thread-variables
##############################################################################
CONFIG proxy.config.exec_thread.autoconfig INT 1
CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.5
CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.0
CONFIG proxy.config.exec_thread.limit INT 2
CONFIG proxy.config.accept_threads INT 1
CONFIG proxy.config.task_threads INT 2
Expand Down
20 changes: 2 additions & 18 deletions doc/admin-guide/files/records.config.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -282,11 +282,11 @@ Thread Variables
When enabled (the default, ``1``), |TS| scales threads according to the
available CPU cores. See the config option below.

.. ts:cv:: CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.5
.. ts:cv:: CONFIG proxy.config.exec_thread.autoconfig.scale FLOAT 1.0
Factor by which |TS| scales the number of threads. The multiplier is usually
the number of available CPU cores. By default this is scaling factor is
``1.5``.
``1.0``.

.. ts:cv:: CONFIG proxy.config.exec_thread.limit INT 2
Expand Down Expand Up @@ -3561,22 +3561,6 @@ Client-Related Configuration
:code:`ALL`
Check both the signature and the name.

.. ts:cv:: CONFIG proxy.config.ssl.client.verify.server INT 0
:reloadable:
:deprecated:

This setting has been deprecated and :ts:cv:`proxy.config.ssl.client.verify.server.policy` and
:ts:cv:`proxy.config.ssl.client.verify.server.properties` should be used instead.

Configures |TS| to verify the origin server certificate
with the Certificate Authority (CA). This configuration takes a value between 0 to 2.

You can override this global setting on a per domain basis in the :file:`sni.yaml` file using the :ref:`verify_origin_server attribute<override-verify-origin-server>`.

:0: Server Certificate will not be verified
:1: Certificate will be verified and the connection will not be established if verification fail
:2: The provided certificate will be verified and the connection will be established

.. ts:cv:: CONFIG proxy.config.ssl.client.cert.filename STRING NULL
:reloadable:
:overridable:
Expand Down
5 changes: 0 additions & 5 deletions doc/admin-guide/files/sni.yaml.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ Each table is a set of key / value pairs that create a configuration item. This
wildcard entries. To apply an SNI based setting on all the server names with a common upper level domain name,
the user needs to enter the fqdn in the configuration with a ``*.`` followed by the common domain name. (``*.yahoo.com`` for example).

.. _override-verify-origin-server:
.. _override-verify-server-policy:
.. _override-verify-server-properties:
.. _override-host-sni-policy:
Expand All @@ -67,10 +66,6 @@ verify_server_properties One of the values :code:`NONE`, :code:`SIGNATURE`, :co
By default this is :ts:cv:`proxy.config.ssl.client.verify.server.properties`.
This controls what Traffic Server checks when evaluating the origin certificate.

verify_origin_server Deprecated. Use verify_server_policy and verify_server_properties instead.
One of the values :code:`NONE`, :code:`MODERATE`, or :code:`STRICT`.
By default this is :ts:cv:`proxy.config.ssl.client.verify.server`.

verify_client One of the values :code:`NONE`, :code:`MODERATE`, or :code:`STRICT`.
If ``NONE`` is specified, |TS| requests no certificate. If ``MODERATE`` is specified
|TS| will verify a certificate that is presented by the client, but it will not
Expand Down
27 changes: 27 additions & 0 deletions doc/admin-guide/files/volume.config.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,33 @@ sits in front of a volume. This may be desirable if you are using something lik
ramdisks, to avoid wasting RAM and cpu time on double caching objects.


Exclusive spans and volume sizes
================================

In the following sample configuration 2 spans `/dev/disk1` and `/dev/disk2` are defined
in :file:`storage.config`, where span `/dev/disk2` is assigned to `volume 3` exclusively
(`volume 3` is forced to an "exclusive" span `/dev/disk2`).
In :file:`volume.config` there are 3 volumes defined, where `volume 1` and `volume 2`
occupy span `/dev/disk1` taking each 50% of its space and `volume 3` takes 100% of span
`/dev/disk2` exclusively.

storage.config::

/dev/disk1
/dev/disk2 volume=3 # <- exclusinve span

volume.config::

volume=1 scheme=http size=50%
volume=2 scheme=http size=50%
volume=3 scheme=http size=512 # <- volume forced to a specific exclusive span

It is important to note that when percentages are used to specify volume sizes
and "exclusive" spans are assigned (forced) to a particular volume (in this case `volume 3`),
the "exclusive" spans (in this case `/dev/disk2`) are excluded from the total cache
space when the "non-forced" volumes sizes are calculated (in this case `volume 1` and `volume 2`).


Examples
========

Expand Down
2 changes: 1 addition & 1 deletion doc/admin-guide/performance/index.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ CPU and Thread Optimization
Thread Scaling
~~~~~~~~~~~~~~

By default, |TS| creates 1.5 threads per CPU core on the host system. This may
By default, |TS| creates one thread per CPU core on the host system. This may
be adjusted with the following settings in :file:`records.config`:

* :ts:cv:`proxy.config.exec_thread.autoconfig`
Expand Down
1 change: 1 addition & 0 deletions doc/admin-guide/plugins/index.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ directory of the |TS| source tree. Experimental plugins can be compiled by passi
JA3 Fingerprint <ja3_fingerprint.en>
Maxmind ACL <maxmind_acl.en>
Memcache <memcache.en>
Memory Profile <memory_profile.en>
Metalink <metalink.en>
Money Trace <money_trace.en>
MP4 <mp4.en>
Expand Down
106 changes: 106 additions & 0 deletions doc/admin-guide/plugins/memory_profile.en.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
Memory_profile Plugin
*********************

.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
This plugin listens for plugin msgs and invokes jemalloc control
operations.

Installation
============

Add the following line to :file:`plugin.config`::

memory_profile.so

In addition, |TS| must be able to read jemalloc configuration
information either through the ``JEMALLOC_CONF`` environment variable
or via the string sym linked to ``/etc/malloc.conf``.

For example, if the string below is in ``JEMALLOC_CONF`` or in the sym link string, it
enables profiling and indicates that the memory dump prefix is ``/tmp/jeprof``.::

prof:true,prof_prefix:/tmp/jeprof

Details on configuration jemalloc options at `<http://jemalloc.net/jemalloc.3.html>`.
Changes to the configuration in ``JEMALLOC_CONF`` or ``/etc/malloc.conf`` require a process
restart to pick up.

Plugin Messages
===============

The plugin responds to the following mesages sent via traffic_ctl.

Message Action
========== ===================================================================================
activate Start jemalloc profiling. Useful if prof_active:false was in the configure string.

deactivate Stop jemalloc profiling.

dump If profiling is enabled and active, it will generate a profile dump file.

stats Print jemalloc statistics in traffic.out

The command below sends the stats message to the plugin causing the current statistics to be written to traffic.out::

traffic_ctl plugin msg memory_profile stats

Example Usage
=============

If your run time configuration string is::

prof:true,prof_prefix:/tmp/jeprof:prof_active:false

|TS| has started without profile sampling started. Perhaps you didn't want to profile the start up phase of |TS|. To start
you need to send the activate message to the plugin::

traffic_ctl plugin msg memory_profile activate

If your run time configuration string does not indicate that the profiling is not started (e.g. the prof_active field is missing or set to true), you do not
need to send the activate message.

After waiting sometime for |TS| to gather some memory allocation data, you can send the dump message::

traffic_ctl plugin msg memory_profile dump

This will cause a file containing information about the current state of the |TS| memory allocation to be dumped in a file prefixed
by the value of prof_prefix. In this example, it would be something like ``/tmp/jeprof.1234.0.m0.heap``, where 1234 is the process id
and 0 is a running counter indicating how many dumps have been performed on this process. Each dump is independent of the others
and records the current stat of allocations since the profiling was activated. The dump file can be processed by jeprof
to get text output or graphs. Details of how to use jeprof are in the man pages or `<https://manpages.debian.org/unstable/libjemalloc-dev/jeprof.1.en.html>`.

You may want to send the dump message periodically to analyze how the |TS| memory allocation changes over time. This periodic dump can also be achieved by setting the
``lg_prof_interval`` option in the run time configuration string.

If the profiling is taking a significating amount of processing time and affecting |TS| performance, send the deactivate message to turn off profiling.::

traffic_ctl plugin msg memory_profile deactivate

Send the stats message to cause detailed jemalloc stats to be printed in traffic.out. These stats represent actvitiy since the start of the |TS| process.::

traffic_ctl plugin msg memory_profile stats

Limitations
===========

Currently the plugin only functions for systems compiled against jemalloc.
Perhaps in the future, it can be augmented to interact with other memory
allocation systems.

Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ Synopsis
#include <ts/ts.h>
.. function:: TSReturnCode TSHttpTxnClientProtocolStackGet(TSHttpTxn txnp, int n, char const** result, int* actual)
.. function:: TSReturnCode TSHttpTxnClientProtocolStackGet(TSHttpTxn txnp, int count, char const** result, int* actual)

.. function:: TSReturnCode TSHttpTxnServerProtocolStackGet(TSHttpTxn txnp, int n, const char** result, int* actual)
.. function:: TSReturnCode TSHttpTxnServerProtocolStackGet(TSHttpTxn txnp, int count, const char** result, int* actual)

.. function:: TSReturnCode TSHttpSsnClientProtocolStackGet(TSHttpSsn ssnp, int n, char const** result, int* actual)
.. function:: TSReturnCode TSHttpSsnClientProtocolStackGet(TSHttpSsn ssnp, int count, char const** result, int* actual)

.. function:: char const* TSHttpTxnClientProtocolStackContains(TSHttpTxn txnp)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,6 @@ TSOverridableConfigKey Value Configuratio
:c:macro:`TS_CONFIG_SRV_ENABLED` :ts:cv:`proxy.config.srv_enabled`
:c:macro:`TS_CONFIG_SSL_CLIENT_CERT_FILENAME` :ts:cv:`proxy.config.ssl.client.cert.filename`
:c:macro:`TS_CONFIG_SSL_CERT_FILEPATH` :ts:cv:`proxy.config.ssl.client.cert.path`
:c:macro:`TS_CONFIG_SSL_CLIENT_VERIFY_SERVER` :ts:cv:`proxy.config.ssl.client.verify.server`
:c:macro:`TS_CONFIG_SSL_CLIENT_VERIFY_SERVER_PROPERTIES` :ts:cv:`proxy.config.ssl.client.verify.server.properties`
:c:macro:`TS_CONFIG_SSL_CLIENT_VERIFY_SERVER_POLICY` :ts:cv:`proxy.config.ssl.client.verify.server.policy`
:c:macro:`TS_CONFIG_SSL_CLIENT_SNI_POLICY` :ts:cv:`proxy.config.ssl.client.sni_policy`
Expand All @@ -181,8 +180,6 @@ TSOverridableConfigKey Value Configuratio
:c:macro:`TS_CONFIG_URL_REMAP_PRISTINE_HOST_HDR` :ts:cv:`proxy.config.url_remap.pristine_host_hdr`
:c:macro:`TS_CONFIG_WEBSOCKET_ACTIVE_TIMEOUT` :ts:cv:`proxy.config.websocket.active_timeout`
:c:macro:`TS_CONFIG_WEBSOCKET_NO_ACTIVITY_TIMEOUT` :ts:cv:`proxy.config.websocket.no_activity_timeout`
:c:macro:`TS_CONFIG_SSL_CLIENT_VERIFY_SERVER_POLICY` :ts:cv:`proxy.config.ssl.client.verify.server.policy`
:c:macro:`TS_CONFIG_SSL_CLIENT_VERIFY_SERVER_PROPERTIES` :ts:cv:`proxy.config.ssl.client.verify.server.properties`
:c:macro:`TS_CONFIG_SSL_CLIENT_CERT_FILENAME` :ts:cv:`proxy.config.ssl.client.cert.filename`
:c:macro:`TS_CONFIG_SSL_CLIENT_PRIVATE_KEY_FILENAME` :ts:cv:`proxy.config.ssl.client.private_key.filename`
:c:macro:`TS_CONFIG_SSL_CLIENT_CA_CERT_FILENAME` :ts:cv:`proxy.config.ssl.client.CA.cert.filename`
Expand Down

0 comments on commit f6e1744

Please sign in to comment.