New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-1019: Implemented prior knowledge approach to h2c #3873
Conversation
I’m reluctant to implement support for H2C because it’s a cleartext protocol and cleartext is dead to me. |
@swankjesse While understandable we have a pretty reasonable use case. Transparent TLS via a service mesh. We have clients that use Retrofit and Feign but can't use h2c to talk to localhost without this kind of change. |
Gotcha. So cleartext doesn’t leave the host. That’s not so bad. |
|
||
// this implementation takes the "prior knowledge" approach to the spec | ||
// https://tools.ietf.org/html/rfc7540#section-3.4 | ||
if (route.address().protocols().contains(Protocol.H2C)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest we change
To allow H2C on it's own, but only on it's own, no fallback to HTTP/1.1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. This ended up cleaning up some of the logic in the establishment.
I'm a big fan of this, have hit this a couple of times before during development and ended up using Netty. |
To be clear, my main concern is this makes a client that is single purpose. So we should probably only allow H2C as the single supported protocol for a client. Otherwise this client instance will be mostly usable for https but broken for most external plaintext traffic. It seems better to be really explicit that the client is single purpose. |
// when using h2c prior knowledge, no other protocol should be supported. | ||
throw new IllegalArgumentException("protocols containing h2c cannot use other protocols:" | ||
+ uniqueProtocols); | ||
} else if (!uniqueProtocols.contains(Protocol.HTTP_1_1)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bug: this needs to check if uniqueProtocols does not contain H2c
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is checked implicitly via the if case above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually it's not. In the event that someone specifies only H2C, then the second if check evaluates to false, triggering this else if. Caught it in one of the unit tests I was writing.
* Cleartext implementation of HTTP2. This enumeration exists for the "prior knowledge" upgrade | ||
* semantic supported by the protocol. | ||
*/ | ||
H2C("h2c"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bug: this needs to be added to the static get method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resolved
// Validate that the list has everything we require and nothing we forbid. | ||
if (!protocols.contains(Protocol.HTTP_1_1)) { | ||
throw new IllegalArgumentException("protocols doesn't contain http/1.1: " + protocols); | ||
final Set<Protocol> uniqueProtocols = new HashSet<>(protocols); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@swankjesse has stated a preference for LinkedHashSet makes these a lot more predictable in tests, e.g. checking the exception message. But generally you could probably just check the contains(H2C) and size > 1 on the list and fold in duplicates as bad input. But this isn't super critical code path, so that's minor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. -It wasn't super clear if the list of protocols was treated as a prioritized.- Adding to preserve ordering since it's in the Javadoc that order is expected to be preserved.
|
||
// Assign as an unmodifiable list. This is effectively immutable. | ||
this.protocols = Collections.unmodifiableList(protocols); | ||
this.protocols = Collections.unmodifiableList(new ArrayList<>(uniqueProtocols)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think changing the selected protocol because of hashing could affect the ALPN selection IIRC. Not so much an issue without SPDY, but undesirable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah. I think that change went a little too far. Going to revert back to using just the list and suffer the minimal performance cost.
… using prior knowledge
…ted javadoc to represent handling of h2c.
Do y'all prefer rebase or merge when conflicts arise? I didn't see a CONTRIBUTING guide anywhere. I tend to do merges, but don't have a preference for either way. |
We'll squash when merging a multi-commit PR so it's up to you.
…On Thu, Feb 22, 2018, 9:10 PM Jaye Pitzeruse ***@***.***> wrote:
Do y'all prefer rebase or merge when conflicts arise? I didn't see a
CONTRIBUTING guide anywhere.
I tend to do merges, but don't have a preference for either way.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#3873 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEEEf7jKRnaXKWbHhlVxRfr2cyBB9ogks5tXh4AgaJpZM4SMzgI>
.
|
…on and added a concrete check on the protocols list.
Working checkpoint. I tested this against a service I have that supports H2C out of box. Started to make modifications to the MockWebServer, but that might be a bit too ambitious. I tried putting in a shortcut, but that didn't seem to work. I'll try a few more things over the next couple of days |
Alright, I managed to get the MockWebServer working and pulled the test cases from HttpOverHttp2Test up to an Http2TestBase that can be shared by both the cleartext and TLS flavors. Added to this review because creating another one would effectively re-diff this entire set. |
/** | ||
* Test H2C, the cleartext implementation of HTTP/2. | ||
* | ||
* @author jpitz |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
strip author tags
@@ -124,6 +124,9 @@ public Http2Codec(OkHttpClient client, Interceptor.Chain chain, StreamAllocation | |||
@Override public Response.Builder readResponseHeaders(boolean expectContinue) throws IOException { | |||
List<Header> headers = stream.takeResponseHeaders(); | |||
Response.Builder responseBuilder = readHttp2HeadersList(headers); | |||
if (client.protocols().contains(Protocol.H2C)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why set it here and not inside readHttp2HeadersList?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That method was public static
and so I would need to thread the value through the method signature, causing a breaking API change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could actually be cached on the class. No need to do the linear scan all the time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternatively, another attribute could be added to the OkHttpClient and that can be a cached referenced for the lifetime of the client
@@ -260,9 +260,14 @@ private void connectSocket(int connectTimeout, int readTimeout, Call call, | |||
|
|||
private void establishProtocol(ConnectionSpecSelector connectionSpecSelector, | |||
int pingIntervalMillis, Call call, EventListener eventListener) throws IOException { | |||
if (route.address().sslSocketFactory() == null) { | |||
protocol = Protocol.HTTP_1_1; | |||
if (route.address().protocols().contains(Protocol.H2C)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like this check could be a subset of the "if (route.address().sslSocketFactory() == null) " case
meaning 0 cost in the more common https case, and clearer that this is a special case of clear text. Thoughts?
@@ -844,15 +844,25 @@ public Builder dispatcher(Dispatcher dispatcher) { | |||
* HTTP/1.1} only. If the server responds with {@code HTTP/1.0}, that will be exposed by {@link | |||
* Response#protocol()}. | |||
* | |||
* @param protocols the protocols to use, in order of preference. The list must contain {@link | |||
* Protocol#HTTP_1_1}. It must not contain null or {@link Protocol#HTTP_1_0}. | |||
* @param protocols the protocols to use, in order of preference. If the list contains {@link |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we reword this to start with the normal case of HTTP_1_1. As it is has a very H2C centric view of the world.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll take another crack. I started trying to word it that way, but ran into issues with clarity.
* Base test cases for HTTP2 communication. These tests are executed for both H2C and HTTP2 | ||
* based requests. For implementation specific tests, include in the appropriate test case. | ||
* | ||
* This code originally lived in the HttpOverHttp2Test class, but was abstracted out for use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this history is needed. We have git for that.
} | ||
|
||
@Test | ||
public void testProtocol() throws Exception { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels like these tests from here onwards would be more natural in ProtocolTest (new), MockWebserverTest and OkHttpClientTest. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. I looked for a ProtocolTest and there wasn't one. Didn't look again after handling the merge. I'll move them.
|
||
@Before | ||
public void setUp() throws Exception { | ||
super.setUp( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels like this could be handled without as much major surgery with a @parameterized test. Not sure which is generally favoured, but I try to avoid subclassing in case there is another dimension of freedom to the tests in the future e.g. Platform.
But interested to hear input from others and yours.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It also means you can't go to a single test and run it from within Http2TestBase.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huh.. This week I learned. Didn't know about Parameterized builds in JUnit. This is exactly what I am looking for.
…emaining test methods into appropriate classes
@yschimke : master doesn't appear to have a ProtocolTest yet. Where is it? |
@Parameters | ||
public static Collection<Object[]> data() { | ||
|
||
return Arrays.asList(new Object[][] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indentation looks off
Looking good to me, only nits and formatting. |
@@ -456,4 +457,22 @@ | |||
RecordedRequest request = server.takeRequest(); | |||
assertEquals("request", request.getBody().readUtf8()); | |||
} | |||
|
|||
@Test(expected = IllegalArgumentException.class) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as previous comment, OkHttp tests favour explicit catch and check exception message over annotation expected tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Care to update the description, remove "quick stab", before we merge ?
@yschimke : done. any idea what's going on with the travis test? I'd prefer to have a green build before pushing forward |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great.
@@ -420,7 +428,7 @@ public void processConnection() throws Exception { | |||
return; | |||
} | |||
socket = sslSocketFactory.createSocket(raw, raw.getInetAddress().getHostAddress(), | |||
raw.getPort(), true); | |||
raw.getPort(), true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: indent
@@ -436,6 +444,10 @@ public void processConnection() throws Exception { | |||
protocol = protocolString != null ? Protocol.get(protocolString) : Protocol.HTTP_1_1; | |||
} | |||
openClientSockets.remove(raw); | |||
} else if (protocols.contains(Protocol.H2C)) { | |||
// force use of H2c |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I don’t think the comment is necessary!
} | ||
} | ||
|
||
@Test public void testH2COkHttpClientConstructionSuccess() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice.
} | ||
|
||
@Test public void testH2COkHttpClientConstructionSuccess() { | ||
final OkHttpClient okHttpClient = new OkHttpClient.Builder() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: final
isn’t necessary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
habbit. we prefer having every thing be final.
@@ -0,0 +1,34 @@ | |||
package okhttp3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: copyright header
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
public final class HttpOverHttp2Test { | ||
private static final Logger http2Logger = Logger.getLogger(Http2.class.getName()); | ||
private static final SslClient sslClient = SslClient.localhost(); | ||
|
||
@Parameters |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
neat
@@ -1147,6 +1183,8 @@ private int countFrames(List<String> logs, String message) { | |||
* <p>This test uses proxy tunnels to get a hook while a connection is being established. | |||
*/ | |||
@Test public void concurrentHttp2ConnectionsDeduplicated() throws Exception { | |||
if (protocol == Protocol.H2C) return; // only run for http2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, I think you want assumeTrue() here
@@ -261,8 +261,15 @@ private void connectSocket(int connectTimeout, int readTimeout, Call call, | |||
private void establishProtocol(ConnectionSpecSelector connectionSpecSelector, | |||
int pingIntervalMillis, Call call, EventListener eventListener) throws IOException { | |||
if (route.address().sslSocketFactory() == null) { | |||
protocol = Protocol.HTTP_1_1; | |||
if (route.address().protocols().contains(Protocol.H2C)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice
private void startHttp2(int pingIntervalMillis) throws IOException { | ||
socket.setSoTimeout(0); // HTTP/2 connection timeouts are set per-stream. | ||
http2Connection = new Http2Connection.Builder(true) | ||
.socket(socket, route.address().url().host(), source, sink) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
supernit: +4 on wrapped lines
|
||
public Http2Codec(OkHttpClient client, Interceptor.Chain chain, StreamAllocation streamAllocation, | ||
Http2Connection connection) { | ||
this.client = client; | ||
this.chain = chain; | ||
this.streamAllocation = streamAllocation; | ||
this.connection = connection; | ||
|
||
// cache this so we don't do linear scans on every call |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this comment is misleading. “linear scans” is on a list of a very small constant size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I think I’d just omit the comment)
public final class HttpOverHttp2Test { | ||
private static final Logger http2Logger = Logger.getLogger(Http2.class.getName()); | ||
private static final SslClient sslClient = SslClient.localhost(); | ||
|
||
@Parameters |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running fine locally in both IDE and "mvn test", so the JVM died during HttpOverHttp2 is baffling
I'll dig into why it's failing, particularly as I pushed you towards the parameterised test, which is a likely suspect. |
Failing with
|
It's something wacky with reuse of clients and mockwebserver instances and/or Http2Connections (these are used separately in both client and MockWebServer). Anyway, the shutdown of pushExecutor causes uncaught exceptions which we barf on. I suspect a convergence of
|
… back to using assumeTrue
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we will need to resolve the problems before landing this
private static final SslClient sslClient = SslClient.localhost(); | ||
|
||
@Parameters(name = "{2}") | ||
public static Collection<Object[]> data() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes the client instances used across test runs. Can we change this to
@Parameters(name = "{0}")
public static Collection<Protocol> data() {
return Arrays.asList(Protocol.H2C, Protocol.HTTP_2);
}
private String scheme; | ||
private Protocol protocol; | ||
|
||
public HttpOverHttp2Test(OkHttpClient client, String scheme, Protocol protocol) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public HttpOverHttp2Test(Protocol protocol) {
this.protocol = protocol;
this.client = protocol == Protocol.HTTP_2 ? buildClient() : buildH2cClient();
this.scheme = protocol == Protocol.HTTP_2 ? "https" : "http";
}
private static OkHttpClient buildH2cClient() {
return defaultClient().newBuilder()
.connectionPool(new ConnectionPool())
.protocols(Arrays.asList(Protocol.H2C))
.build();
}
private static OkHttpClient buildClient() {
return defaultClient().newBuilder()
.protocols(Arrays.asList(Protocol.HTTP_2, Protocol.HTTP_1_1))
.dns(new SingleInetAddressDns())
.sslSocketFactory(sslClient.socketFactory, sslClient.trustManager)
.hostnameVerifier(new RecordingHostnameVerifier())
.connectionPool(new ConnectionPool())
.build();
}
then in tear down
@After public void tearDown() throws Exception {
Authenticator.setDefault(null);
http2Logger.removeHandler(http2Handler);
http2Logger.setLevel(previousLevel);
client.connectionPool().evictAll();
}
… connections from the pool in teardown
@@ -223,6 +227,10 @@ public void setProtocols(List<Protocol> protocols) { | |||
this.protocols = protocols; | |||
} | |||
|
|||
public List<Protocol> protocols() { | |||
return Collections.unmodifiableList(protocols); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
when we set the protocols we call Util.immutableList()
so I think unmodifiableList() is not necessary
new OkHttpClient.Builder() | ||
.protocols(Arrays.asList(Protocol.H2C, Protocol.HTTP_1_1)); | ||
fail("When H2C is specified, no other protocol can be specified"); | ||
} catch (final IllegalArgumentException e) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: more finals
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also I really like expected
instead of e
for these
This push failure happened on master https://api.travis-ci.org/v3/job/346125271/log.txt So I suspect it is an existing problem, unrelated to this test now. The reused static clients you had in an earlier version would have made it a lot more noticeable. |
Alright, I believe all feedback has been address at this point. Let me know if theres anything else y'all need from me 😄 |
Thanks @jpitz. Very nice first PR! |
No problem. Happy to help! Let me know if any issues arise and would like me to assist.
… On Feb 26, 2018, at 10:05 PM, Jesse Wilson ***@***.***> wrote:
Thanks @jpitz. Very nice first PR!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
Generates following table with PR count. ``` ┌──────────────┬──────────────────────────────────────────────────────┐ │ Title │ gh-1019: Implemented prior knowledge approach to h2c │ ├──────────────┼──────────────────────────────────────────────────────┤ │ PR Author │ mjpitz │ ├──────────────┼──────────────────────────────────────────────────────┤ │ URL │ square/okhttp#3873 │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Ready On │ Feb 20, 2018, 5:55:35 PM │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Review Time │ yschimke=1d 2h 45m │ │ ├──────────────────────────────────────────────────────┤ │ │ swankjesse=1d 2h 45m │ ├──────────────┼──────────────────────────────────────────────────────┤ │ PR Comments │ swankjesse=3 │ │ ├──────────────────────────────────────────────────────┤ │ │ jjshanks=1 │ │ ├──────────────────────────────────────────────────────┤ │ │ yschimke=9 │ │ ├──────────────────────────────────────────────────────┤ │ │ mjpitz=10 │ │ ├──────────────────────────────────────────────────────┤ │ │ JakeWharton=1 │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Merged On │ Feb 26, 2018, 6:55:11 PM │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Open → Merge │ 6d 0h 59m 36s │ └──────────────┴──────────────────────────────────────────────────────┘ ```
Closes #155 Closes #161 Generates following table: ``` ┌──────────────┬──────────────────────────────────────────────────────┐ │ Title │ gh-1019: Implemented prior knowledge approach to h2c │ ├──────────────┼──────────────────────────────────────────────────────┤ │ PR Author │ mjpitz │ ├──────────────┼──────────────────────────────────────────────────────┤ │ URL │ square/okhttp#3873 │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Ready On │ Feb 20, 2018, 5:55:35 PM │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Review Time │ yschimke=1d 2h 45m │ │ ├──────────────────────────────────────────────────────┤ │ │ swankjesse=1d 2h 45m │ ├──────────────┼──────────────────────────────────────────────────────┤ │ PR Comments │ swankjesse made total 19 comments. │ │ │ Code Review Comments = 16, Issue Comments = 3 │ │ ├──────────────────────────────────────────────────────┤ │ │ jjshanks made total 1 comments. │ │ │ Code Review Comments = 0, Issue Comments = 1 │ │ ├──────────────────────────────────────────────────────┤ │ │ yschimke made total 30 comments. │ │ │ Code Review Comments = 21, Issue Comments = 9 │ │ ├──────────────────────────────────────────────────────┤ │ │ mjpitz made total 26 comments. │ │ │ Code Review Comments = 16, Issue Comments = 10 │ │ ├──────────────────────────────────────────────────────┤ │ │ JakeWharton made total 1 comments. │ │ │ Code Review Comments = 0, Issue Comments = 1 │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Merged On │ Feb 26, 2018, 6:55:11 PM │ ├──────────────┼──────────────────────────────────────────────────────┤ │ Open → Merge │ 6d 0h 59m 36s │ └──────────────┴──────────────────────────────────────────────────────┘ ```
Needs a unit test, but I wanted to get the idea out there of how easy this could be.
An Address's List comes from the initial configuration. If someone specifies the H2C protocol in that list, then we assume prior knowledge since it requires users to specify the protocol in their configuration.
I tried to wire a boolean attribute through the constructor chain, but that got really gross. This seemed like an easier approach with a lot fewer changes in code.