Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send HTTP Warning Header(s) for any Deprecation Usage from a REST request #17804

Merged

Conversation

Projects
None yet
7 participants
@pickypg
Copy link
Member

commented Apr 16, 2016

This adds support for ThreadContexts to maintain unique Response headers (repeated header values are ignored) in addition to Request headers. In particular, this enables REST requests (e.g., PUT /test) that hop across the nodes to properly return headers as part of the overall request. With the response headers Thread aware thanks to the ThreadContext, we can associate individual requests with deprecation logging and therefore warn developers when they unwittingly use deprecated features:

PUT /test
{
  "mappings": {
    "type": {
      "properties": {
        "field": {
          "type": "string"
        },
        "field2": {
          "type": "string"
        }
      }
    }
  }
}

The headers that would get returned by this would be:

HTTP/1.1 200 OK
Warning: The [string] field is deprecated, please use [text] or [keyword] instead on [field]
Warning: The [string] field is deprecated, please use [text] or [keyword] instead on [field2]
Content-Type: application/json; charset=UTF-8
Content-Length: 21

Alongside that feature, this adds a DeprecationRestHandler that logs a deprecation warning whenever it is used, but it otherwise behaves identically to normal RestHandlers. This allows REST APIs to be deprecated in a consistent manner instead of dropped without any non-documentation-level deprecation warnings in advance.

Closes #17687

@jasontedor

View changes

core/src/main/java/org/elasticsearch/common/Strings.java Outdated
*
* @return {@code true} if the {@code value} is not obviously wrong.
*/
public static boolean validHeaderValue(String value) {

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I don't think this method belongs on Strings, especially since it's only used in only place. I'd be fine with it as a static package-private method on DeprecationRestChannel so it's visible for testing.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 16, 2016

Author Member

I honestly think that it should be used in other places.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

That's probably correct, but right now it's not and I don't think Strings is the right place anyway.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

Moved to DeprecationRestChannel and also added String requireValidHeader(String value) that behaves similar to Objects.requireNonNull to use for all creations. I did leave it as public as it's used by DeprecationRestHandler (even though they're in the same package).

@jasontedor

View changes

core/src/main/java/org/elasticsearch/common/settings/ClusterSettings.java Outdated
IndexingMemoryController.MIN_INDEX_BUFFER_SIZE_SETTING,
IndexingMemoryController.MAX_INDEX_BUFFER_SIZE_SETTING,
IndexingMemoryController.SHARD_INACTIVE_TIME_SETTING,
IndexingMemoryController.SHARD_MEMORY_INTERVAL_TIME_SETTING

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I'm confused by these changes? What's happening here?

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 16, 2016

Author Member

It was a local rebase gone crazy. Those weren't from here.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/common/settings/Setting.java Outdated
Property... properties) {
return byteSizeSetting(key, (s) -> value.toString(), minValue, maxValue, properties);
return byteSizeSetting(key, (s) -> defaultValue.toString(), minValue, maxValue, properties);

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

These too, they look like they are from #17778 which I reviewed earlier today?

@pickypg pickypg force-pushed the pickypg:feature/deprecated-rest-handler-17687 branch Apr 16, 2016

@jasontedor

View changes

core/src/main/java/org/elasticsearch/index/shard/IndexShard.java Outdated
@@ -1404,7 +1404,7 @@ private final EngineConfig newEngineConfig(EngineConfig.OpenMode openMode, Trans
return new EngineConfig(openMode, shardId,
threadPool, indexSettings, warmer, store, deletionPolicy, indexSettings.getMergePolicy(),
mapperService.indexAnalyzer(), similarityService.similarity(mapperService), codecService, shardEventListener, translogRecoveryPerformer, indexCache.query(), cachingPolicy, translogConfig,
indexSettings.getSettings().getAsTime(IndexingMemoryController.SHARD_INACTIVE_TIME_SETTING, IndexingMemoryController.SHARD_DEFAULT_INACTIVE_TIME));
IndexingMemoryController.SHARD_INACTIVE_TIME_SETTING.get(indexSettings.getSettings()));

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

This too is confusing.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/indices/IndexingMemoryController.java Outdated
@@ -50,22 +52,27 @@
public class IndexingMemoryController extends AbstractComponent implements IndexingOperationListener, Closeable {

/** How much heap (% or bytes) we will share across all actively indexing shards on this node (default: 10%). */
public static final String INDEX_BUFFER_SIZE_SETTING = "indices.memory.index_buffer_size";
public static final Setting<ByteSizeValue> INDEX_BUFFER_SIZE_SETTING = Setting.byteSizeSetting("indices.memory.index_buffer_size", "10%", Property.NodeScope);

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

More that shouldn't be here.

@jasontedor

View changes

core/src/test/java/org/elasticsearch/indices/IndexingMemoryControllerTests.java Outdated
@@ -76,7 +74,7 @@

public MockController(Settings settings) {
super(Settings.builder()
.put(SHARD_MEMORY_INTERVAL_TIME_SETTING, "200h") // disable it
.put("indices.memory.interval", "200h") // disable it

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

More from #17778.

@jasontedor

View changes

core/src/test/java/org/elasticsearch/rest/DeprecationRestHandlerTests.java Outdated
*/
public class DeprecationRestHandlerTests extends ESTestCase {
@Rule
public ExpectedException expectedException = ExpectedException.none();

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

We prefer using expectThrows instead of this JUnit rule. Can you convert to using that pattern?

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 16, 2016

Author Member

Of course. Noticed this one first :)

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Yeah, it's relatively new but it's the clear path forward especially with JUnit 5 coming with built-in support for the same.

@jasontedor

View changes

core/src/test/java/org/elasticsearch/rest/RestControllerTests.java Outdated

public class RestControllerTests extends ESTestCase {

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I think we prefer the extra blank line between the definition of the class and the body of the class (and between the body of the class and the closing brace for the class). No, this isn't held to perfectly, but you'll see it across the codebase.

This comment applies to quite a few places throughout this PR, I'll just let you find them and we'll catch them in final review?

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestChannel.java Outdated
public static final String DEPRECATION_HEADER = "Warning";

/**
* The proxied canchannel.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I think these comments can be removed.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestChannel.java Outdated
*/
private final RestChannel channel;
/**
* The deprecation message.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I think these comments can be removed.

This comment has been minimized.

Copy link
@nik9000

nik9000 Apr 18, 2016

Contributor

If you make the variable's name deprecationMessage then, sure.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

I prefer that name. I think I got the "Warning" header name stuck in my head early on. Renamed.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestChannel.java Outdated
}

/**
* {@inheritDoc}

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

If you do not provide any Javadoc, inheritdoc is implied.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

I am aware of its implicit nature, but explicit documentation shows intent while implicit documentation can mean anything. I would strongly prefer to keep these here even though I know that it's implicit.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 18, 2016

Member

I am aware of its implicit nature, but explicit documentation shows intent while implicit documentation can mean anything.

I'm not sure what you mean by this? The Javadoc for this method will have exactly the same documentation whether or not the inheritdoc tag is there.

I would strongly prefer to keep these here even though I know that it's implicit.

It just adds source code clutter; I think that it should be removed. This is also inconsistent with how we do intend to do it everywhere else in the codebase (is it 100% perfect? probably not

This comment has been minimized.

Copy link
@nik9000

nik9000 Apr 18, 2016

Contributor

It just adds source code clutter; I think that it should be removed.

+1

@Override is sufficient to say "I'm doing what the interface says." The biggest problem with this is that you have to jump to the overridden method to check its docs. inheritdoc doesn't solve that either.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

I'm not sure what you mean by this? The Javadoc for this method will have exactly the same documentation whether or not the inheritdoc tag is there.

It's not what appears in the theoretical javadoc jar, rather it's what it says in the code by having it there. Not having it there says one of two things:

  1. I quickly wrote this code
  2. I accept the same contract as the parent

This is also inconsistent with how we do intend

I've never really taken it as intent versus just not happening (noting that the parent has zero javadoc except on one method). If we are actually intentionally not javadocing things, then I think that's both wrong and dangerous, but I will conform for inheritdoc.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 18, 2016

Member

It's not what appears in the theoretical javadoc jar, rather it's what it says in the code by having it there. Not having it there says one of two things:

  1. I quickly wrote this code
  2. I accept the same contract as the parent

If we're writing and reviewing code carefully, it only means the latter because those are the very straightforward rules of Javadocs.

If we are actually intentionally not javadocing things, then I think that's both wrong and dangerous

I completely disagree. Explicit is good when things are confusing, but not when things have straightforward rules.

For example, I note that you didn't use explicitly this in all cases when accessing a member variable. Why not? Probably because in those cases when it's not needed (i.e., there is no other variable that is shadowing the member variable), it is straightforward to understand.

, but I will conform for inheritdoc.

There is no "conforming". It's better than we reach an agreement on this.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

If we're writing and reviewing code carefully
It's better than we reach an understanding on this.

I disagree with the notion that the reviews make it acceptable to drop it because only the submitter and reviewer(s) are aware of what happened, the reasoning, and any assumptions that goes into the changes. Particularly because of Guice, it's extremely annoying to jump through the code with undocumented-after-undocumented thing, having to dig into the weeds to find things that should be pretty simple. (This is more in regard to our general disregard for Javadoc, not inheritdoc)

I rarely document things for myself because I inherently know what I did (I'd hope, right? :)); it's intended for someone down the road that is not coming in with my biases and assumptions, or a total awareness of the entire codebase. In the case of inheritdoc, I agree it's much looser than full documentation, but the explicit nature of it is far different as a reader to show that it's not supposed to change from parent behavior, especially in the context of the rest of our codebase. Furthermore, it suggests to any future reviewer to have someone changing the method to document their changes rather than just keep that as-is in a review somewhere. I got this way because I've inherited too many bad codebases (not suggesting that's what we have by any stretch) where they didn't document anything and the code was bad/broken, which led to all sorts of pain whenever anything needed to be changed by a new member of the team. Applying inheritdoc also starts a good practice of actually adding Javadoc.

For me, it's very different from using/not using syntax like this because that requires you to read the code either way. Documentation is supposed to help to give context about the contract without reading the raw implementation (e.g., side-effects, can it ever return null, when does it throw exceptions, etc.). It's like looking up JDK classes without looking up their implementation. For very advanced stuff, I certainly jump around the implementation, but for simple things it's nice to always know about explicit intent. I think our own inherent disagreement on when to use it shows that the implicit nature leads to different assumptions.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 18, 2016

Member

I disagree with the notion that the reviews make it acceptable to drop it because only the submitter and reviewer(s) are aware of what happened, the reasoning, and any assumptions that goes into the changes.

Your claim was that not having it there means one of two things: quickly written code, or we accept the same contract as the parent. I'm saying if we are writing and reviewing code carefully (which we are, right?), then it can not be the case that it's there because the code was written quickly. At least one of the writer and the reviewer (preferably both) have carefully considered the issue at hand.

Particularly because of Guice, it's extremely annoying to jump through the code with undocumented-after-undocumented thing, having to dig into the weeds to find things that should be pretty simple.

This has nothing to do with whether or not inheritdoc is present.

This is more in regard to our general disregard for Javadoc, not inheritdoc

I don't think we have "disregard" for it so much as we don't do a good job of using it when applicable.

I consider the rules around the inheriting of Javadocs here to be incredibly straightforward that they require no clarification. Do you explicitly cast every int parameter to a long when assigning an int to long? I don't, because the rules are straightforward.

Applying inheritdoc also starts a good practice of actually adding Javadoc.

It's just clutter, and I don't think it does anything to promote adding Javadocs. That is something that is handled by carefully writing code and carefully reviewing code, and saying that our culture is going to be one that drives towards more Javadocs.

Documentation is supposed to help to give context about the contract without reading the raw implementation (e.g., side-effects, can it ever return null, when does it throw exceptions, etc.).

We are not arguing over whether or not documentation should be present, we are arguing about cluttering the code with three lines that add nothing because they are otherwise implied.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

I think we just disagree on the semantic value of it being explicit. Honestly, I still think we should have it, but I don't feel that strongly about it since it is implied. Anything past this will be circular. I'm in the minority opinion and that's okay.

Do you explicitly cast every int parameter to a long when assigning an int to long?

Nope (except in a ByteBuffer situation, but I know you didn't mean those edge cases). But that's an implementation detail rather than a contract. I would even call out such an unnecessary cast in a code review, like you've done here. However, I think the semantics of casting int to long is different (it doesn't just add clutter, it creates questions) from explicitly specifying inheritDoc -- even though both are automatic expansions, doing so with a contract means something to me and at worst clutter. I've removed it since clearly it's not adding value for others.

@jasontedor

View changes

core/src/test/java/org/elasticsearch/common/StringsTests.java Outdated
* Create a generator for characters [32, 126].
*/
public ASCIIHeaderGenerator() {
super(asciiFromTo(32, 126));

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

The indentation is off here.

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

IntelliJ's folding hid this from me. Fixed (and moved).

@jasontedor

View changes

core/src/test/java/org/elasticsearch/common/StringsTests.java Outdated
*/
public ASCIIHeaderGenerator() {
super(asciiFromTo(32, 126));
}

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

The indentation is off here.

@jasontedor

View changes

core/src/test/java/org/elasticsearch/rest/DeprecationRestChannelTests.java Outdated
*/
public class DeprecationRestChannelTests extends ESTestCase {
@Rule
public ExpectedException expectedException = ExpectedException.none();

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Same thing, prefer expectThrows.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestChannel.java Outdated
public DeprecationRestChannel(RestChannel channel, String warning) {
assert Strings.validHeaderValue(warning);

// required

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Unneeded comment, it's clear from the code that they are required.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestChannel.java Outdated
public void sendResponse(RestResponse response) {
response.addHeader(DEPRECATION_HEADER, warning);

// actually send the response

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Unneeded comment.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestHandler.java Outdated
*/
public DeprecationRestHandler(RestHandler handler, String warning, DeprecationLogger deprecationLogger) {
// it had better not be empty or you're deprecating it wrong
assert Strings.hasText(warning);

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Is this right? Shouldn't it be validHeaderValue? And I don't think this should be an assertion, but a hard failure (assertions are disabled in production and if we are sending bad headers something is seriously wrong and we need to die hard).

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 16, 2016

Author Member

I thought about it, but this is just logging. It's not until we get to the channel where it changes purpose and sends a header.

Though I do prefer that it fails fast instead of lazily later.

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

Though I do prefer that it fails fast instead of lazily later.

++

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

Though does this change your opinion of where that utility function lives?

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 18, 2016

Member

No, I still think that it should not be a method on the Strings class.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestHandler.java Outdated
// it had better not be empty or you're deprecating it wrong
assert Strings.hasText(warning);

// required

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

No comment necessary.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/rest/DeprecationRestHandler.java Outdated
*/
@Override
public void handleRequest(RestRequest request, RestChannel channel) throws Exception {
// note the deprecation for ES logs

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

I think this method is clear enough without the comments.

@jasontedor

View changes

core/src/main/java/org/elasticsearch/common/Strings.java Outdated
char c = value.charAt(i);

// 32 = ' ' (31 = unit separator); 126 = '~' (127 = DEL)
// http://www.asciitable.com/ in case you need a reference

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

This comment can be removed, it's not authoritative, it might get broken, and it's easily discovered via search. What really matters is the RFC link which you've provided above already. 😄

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

(Removed link, but method was moved)

@jasontedor

View changes

core/src/main/java/org/elasticsearch/common/Strings.java Outdated
return false;
}

for (int i = 0; i < value.length(); i++) {

This comment has been minimized.

Copy link
@jasontedor

jasontedor Apr 16, 2016

Member

This can probably just be simplified to value.chars().allMatch(i -> i >= 32 && i <= 126); But take caution when backporting to 2.x!

This comment has been minimized.

Copy link
@pickypg

pickypg Apr 18, 2016

Author Member

I think I'll do a second pass and look to make that kind of cleanup apply to all of the Strings functions as a master-only PR afterward?

@jaymode

View changes

core/src/main/java/org/elasticsearch/transport/local/LocalTransportChannel.java Outdated
@@ -104,7 +109,7 @@ private void sendResponseData(byte[] data) {
close();
targetTransport.workers().execute(() -> {
ThreadContext threadContext = targetTransport.threadPool.getThreadContext();
try (ThreadContext.StoredContext ignore = threadContext.stashContext()) {
try (ThreadContext.StoredContext ignore = threadContext.newStoredContext()) {

This comment has been minimized.

Copy link
@jaymode

jaymode Jul 1, 2016

Member

I don't think this should have been changed

This comment has been minimized.

Copy link
@pickypg

pickypg Jul 1, 2016

Author Member

Correct! Reverted that change.

@jaymode

View changes

core/src/test/java/org/elasticsearch/rest/DeprecationHttpIT.java Outdated
/**
* Attempts to do a scatter/gather request that expects unique responses per sub-request.
*/
@AwaitsFix(bugUrl = "")

This comment has been minimized.

Copy link
@jaymode

jaymode Jul 1, 2016

Member

can we add text here/open up a issue?

This comment has been minimized.

Copy link
@pickypg

pickypg Jul 1, 2016

Author Member

Created #19222 (and placed it appropriately)

@jaymode

View changes

core/src/test/java/org/elasticsearch/rest/DeprecationHttpIT.java Outdated
final String quotedCommaSeparatedIndices = Stream.of(indices).map((i) -> "\"" + i + "\"").collect(Collectors.joining(","));
final String body =
"{\"query\":{\"bool\":{\"filter\":[" +
// "{\"indices\":{\"indices\":[" + quotedCommaSeparatedIndices + "],\"query\":{\"" + TestDeprecatedQueryBuilder.NAME + "\":{}}}}," +

This comment has been minimized.

Copy link
@jaymode

jaymode Jul 1, 2016

Member

is this needed?

This comment has been minimized.

Copy link
@pickypg

pickypg Jul 1, 2016

Author Member

No, I was using that to mess with my deprecation warnings. Removed from the committed code.

@pickypg pickypg force-pushed the pickypg:feature/deprecated-rest-handler-17687 branch 2 times, most recently Jul 1, 2016

@pickypg

This comment has been minimized.

Copy link
Member Author

commented Jul 5, 2016

@jaymode Please take a second pass now that I have rebased it back onto master. I think it's ready to merge with all tests passing.

@jaymode

View changes

core/src/main/java/org/elasticsearch/common/logging/DeprecationLogger.java Outdated
}

/**
* Note: {@code threadContexts} <em>must</em> be a modifiable {@link Set} so that stale contexts can be removed.

This comment has been minimized.

Copy link
@jaymode

jaymode Jul 6, 2016

Member

I think it would be simpler to have the node remove its threadcontext from here on close

@jaymode

This comment has been minimized.

Copy link
Member

commented Jul 6, 2016

left one comment, other than that LGTM

@pickypg pickypg force-pushed the pickypg:feature/deprecated-rest-handler-17687 branch Jul 6, 2016

Add DeprecationRestHandler to automatically log deprecated REST calls
This adds a new proxy for RestHandlers and RestControllers so that requests made
to deprecated REST APIs can be automatically logged in the ES logs via the
DeprecationLogger as well as via a "Warning" header (RFC-7234) for all responses.

@pickypg pickypg force-pushed the pickypg:feature/deprecated-rest-handler-17687 branch to b927cfe Jul 6, 2016

@pickypg pickypg removed the review label Jul 6, 2016

@pickypg pickypg merged commit b927cfe into elastic:master Jul 6, 2016

1 check passed

CLA Commit author has signed the CLA
Details

@pickypg pickypg changed the title Add DeprecationRestHandler to automatically log deprecated REST calls Send HTTP Warning Header(s) for any Deprecation Usage from a REST request Jul 7, 2016

@pickypg pickypg deleted the pickypg:feature/deprecated-rest-handler-17687 branch Jul 7, 2016

@frioux

This comment has been minimized.

Copy link

commented Jan 17, 2018

In case anyone in the internet superfuture runs into this like we did, this can pretty easily cause a long enough set of headers that nginx cannot even parse the response, and thus can cause pretty serious outages. Would be great if it could be disabled entirely, but you can set proxy_buffer_size to something higher to resolve it at least temporarily.

@jasontedor

This comment has been minimized.

Copy link
Member

commented Jan 18, 2018

@frioux Can you share the contents of these large responses? I would be interested in seeing if there are headers that we need to de-duplicate (we do this already to some extent, this is part of what pushes my interest here).

@frioux

This comment has been minimized.

Copy link

commented Jan 18, 2018

HTTP/1.1 200 OK
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
Warning: 299 Elasticsearch-5.4.0-780f8c4 "The [string] field is deprecated, please use [text] or [keyword] instead on [REDACTED]" "Wed, 17 Jan 2018 19:03:29 GMT"
content-type: application/json; charset=UTF-8
content-encoding: gzip
transfer-encoding: chunked

1

Of course instead of REDACTED it was a distinct field for each thing. 12k is a lot of data to jam into headers ;)

@jasontedor

This comment has been minimized.

Copy link
Member

commented Jan 18, 2018

Okay; it is because it's a distinct field. Thanks for that, I will see what, if anything, can be done (I am doubtful because of the way these are collected).

@frioux

This comment has been minimized.

Copy link

commented Jan 18, 2018

It would be good to limit this, because this is pretty gnarly to debug (I could only do it with a network trace) and anyone who puts nginx in front of ES would run into it.

@jasontedor

This comment has been minimized.

Copy link
Member

commented Jan 18, 2018

I agree with you. I opened #28301 and this work is assigned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.