Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP multi-headers are randomized in camel-knative-http #69

Closed
nicolaferraro opened this issue May 22, 2019 · 9 comments
Closed

HTTP multi-headers are randomized in camel-knative-http #69

nicolaferraro opened this issue May 22, 2019 · 9 comments
Assignees
@nicolaferraro

Comments

@nicolaferraro
Copy link
Member

This line

camel-k-runtime/camel-knative-http/src/main/java/org/apache/camel/component/knative/http/KnativeHttpComponent.java

Line 222 in 72c9d2a

final String headerVal = headers.get(filterKey);
behave in a random way when the header contains multiple values.

In case of the header ce-knativehistory, values may be like: channel1.ns.channels.cluster.local; channel2.ns.channels.cluster.local. They are interpreted by netty as being present multiple times (which is correct) and only one of them is returned by that call (which is correct), but a random one (which is wrong).
We should find a way to get always the last value of the header, which is the last channel that received the message.

I've tried also to get the full list of values using getAll but the order of elements in the list is still randomized.

I've found there's a CombinedHttpHeader class in netty that may solve the issue by appending values to the same header instead of adding multiple entries, but I'm not able to plug it in Camel Netty.

Any idea @lburgazzoli ?
@nicolaferraro nicolaferraro mentioned this issue May 22, 2019
Merged
@lburgazzoli
Copy link
Contributor

@nicolaferraro can you check that this is solved with the latest code based on undertow ?

@nicolaferraro nicolaferraro self-assigned this Aug 2, 2019
@nicolaferraro
Copy link
Member Author

I'll write some integration tests

@nicolaferraro
Copy link
Member Author

Still present on latest version.

Scenario:

integration 1

from('timer:tick')
  .setBody().constant("message")
  .to('knative:channel/messages')

from('timer:tick')
  .setBody().constant("word")
  .to('knative:channel/words')

integration 2:

from('knative:channel/messages')
  .transform().simple('transformed ${body}')
  .log('Forwarding: ${body}')
  .to('knative:channel/words')

integration 3:

from('knative:channel/messages')
  .log('From messages: ${body}')
  .to('log:messages?showHeaders=true')

from('knative:channel/words')
  .log('From words: ${body}')
  .to('log:words?showHeaders=true')

Technically only 3 messages should be printed by integration 3:

From words: word
From messages: message
From words: transformed message

But sometimes I get:

From messages: transformed message

Looking at the headers, I see they are randomized:

Case 1:

knativemultihop3-plqdx-deployment-74c5bfb57c-hl27h integration 2019-10-17 14:30:11.566 INFO  [vert.x-eventloop-thread-1] route2 - From words: transformed message
knativemultihop3-plqdx-deployment-74c5bfb57c-hl27h integration 2019-10-17 14:30:11.567 INFO  [vert.x-eventloop-thread-1] words - Exchange[ExchangePattern: InOut, Headers: {Accept-Encoding=gzip, CamelHttpMethod=POST, CamelHttpPath=, CamelHttpQuery=null, CamelHttpUri=/, Ce-Knativehistory=[words-kn-channel.default.svc.cluster.local, messages-kn-channel.default.svc.cluster.local], Content-Length=19, Forwarded=for=172.17.0.18;proto=http, Host=knativemultihop3.default.svc.cluster.local, User-Agent=Go-http-client/1.1, X-B3-Parentspanid=caaf9276d3264172, X-B3-Sampled=0, X-B3-Spanid=effe710e68739a37, X-B3-Traceid=b6a3dfa9d50abc9f9dc41c7803c25ec8, X-Envoy-Decorator-Operation=knativemultihop3-plqdx.default.svc.cluster.local:80/*, X-Envoy-Expected-Rq-Timeout-Ms=600000, X-Envoy-Internal=true, X-Forwarded-For=172.17.0.18, 172.17.0.9, X-Forwarded-Proto=http, X-Request-Id=7ac64e5c-a8d9-9d73-b6db-a904afb4e5cc}, BodyType: byte[], Body: transformed message]

Case 2:

knativemultihop3-plqdx-deployment-74c5bfb57c-hl27h integration 2019-10-17 14:30:12.581 INFO  [vert.x-eventloop-thread-1] route1 - From messages: transformed message
knativemultihop3-plqdx-deployment-74c5bfb57c-hl27h integration 2019-10-17 14:30:12.582 INFO  [vert.x-eventloop-thread-1] messages - Exchange[ExchangePattern: InOut, Headers: {Accept-Encoding=gzip, CamelHttpMethod=POST, CamelHttpPath=, CamelHttpQuery=null, CamelHttpUri=/, Ce-Knativehistory=[messages-kn-channel.default.svc.cluster.local, words-kn-channel.default.svc.cluster.local], Content-Length=19, Forwarded=for=172.17.0.18;proto=http, Host=knativemultihop3.default.svc.cluster.local, User-Agent=Go-http-client/1.1, X-B3-Parentspanid=6240cbbfd50bc5ec, X-B3-Sampled=0, X-B3-Spanid=9f6716a0ffd5b461, X-B3-Traceid=26669ebb2cf36e75c32df7ae62add8e8, X-Envoy-Decorator-Operation=knativemultihop3-plqdx.default.svc.cluster.local:80/*, X-Envoy-Expected-Rq-Timeout-Ms=600000, X-Envoy-Internal=true, X-Forwarded-For=172.17.0.18, 172.17.0.9, X-Forwarded-Proto=http, X-Request-Id=5141fe62-5122-9f70-8983-c4c5023ad3cb}, BodyType: byte[], Body: transformed message]

Note that the component seems to take the first value of that header, which is the wrong way to deal with it, since channels are always appended as they are encountered.

lburgazzoli added a commit to lburgazzoli/apache-camel-k-runtime that referenced this issue Oct 17, 2019
@lburgazzoli
HTTP multi-headers are randomized in camel-knative-http apache#69
5235570
@lburgazzoli
Copy link
Contributor

I've updated the code to always look for the latest value and wrote this test: https://github.com/lburgazzoli/apache-camel-k-runtime/blob/github-69/camel-knative/camel-knative-http/src/test/java/org/apache/camel/component/knative/http/KnativeHttpTest.java#L1144-L1182

lburgazzoli added a commit to lburgazzoli/apache-camel-k-runtime that referenced this issue Oct 17, 2019
@lburgazzoli
HTTP multi-headers are randomized in camel-knative-http apache#69
34e81d3
@nicolaferraro
Copy link
Member Author

Here's a dump of the raw http requests. They arrive randomized.... Knative bug..

netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 POST / HTTP/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 Host: netcat.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 Accept-Encoding: gzip
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-B3-Sampled: 0
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-B3-Spanid: 09ec924b1d4fa090
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Forwarded-Proto: http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 Content-Length: 19
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 Forwarded: for=172.17.0.18;proto=http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Envoy-Internal: true
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 User-Agent: Go-http-client/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Forwarded-For: 172.17.0.18, 172.17.0.9
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Request-Id: 1caeb6e1-8f42-95f5-8181-00f03c14df0e
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 Ce-Knativehistory: words-kn-channel.default.svc.cluster.local,messages-kn-channel.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-B3-Parentspanid: 59ee03986829726a
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-B3-Traceid: 90cd833dc27a7bad78336de44206f540
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Envoy-Decorator-Operation: netcat-tssbl.default.svc.cluster.local:80/*
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:46 X-Envoy-Expected-Rq-Timeout-Ms: 600000

netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 POST / HTTP/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 Host: netcat.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 Content-Length: 19
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-B3-Sampled: 0
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Envoy-Decorator-Operation: netcat-tssbl.default.svc.cluster.local:80/*
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Forwarded-Proto: http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Request-Id: 11d880b7-9111-98c9-9d94-423fa6f2e075
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 Accept-Encoding: gzip
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-B3-Spanid: 4dd31f73fe46a4f6
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Envoy-Internal: true
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Forwarded-For: 172.17.0.18, 172.17.0.9
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 User-Agent: Go-http-client/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 Forwarded: for=172.17.0.18;proto=http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-B3-Parentspanid: b848c7058a9804d7
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-Envoy-Expected-Rq-Timeout-Ms: 600000
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 Ce-Knativehistory: messages-kn-channel.default.svc.cluster.local,words-kn-channel.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:47 X-B3-Traceid: fbf3d9b3f121a24fc5038c95d51e7ca6

netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 POST / HTTP/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 Host: netcat.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 Content-Length: 19
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 Forwarded: for=172.17.0.18;proto=http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-B3-Spanid: e360eff8f7d99b1b
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 User-Agent: Go-http-client/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Envoy-Expected-Rq-Timeout-Ms: 600000
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Forwarded-For: 172.17.0.18, 172.17.0.9
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Forwarded-Proto: http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Request-Id: 18c7f954-89ad-91b4-9950-e772f96a7247
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 Accept-Encoding: gzip
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 Ce-Knativehistory: words-kn-channel.default.svc.cluster.local,messages-kn-channel.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-B3-Parentspanid: 17a38a73ab079743
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-B3-Sampled: 0
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-B3-Traceid: 1061ba846c2426afdc34ebbb31385e6a
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Envoy-Decorator-Operation: netcat-tssbl.default.svc.cluster.local:80/*
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:48 X-Envoy-Internal: true

netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 POST / HTTP/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 Host: netcat.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Envoy-Decorator-Operation: netcat-tssbl.default.svc.cluster.local:80/*
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Forwarded-Proto: http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 User-Agent: Go-http-client/1.1
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 Forwarded: for=172.17.0.18;proto=http
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-B3-Traceid: 97b9e94082bffd204e8cad994f478a5b
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-B3-Parentspanid: 76fd4de1cc7629b0
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Envoy-Expected-Rq-Timeout-Ms: 600000
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Forwarded-For: 172.17.0.18, 172.17.0.9
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 Content-Length: 19
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 Ce-Knativehistory: words-kn-channel.default.svc.cluster.local,messages-kn-channel.default.svc.cluster.local
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Envoy-Internal: true
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-Request-Id: 3d9f5de6-9c6d-99f7-8be4-9bcf30fdf8c8
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 Accept-Encoding: gzip
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-B3-Sampled: 0
netcat-tssbl-deployment-5d7b648b4-ns6zs user-container 2019/10/17 22:09:49 X-B3-Spanid: 62bc3c278ede46bd

lburgazzoli added a commit to lburgazzoli/apache-camel-k-runtime that referenced this issue Oct 17, 2019
@lburgazzoli
HTTP multi-headers are randomized in camel-knative-http apache#69
7aad5df
@lburgazzoli lburgazzoli mentioned this issue Oct 17, 2019
Merged
lburgazzoli added a commit that referenced this issue Oct 18, 2019
@lburgazzoli
HTTP multi-headers are randomized in camel-knative-http #69
c17f8ab
@lburgazzoli
Copy link
Contributor

The issue about taking the first value of an header has been fixed: #168

@lburgazzoli
Copy link
Contributor

@nicolaferraro I think the best would be to have the latest channel/endpoint being set to a dedicated header as single value

@lburgazzoli
Copy link
Contributor

@nicolaferraro should we close this one as it is not a camel-k-runtime issue ?

@nicolaferraro
Copy link
Member Author

Not a runtime issue and no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nicolaferraro nicolaferraro

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lburgazzoli @nicolaferraro