New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dns Codec updated pull request #1622

Closed
wants to merge 29 commits into
base: master
from

Conversation

Projects
None yet
6 participants
@mbakkar

mbakkar commented Jul 20, 2013

I had already closed the other pull request and deleted the repository. I don't think I can reopen pull requests. But here is an updated one (with some changes to work with 4.0.2).

I noticed when writing to a channel information isn't flushed automatically, I have to use writeAndFlush(). Intentional?

@ghost

This comment has been minimized.

ghost commented Jul 20, 2013

Build result for #1622 at ef5c715: Failure

@normanmaurer

This comment has been minimized.

Member

normanmaurer commented Jul 20, 2013

Yes... See the release notes of 4.0.0.Final.

Will check later

Am 20.07.2013 um 18:59 schrieb mbakkar notifications@github.com:

I had already closed the other pull request and deleted the repository. I don't think I can reopen pull requests. But here is an updated one (with some changes to work with 4.0.2).

I noticed when writing to a channel information isn't flushed automatically, I have to use writeAndFlush(). Intentional?

You can merge this Pull Request by running

git pull https://github.com/mbakkar/netty-1 master
Or view, comment on, or merge it at:

#1622

Commit Summary

Updated for netty 4.0.2
File Changes

A codec/src/main/java/io/netty/handler/codec/dns/DnsEntry.java (381)
A codec/src/main/java/io/netty/handler/codec/dns/DnsHeader.java (175)
A codec/src/main/java/io/netty/handler/codec/dns/DnsMessage.java (132)
A codec/src/main/java/io/netty/handler/codec/dns/DnsQuery.java (31)
A codec/src/main/java/io/netty/handler/codec/dns/DnsQueryEncoder.java (109)
A codec/src/main/java/io/netty/handler/codec/dns/DnsQueryHeader.java (65)
A codec/src/main/java/io/netty/handler/codec/dns/DnsResponse.java (87)
A codec/src/main/java/io/netty/handler/codec/dns/DnsResponseDecoder.java (211)
A codec/src/main/java/io/netty/handler/codec/dns/DnsResponseHeader.java (242)
A codec/src/main/java/io/netty/handler/codec/dns/Question.java (71)
A codec/src/main/java/io/netty/handler/codec/dns/Resource.java (128)
A codec/src/main/java/io/netty/handler/codec/dns/ResponseCode.java (125)
A codec/src/main/java/io/netty/handler/codec/dns/package-info.java (21)
A codec/src/test/java/io/netty/handler/codec/dns/DnsCodecTest.java (105)
A codec/src/test/java/io/netty/handler/codec/dns/package-info.java (21)
A handler/src/main/java/io/netty/handler/dns/DnsCallback.java (220)
A handler/src/main/java/io/netty/handler/dns/DnsClientInitializer.java (37)
A handler/src/main/java/io/netty/handler/dns/DnsExchangeFactory.java (494)
A handler/src/main/java/io/netty/handler/dns/InboundDnsMessageHandler.java (53)
A handler/src/main/java/io/netty/handler/dns/ResourceCache.java (172)
A handler/src/main/java/io/netty/handler/dns/SingleResultCallback.java (59)
A handler/src/main/java/io/netty/handler/dns/decoder/AddressDecoder.java (64)
A handler/src/main/java/io/netty/handler/dns/decoder/DomainDecoder.java (42)
A handler/src/main/java/io/netty/handler/dns/decoder/MailExchangerDecoder.java (47)
A handler/src/main/java/io/netty/handler/dns/decoder/RecordDecoder.java (45)
A handler/src/main/java/io/netty/handler/dns/decoder/RecordDecoderFactory.java (135)
A handler/src/main/java/io/netty/handler/dns/decoder/ServiceDecoder.java (49)
A handler/src/main/java/io/netty/handler/dns/decoder/StartOfAuthorityDecoder.java (52)
A handler/src/main/java/io/netty/handler/dns/decoder/TextDecoder.java (54)
A handler/src/main/java/io/netty/handler/dns/decoder/package-info.java (21)
A handler/src/main/java/io/netty/handler/dns/decoder/record/MailExchangerRecord.java (57)
A handler/src/main/java/io/netty/handler/dns/decoder/record/ServiceRecord.java (113)
A handler/src/main/java/io/netty/handler/dns/decoder/record/StartOfAuthorityRecord.java (123)
A handler/src/main/java/io/netty/handler/dns/decoder/record/package-info.java (21)
A handler/src/main/java/io/netty/handler/dns/package-info.java (21)
A handler/src/test/java/io/netty/handler/dns/DnsImplementationTest.java (85)
A handler/src/test/java/io/netty/handler/dns/package-info.java (21)
Patch Links:

https://github.com/netty/netty/pull/1622.patch
https://github.com/netty/netty/pull/1622.diff

@caniszczyk

This comment has been minimized.

caniszczyk commented Jul 20, 2013

We need to make sure @mbakkar signs a CLA too if that wasn't done already :)

Really looking forward to getting this in by the end of the summer :)

@mbakkar

This comment has been minimized.

mbakkar commented Jul 20, 2013

I believe I did sign a CLA in the beginning

@ghost

This comment has been minimized.

ghost commented Jul 20, 2013

Build result for #1622 at f16cd9b: Success

@normanmaurer

This comment has been minimized.

Member

normanmaurer commented Jul 20, 2013

@caniszczyk let me review this and merge in at the end...

@ghost ghost assigned normanmaurer Jul 20, 2013

*/
@Override
public ByteBuf content() {
return rawPacket.readerIndex(originalIndex);

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

Why reset the index here ? I think this should not be done as we not do this in other ByteBufHolder impls

}
buf.readerIndex((len & 0x3f) << 8 | buf.readUnsignedByte());
} else {
name.append(buf.toString(buf.readerIndex(), len, Charset.forName("UTF-8"))).append(".");

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

Use CharsetUtil.UTF_8

int aClass = buf.readUnsignedShort();
long ttl = buf.readUnsignedInt();
int len = buf.readUnsignedShort();
ByteBuf resourceData = Unpooled.buffer(len);

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

Would be better do pass in the ByteBufAllocator and use it for allocate new buffer

String[] parts = question.name().split("\\.");
for (int i = 0; i < parts.length; i++) {
buf.writeByte(parts[i].length());
buf.writeBytes(parts[i].getBytes());

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

Pass in Charset to getBytes(...)

if (pointer) {
offset = (len & 0x3f) << 8 | buf.getUnsignedByte(offset++);
} else {
name.append(buf.toString(offset, len, Charset.forName("UTF-8"))).append(".");

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

Use CharsetUtil.UTF_8

*/
@Override
protected void decode(ChannelHandlerContext ctx, DatagramPacket packet, List<Object> out) throws Exception {
out.add(decodeResponse(packet.content()).retain());

This comment has been minimized.

@normanmaurer

normanmaurer Jul 20, 2013

Member

why retain ?

-Replaced Charset.forName with CharsetUtil.UTF_8
-DnsResponse returns content with unmodified index
@normanmaurer

This comment has been minimized.

Member

normanmaurer commented Jul 20, 2013

@mbakkar what I would like to see is to create a new "module" called codec-dns and merge everything you have currently split into codec and handler into it.

- Moved dns to its own module "codec-dns"
- Decoding and encoding methods now pass ByteBufAllocator and Charset
@mbakkar

This comment has been minimized.

mbakkar commented Jul 20, 2013

I have to retain it because the DnsResponse is used in the next handler in the chain (InboundDnsMessageHandler for the impl test, and for the codec test it is the "Handler" inner-class in DnsCodecTest)

EDIT: some package-infos got erased, re-committing

@ghost

This comment has been minimized.

ghost commented Jul 20, 2013

Build result for #1622 at 17e3bb7: Success

Callable<List<ByteBuf>> callback = new DnsCallback<List<ByteBuf>>(-1, sendQuery(DnsEntry.TYPE_A,
"google.com", id, channel));
return callback.call() != null;
} catch (Exception e) {

This comment has been minimized.

@normanmaurer

normanmaurer Jul 21, 2013

Member

Don't swallow exceptions... Please at least log it.

try {
result = (T) decoder.decode(response, resource);
} catch (Exception e) {
e.printStackTrace();

This comment has been minimized.

@normanmaurer

normanmaurer Jul 21, 2013

Member

Wouldn't it be better to throw a DecoderException here ?

channel.write(queries[i]).sync();
}
} catch (Exception e) {
e.printStackTrace();

This comment has been minimized.

@normanmaurer

normanmaurer Jul 21, 2013

Member

Log the exception

try {
channel.close().sync();
} catch (Exception e) {
e.printStackTrace();

This comment has been minimized.

@normanmaurer

normanmaurer Jul 21, 2013

Member

Log the exception

DnsExchangeFactory.removeChannel(ctx.channel());
return;
}
cause.printStackTrace();

This comment has been minimized.

@normanmaurer

normanmaurer Jul 21, 2013

Member

forward to the next handler via ctx.fireExceptionCaught(cause)

- Added more logging, removed printing of exceptions
- RecordDecoderFactory throws DecoderException now in constructor
@ghost

This comment has been minimized.

ghost commented Jul 21, 2013

Build result for #1622 at 0e4bb81: Success

@normanmaurer

This comment has been minimized.

Check for logger.isWarnEnabled(...) before construct StringBuilder etc.

@normanmaurer

This comment has been minimized.

Check for logger.isErrorEnabled(...) before construct StringBuilder etc.

@ghost

This comment has been minimized.

ghost commented Aug 11, 2013

Build result for #1622 at 562767b: Failure

@ghost

This comment has been minimized.

ghost commented Aug 11, 2013

Build result for #1622 at 218a65d: Failure

.
@ghost

This comment has been minimized.

ghost commented Aug 11, 2013

Build result for #1622 at 6b8b35e: Failure

@ghost ghost referenced this pull request Aug 14, 2013

Closed

circular-dependency refactorings #1740

@trustin

This comment has been minimized.

trustin commented on 218a65d Aug 16, 2013

Could you explain why netty-transport has to depend on netty-dns?

@mbakkar

This comment has been minimized.

mbakkar commented Aug 16, 2013

Doesn't it need to use its resolver in Bootstrap classes?

@trustin

This comment has been minimized.

Member

trustin commented Aug 16, 2013

Not really. The resolver interface should reside at netty-transport. netty-dns should provide only the async resolver implementation.

@trustin

This comment has been minimized.

Member

trustin commented Aug 16, 2013

I guess this could clear things a lot:

netty-transport:
public interface io.netty.dns.DomainNameResolver

netty-asyncdns:
public class io.netty.dns.async.AsyncDomainNameResolver implements io.netty.dns.DomainNameResolver

i.e. io.netty.dns belongs to netty-transport. io.netty.dns.async belongs to netty-asyncdns.

@mbakkar

This comment has been minimized.

mbakkar commented Aug 16, 2013

I see, and AsyncDomainNameResolver would have an instance of/extend DnsResolver, good idea.

response.addQuestion(decodeQuestion(buf));
}
if (header.getResponseCode() != 0) {
System.err.println("Encountered error decoding DNS response for domain \""

This comment has been minimized.

@normanmaurer

normanmaurer Aug 19, 2013

Member

I think you should just use "return response" here.

@travishaagen

This comment has been minimized.

Contributor

travishaagen commented Aug 19, 2013

I see that DnsResource provides access to the TTL value, but the decoders in io.netty.dns.decoder do not expose the TTL. For some kinds of DNS clients, we might need the TTL, to provide a secondary caching layer (e.g., Sender Policy Framework).

If this is an uncommon need, then one could extend all of the decoders for their particular use-case, but if clients will often need to know the TTL, then perhaps it should be copied from DnsResource within the decode-function and exposed by the decoders.

@caniszczyk

This comment has been minimized.

caniszczyk commented Sep 3, 2013

@mbakkar how are things going with this project?

@mbakkar

This comment has been minimized.

mbakkar commented Sep 4, 2013

Hi just a brief update.. I haven't really made progress on this in the past 2 weeks. I know clearly what I need to do but I've been stretched for time since I started school. I finish handing in a couple major assignments tomorrow at 11:59 PM, and then I should have a few days leeway where I can continue working on this after that. Sorry for getting back so late..!

@caniszczyk

This comment has been minimized.

caniszczyk commented Sep 9, 2013

Thanks @mbakkar! Pencils down starts next week so it would be good to start aiming for the finish line on the project.

@trustin is in the main Twitter office this week so we can hopefully work on getting the code in a state that it can be merged into master :)

@trustin

This comment has been minimized.

Member

trustin commented Sep 9, 2013

@mbakkar, no worries! You've done a great job so far. Please take your time.

@caniszczyk, sure. I'll review it and merge it into master once it's ready. :-)

@normanmaurer

This comment has been minimized.

Member

normanmaurer commented Nov 5, 2013

@mbakkar @caniszczyk @trustin working on clean this up now and merge to master.. stay tuned.

@normanmaurer

This comment has been minimized.

Member

normanmaurer commented Nov 5, 2013

I merged everything into dns branch.. Let me polish the code there. So closing this now.. @mbakkar thanks again for all your work

normanmaurer pushed a commit that referenced this pull request Mar 10, 2014

Norman Maurer
DNS codec for Netty which is based on the work of [#1622].
This PR also includes a AsynchronousDnsResolver which allows to resolve DNS entries in a non blocking way by make use
of the dns codec and netty transport itself.

normanmaurer pushed a commit that referenced this pull request Jun 2, 2014

Norman Maurer
DNS codec for Netty which is based on the work of [#1622].
This PR also includes a AsynchronousDnsResolver which allows to resolve DNS entries in a non blocking way by make use
of the dns codec and netty transport itself.

normanmaurer pushed a commit that referenced this pull request Jun 2, 2014

Norman Maurer
DNS codec for Netty which is based on the work of [#1622].
Motivation:
As part of GSOC 2013 we had @mbakkar working on a DNS codec but did not integrate it yet as it needs some cleanup. This commit is based on @mbakkar's work and provide the codec for DNS.

Modifications:
Add DNS codec

Result:
Reusable DNS codec will be included in netty.

This PR also includes a AsynchronousDnsResolver which allows to resolve DNS entries in a non blocking way by make use
of the dns codec and netty transport itself.

normanmaurer pushed a commit that referenced this pull request Jun 10, 2014

Norman Maurer
DNS codec for Netty which is based on the work of [#1622].
Motivation:
As part of GSOC 2013 we had @mbakkar working on a DNS codec but did not integrate it yet as it needs some cleanup. This commit is based on @mbakkar's work and provide the codec for DNS.

Modifications:
Add DNS codec

Result:
Reusable DNS codec will be included in netty.

This PR also includes a AsynchronousDnsResolver which allows to resolve DNS entries in a non blocking way by make use
of the dns codec and netty transport itself.

normanmaurer pushed a commit that referenced this pull request Jun 10, 2014

Norman Maurer
DNS codec for Netty which is based on the work of [#1622].
Motivation:
As part of GSOC 2013 we had @mbakkar working on a DNS codec but did not integrate it yet as it needs some cleanup. This commit is based on @mbakkar's work and provide the codec for DNS.

Modifications:
Add DNS codec

Result:
Reusable DNS codec will be included in netty.

This PR also includes a AsynchronousDnsResolver which allows to resolve DNS entries in a non blocking way by make use
of the dns codec and netty transport itself.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment