Merge branch 'master' into v5 (#784)

* update geoip, geosite

* Chore: bump google.golang.org/grpc from 1.35.0 to 1.36.0 (#711)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Chore: bump github.com/miekg/dns from 1.1.39 to 1.1.40 (#712)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add /opt to assets location (#715)

* Add definition for transport layer chained proxy

* Regenerate protobuf for transport layer chain proxy

* Added Transport Layer Chained Proxy Support

* Fix dependency cycle caused by core import in internet package

* Fix forced outbound tag not set correctly

* Disable routing for platform initialized detour

* Added Auto generated file

* don't build tagged outbound dial on configure setting

* Fix for context with empty content

* Fix ALPN being set to h2 by default when using TCP (#716)

* Deprecate legacy VMess header with a planned decommission (#717)

* Zero Security imaginary security level

* Regenerate protobuf for Zero Security imaginary security level

* Imaginary Security Lever: zero: turn off all security on payload data

* Test for Imaginary Security Level: zero

* Fix panic: index out of range (#727)

* Chore: update dependencies & protobuf (#728)

* A memory-efficient and fast hybrid matcher (#639)

* a faster DomainMatcher implementation

* rename benchmark name

* fix linting errors

* add hybrid matcher

* add rabin-karp algorithm

* rename test & fix linting errors

* add more comment

* format code

* revert `MatcherGroup` match func

* fix linting errors

* Allow the selection of domain matcher

* Apply domain selector choice

* json parsing rule for domain matcher

* output debug message when ACAutomatonDomainMatcher is enabled.

* update version

* update geoip, geosite

* Chore: bump github.com/google/go-cmp from 0.5.4 to 0.5.5 (#732)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* workaround crash when V is not in context

* rename config for NewACAutomatonDomainMatcher to hybrid

* Allow bulk definition of domain matcher at parent level

* fix misbehaving code crash and create bug on transport level front proxy

* fixing misbehaving code in mux that do not propagate context

* create session content in the context if do not exist yet

* Create a name for linear domain matcher

* update version to 4.35.1

* Chore: update protobuf & dependencies (#748)

* Chore: bump actions/stale from v3.0.17 to v3.0.18 (#752)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* DNS: refine Android bootstrap DNS logic (#767)

* Chore: bump github.com/pires/go-proxyproto from 0.4.2 to 0.5.0 (#751)

Bumps [github.com/pires/go-proxyproto](https://github.com/pires/go-proxyproto) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/pires/go-proxyproto/releases)
- [Commits](pires/go-proxyproto@v0.4.2...v0.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Grpc Gun Transport (#757)

* introduce grpc transport structure

* fix package name inconsistency

* grpc gun transport dialer and listener

* add selective build tag

* add grpc:gun listener

* add grpc:gun config

* add generated files

* various bug fix for gun:grpc transport

* Cache dialed connections

* grpc:gun Use V2Ray Managed Dial function

* Update destination.pb.go

* Update gun.go

* GunSettings -> GunConfig

* gu -> gs

* add grpc alias

Co-authored-by: RPRX <63339210+rprx@users.noreply.github.com>
Co-authored-by: kslr <kslrwang@gmail.com>

* fix applied wrong name, and wrong varible name

* Add grpcSettings (alias of gunSettings)

* update geoip, geosite

* loopback outbound, allow you to redirect connection to the dispatcher again (#770)

* Added Loop back proxy

* Added json processing for lo proxy

* Fix bug for lo proxy

* Fix bug for lo proxy

* rename the outbound name

* Loopback: update naming and fix lint issues

* Chore: change lo to loopback

Co-authored-by: kslr <kslrwang@gmail.com>
Co-authored-by: loyalsoldier <10487845+Loyalsoldier@users.noreply.github.com>

* update version

* Chore: format import using goimports (#780)

* Chore: fix lint according to golangci-lint errors (#781)

* Chore: fix lint according to golangci-lint errors
* Chore: regenerate pb.go files

* Add minimal perfect hash domain matcher (#743)

* rename to HybridDomainMatcher & convert domain to lowercase

* refactor code & add open hashing for rolling hash map

* fix lint errors

* update app/dns/dns.go

* convert domain to lowercase in `strmatcher.go`

* keep the original matcher behavior

* add mph domain matcher & conver domain names to loweercase when matching

* fix lint errors

* fix lint errors

* Route: mph add alias hybrid

* FakeDNS: use 198.18.0.0/15 as default IP pool (#779)

* Add remote address to grpc transport layer conn (#783)

* Add remote address to grpc transport layer conn

* go fmt

* Revert "Test: fix http2 dial timeout (#570)" (#778)

* Revert "Test: fix http2 dial timeout (#570)"

This reverts commit 405a051.

* Feat: lower the payload size

* Remove state.NegotiatedProtocolIsMutual

It has been deprecated since Go 1.16 because it shouldn't be used: this value is always true.

* Chore: format code

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kid <44045911+kidonng@users.noreply.github.com>
Co-authored-by: Shelikhoo <xiaokangwang@outlook.com>
Co-authored-by: 秋のかえで <autmaple@protonmail.com>
Co-authored-by: DarthVader <61409963+darsvador@users.noreply.github.com>
Co-authored-by: CalmLong <37164399+CalmLong@users.noreply.github.com>
Co-authored-by: RPRX <63339210+rprx@users.noreply.github.com>
Co-authored-by: kslr <kslrwang@gmail.com>
Co-authored-by: maskedeken <52683904+maskedeken@users.noreply.github.com>

.github/workflows/stale.yml

@@ -8,7 +8,7 @@ jobs:
   stale:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/stale@v3.0.17
+      - uses: actions/stale@v3.0.18
         with:
           repo-token: ${{ secrets.GITHUB_TOKEN }}
           stale-issue-message: "This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days"

app/dispatcher/default.go

@@ -326,7 +326,18 @@ func (d *DefaultDispatcher) targetedDispatch(ctx context.Context, link *transpor
 func (d *DefaultDispatcher) routedDispatch(ctx context.Context, link *transport.Link, destination net.Destination) {
 	var handler outbound.Handler
 
-	if d.router != nil {
+	if forcedOutboundTag := session.GetForcedOutboundTagFromContext(ctx); forcedOutboundTag != "" {
+		ctx = session.SetForcedOutboundTagToContext(ctx, "")
+		if h := d.ohm.GetHandler(forcedOutboundTag); h != nil {
+			newError("taking platform initialized detour [", forcedOutboundTag, "] for [", destination, "]").WriteToLog(session.ExportIDToError(ctx))
+			handler = h
+		} else {
+			newError("non existing tag for platform initialized detour: ", forcedOutboundTag).AtError().WriteToLog(session.ExportIDToError(ctx))
+			common.Close(link.Writer)
+			common.Interrupt(link.Reader)
+			return
+		}
+	} else if d.router != nil {
 		if route, err := d.router.PickRoute(routing_session.AsRoutingContext(ctx)); err == nil {
 			tag := route.GetOutboundTag()
 			if h := d.ohm.GetHandler(tag); h != nil {

app/dns/dns.go

@@ -64,9 +64,6 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
 	for _, ns := range config.NameServer {
 		domainRuleCount += len(ns.PrioritizedDomain)
 	}
-	// Fixes https://github.com/v2fly/v2ray-core/issues/529
-	// Compatible with `localhost` nameserver specified in config file
-	domainRuleCount += len(localTLDsAndDotlessDomains)
 
 	// MatcherInfos is ensured to cover the maximum index domainMatcher could return, where matcher's index starts from 1
 	matcherInfos := make([]DomainMatcherInfo, domainRuleCount+1)
@@ -84,7 +81,7 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
 
 	for _, ns := range config.NameServer {
 		clientIdx := len(clients)
-		updateDomain := func(domainRule strmatcher.Matcher, originalRuleIdx int) error {
+		updateDomain := func(domainRule strmatcher.Matcher, originalRuleIdx int, matcherInfos []DomainMatcherInfo) error {
 			midx := domainMatcher.Add(domainRule)
 			matcherInfos[midx] = DomainMatcherInfo{
 				clientIdx:     uint16(clientIdx),
@@ -98,7 +95,7 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
 		case net.IPv4len, net.IPv6len:
 			myClientIP = net.IP(ns.ClientIp)
 		}
-		client, err := NewClient(ctx, ns, myClientIP, geoipContainer, updateDomain)
+		client, err := NewClient(ctx, ns, myClientIP, geoipContainer, &matcherInfos, updateDomain)
 		if err != nil {
 			return nil, newError("failed to create client").Base(err)
 		}

app/dns/fakedns/fake.go

@@ -43,7 +43,7 @@ func NewFakeDNSHolder() (*Holder, error) {
 	if fkdns, err = NewFakeDNSHolderConfigOnly(nil); err != nil {
 		return nil, newError("Unable to create Fake Dns Engine").Base(err).AtError()
 	}
-	err = fkdns.initialize("240.0.0.0/8", 65535)
+	err = fkdns.initialize("198.18.0.0/15", 65535)
 	if err != nil {
 		return nil, err
 	}

app/dns/fakedns/fakedns_test.go

@@ -20,18 +20,18 @@ func TestFakeDnsHolderCreateMapping(t *testing.T) {
 	common.Must(err)
 
 	addr := fkdns.GetFakeIPForDomain("fakednstest.v2fly.org")
-	assert.Equal(t, "240.0.0.0", addr[0].IP().String())
+	assert.Equal(t, "198.18.0.0", addr[0].IP().String())
 }
 
 func TestFakeDnsHolderCreateMappingMany(t *testing.T) {
 	fkdns, err := NewFakeDNSHolder()
 	common.Must(err)
 
 	addr := fkdns.GetFakeIPForDomain("fakednstest.v2fly.org")
-	assert.Equal(t, "240.0.0.0", addr[0].IP().String())
+	assert.Equal(t, "198.18.0.0", addr[0].IP().String())
 
 	addr2 := fkdns.GetFakeIPForDomain("fakednstest2.v2fly.org")
-	assert.Equal(t, "240.0.0.1", addr2[0].IP().String())
+	assert.Equal(t, "198.18.0.1", addr2[0].IP().String())
 }
 
 func TestFakeDnsHolderCreateMappingManyAndResolve(t *testing.T) {
@@ -40,21 +40,21 @@ func TestFakeDnsHolderCreateMappingManyAndResolve(t *testing.T) {
 
 	{
 		addr := fkdns.GetFakeIPForDomain("fakednstest.v2fly.org")
-		assert.Equal(t, "240.0.0.0", addr[0].IP().String())
+		assert.Equal(t, "198.18.0.0", addr[0].IP().String())
 	}
 
 	{
 		addr2 := fkdns.GetFakeIPForDomain("fakednstest2.v2fly.org")
-		assert.Equal(t, "240.0.0.1", addr2[0].IP().String())
+		assert.Equal(t, "198.18.0.1", addr2[0].IP().String())
 	}
 
 	{
-		result := fkdns.GetDomainFromFakeDNS(net.ParseAddress("240.0.0.0"))
+		result := fkdns.GetDomainFromFakeDNS(net.ParseAddress("198.18.0.0"))
 		assert.Equal(t, "fakednstest.v2fly.org", result)
 	}
 
 	{
-		result := fkdns.GetDomainFromFakeDNS(net.ParseAddress("240.0.0.1"))
+		result := fkdns.GetDomainFromFakeDNS(net.ParseAddress("198.18.0.1"))
 		assert.Equal(t, "fakednstest2.v2fly.org", result)
 	}
 }
@@ -64,10 +64,10 @@ func TestFakeDnsHolderCreateMappingManySingleDomain(t *testing.T) {
 	common.Must(err)
 
 	addr := fkdns.GetFakeIPForDomain("fakednstest.v2fly.org")
-	assert.Equal(t, "240.0.0.0", addr[0].IP().String())
+	assert.Equal(t, "198.18.0.0", addr[0].IP().String())
 
 	addr2 := fkdns.GetFakeIPForDomain("fakednstest.v2fly.org")
-	assert.Equal(t, "240.0.0.0", addr2[0].IP().String())
+	assert.Equal(t, "198.18.0.0", addr2[0].IP().String())
 }
 
 func TestFakeDnsHolderCreateMappingAndRollOver(t *testing.T) {

app/dns/nameserver.go

@@ -63,7 +63,7 @@ func NewServer(dest net.Destination, dispatcher routing.Dispatcher) (Server, err
 }
 
 // NewClient creates a DNS client managing a name server with client IP, domain rules and expected IPs.
-func NewClient(ctx context.Context, ns *NameServer, clientIP net.IP, container router.GeoIPMatcherContainer, updateDomainRule func(strmatcher.Matcher, int) error) (*Client, error) {
+func NewClient(ctx context.Context, ns *NameServer, clientIP net.IP, container router.GeoIPMatcherContainer, matcherInfos *[]DomainMatcherInfo, updateDomainRule func(strmatcher.Matcher, int, []DomainMatcherInfo) error) (*Client, error) {
 	client := &Client{}
 	err := core.RequireFeatures(ctx, func(dispatcher routing.Dispatcher) error {
 		// Create a new server for each client for now
@@ -76,6 +76,19 @@ func NewClient(ctx context.Context, ns *NameServer, clientIP net.IP, container r
 		if _, isLocalDNS := server.(*LocalNameServer); isLocalDNS {
 			ns.PrioritizedDomain = append(ns.PrioritizedDomain, localTLDsAndDotlessDomains...)
 			ns.OriginalRules = append(ns.OriginalRules, localTLDsAndDotlessDomainsRule)
+			// The following lines is a solution to avoid core panics（rule index out of range） when setting `localhost` DNS client in config.
+			// Because the `localhost` DNS client will apend len(localTLDsAndDotlessDomains) rules into matcherInfos to match `geosite:private` default rule.
+			// But `matcherInfos` has no enough length to add rules, which leads to core panics (rule index out of range).
+			// To avoid this, the length of `matcherInfos` must be equal to the expected, so manually append it with Golang default zero value first for later modification.
+			// Related issues:
+			// https://github.com/v2fly/v2ray-core/issues/529
+			// https://github.com/v2fly/v2ray-core/issues/719
+			for i := 0; i < len(localTLDsAndDotlessDomains); i++ {
+				*matcherInfos = append(*matcherInfos, DomainMatcherInfo{
+					clientIdx:     uint16(0),
+					domainRuleIdx: uint16(0),
+				})
+			}
 		}
 
 		// Establish domain rules
@@ -102,7 +115,7 @@ func NewClient(ctx context.Context, ns *NameServer, clientIP net.IP, container r
 				rules = append(rules, domainRule.String())
 				ruleCurr++
 			}
-			err = updateDomainRule(domainRule, originalRuleIdx)
+			err = updateDomainRule(domainRule, originalRuleIdx, *matcherInfos)
 			if err != nil {
 				return newError("failed to create prioritized domain").Base(err).AtWarning()
 			}

app/dns/nameserver_doh.go

@@ -221,7 +221,7 @@ func (s *DoHNameServer) sendQuery(ctx context.Context, domain string, clientIP n
 		go func(r *dnsRequest) {
 			// generate new context for each req, using same context
 			// may cause reqs all aborted if any one encounter an error
-			dnsCtx := context.Background()
+			dnsCtx := ctx
 
 			// reserve internal dns server requested Inbound
 			if inbound := session.InboundFromContext(ctx); inbound != nil {

app/dns/nameserver_quic.go

@@ -167,7 +167,7 @@ func (s *QUICNameServer) sendQuery(ctx context.Context, domain string, clientIP
 		go func(r *dnsRequest) {
 			// generate new context for each req, using same context
 			// may cause reqs all aborted if any one encounter an error
-			dnsCtx := context.Background()
+			dnsCtx := ctx
 
 			// reserve internal dns server requested Inbound
 			if inbound := session.InboundFromContext(ctx); inbound != nil {

app/proxyman/outbound/handler.go

@@ -108,14 +108,15 @@ func NewHandler(ctx context.Context, config *core.OutboundHandlerConfig) (outbou
 		h.mux = &mux.ClientManager{
 			Enabled: h.senderSettings.MultiplexSettings.Enabled,
 			Picker: &mux.IncrementalWorkerPicker{
-				Factory: &mux.DialingWorkerFactory{
-					Proxy:  proxyHandler,
-					Dialer: h,
-					Strategy: mux.ClientStrategy{
+				Factory: mux.NewDialingWorkerFactory(
+					ctx,
+					proxyHandler,
+					h,
+					mux.ClientStrategy{
 						MaxConcurrency: config.Concurrency,
 						MaxConnection:  128,
 					},
-				},
+				),
 			},
 		}
 	}
@@ -159,7 +160,7 @@ func (h *Handler) Address() net.Address {
 // Dial implements internet.Dialer.
 func (h *Handler) Dial(ctx context.Context, dest net.Destination) (internet.Connection, error) {
 	if h.senderSettings != nil {
-		if h.senderSettings.ProxySettings.HasTag() {
+		if h.senderSettings.ProxySettings.HasTag() && !h.senderSettings.ProxySettings.TransportLayerProxy {
 			tag := h.senderSettings.ProxySettings.Tag
 			handler := h.outboundManager.GetHandler(tag)
 			if handler != nil {
@@ -196,6 +197,12 @@ func (h *Handler) Dial(ctx context.Context, dest net.Destination) (internet.Conn
 		}
 	}
 
+	if h.senderSettings != nil && h.senderSettings.ProxySettings != nil && h.senderSettings.ProxySettings.HasTag() && h.senderSettings.ProxySettings.TransportLayerProxy {
+		tag := h.senderSettings.ProxySettings.Tag
+		newError("transport layer proxying to ", tag, " for dest ", dest).AtDebug().WriteToLog(session.ExportIDToError(ctx))
+		ctx = session.SetTransportLayerProxyTagToContext(ctx, tag)
+	}
+
 	conn, err := internet.Dial(ctx, dest, h.streamSettings)
 	return h.getStatCouterConnection(conn), err
 }

app/router/condition.go

@@ -66,8 +66,8 @@ type DomainMatcher struct {
 	matchers strmatcher.IndexMatcher
 }
 
-func NewACAutomatonDomainMatcher(domains []*Domain) (*DomainMatcher, error) {
-	g := strmatcher.NewACAutomatonMatcherGroup()
+func NewMphMatcherGroup(domains []*Domain) (*DomainMatcher, error) {
+	g := strmatcher.NewMphMatcherGroup()
 	for _, d := range domains {
 		matcherType, f := matcherTypeMap[d.Type]
 		if !f {
@@ -100,7 +100,7 @@ func NewDomainMatcher(domains []*Domain) (*DomainMatcher, error) {
 }
 
 func (m *DomainMatcher) ApplyDomain(domain string) bool {
-	return len(m.matchers.Match(domain)) > 0
+	return len(m.matchers.Match(strings.ToLower(domain))) > 0
 }
 
 // Apply implements Condition.

app/router/condition_test.go

@@ -358,7 +358,7 @@ func TestChinaSites(t *testing.T) {
 
 	matcher, err := NewDomainMatcher(domains)
 	common.Must(err)
-	acMatcher, err := NewACAutomatonDomainMatcher(domains)
+	acMatcher, err := NewMphMatcherGroup(domains)
 	common.Must(err)
 
 	type TestCase struct {
@@ -399,11 +399,11 @@ func TestChinaSites(t *testing.T) {
 	}
 }
 
-func BenchmarkACDomainMatcher(b *testing.B) {
+func BenchmarkMphDomainMatcher(b *testing.B) {
 	domains, err := loadGeoSite("CN")
 	common.Must(err)
 
-	matcher, err := NewACAutomatonDomainMatcher(domains)
+	matcher, err := NewMphMatcherGroup(domains)
 	common.Must(err)
 
 	type TestCase struct {

app/router/config.go

@@ -67,11 +67,23 @@ func (rr *RoutingRule) BuildCondition() (Condition, error) {
 	conds := NewConditionChan()
 
 	if len(rr.Domain) > 0 {
-		matcher, err := NewACAutomatonDomainMatcher(rr.Domain)
-		if err != nil {
-			return nil, newError("failed to build domain condition").Base(err)
+		switch rr.DomainMatcher {
+		case "mph", "hybrid":
+			matcher, err := NewMphMatcherGroup(rr.Domain)
+			if err != nil {
+				return nil, newError("failed to build domain condition with MphDomainMatcher").Base(err)
+			}
+			newError("MphDomainMatcher is enabled for ", len(rr.Domain), "domain rules(s)").AtDebug().WriteToLog()
+			conds.Add(matcher)
+		case "linear":
+			fallthrough
+		default:
+			matcher, err := NewDomainMatcher(rr.Domain)
+			if err != nil {
+				return nil, newError("failed to build domain condition").Base(err)
+			}
+			conds.Add(matcher)
 		}
-		conds.Add(matcher)
 	}
 
 	if len(rr.UserEmail) > 0 {