Netdev3

README.md

@@ -3,17 +3,22 @@ package.  The subset maintains Go 1 compatiblity guarantee.
 
 The "net" package is modified to use netdev, TinyGo's network device driver interface.
 Netdev replaces the OS syscall interface for I/O access to the networking
-device.
+device.  See drivers/netdev for more information on netdev.
 
 #### Table of Contents
 
-- ["net" Package](#net-package)
-- [Netdev and Netlink](#netdev-and-netlink)
 - [Using "net" and "net/http" Packages](#using-net-and-nethttp-packages)
+- ["net" Package](#net-package)
+- [Maintaining "net"](#maintaining-net)
 
+## Using "net" and "net/http" Packages
+
+See README-net.md in drivers repo to more details on using "net" and "net/http"
+packages in a TinyGo application.
+
 ## "net" Package
 
-The "net" package is ported from Go 1.19.3.  The tree listings below shows the
+The "net" package is ported from Go 1.20.5.  The tree listings below shows the
 files copied.  If the file is marked with an '\*', it is copied _and_ modified
 to work with netdev.  If the file is marked with an '+', the file is new.  If
 there is no mark, it is a straight copy.
@@ -74,7 +79,7 @@ request/response handling code is intact and operational in TinyGo.  Same holds
 true for the server side.  The server side supports the normal server features
 like ServeMux and Hijacker (for websockets).
 
-### Maintaining "net"
+## Maintaining "net"
 
 As Go progresses, changes to the "net" package need to be periodically
 back-ported to TinyGo's "net" package.  This is to pick up any upstream bug
@@ -86,22 +91,45 @@ The files that are marked modified * may contain only a subset of the original
 file.  Basically only the parts necessary to compile and run the example/net
 examples are copied (and maybe modified).
 
-## Netdev and Netlink
+### Upgrade Steps
 
-Netdev is TinyGo's network device driver model.  Network drivers implement the
-netdever interface, providing a common network I/O interface to TinyGo's "net"
-package.  The interface is modeled after the BSD socket interface.  net.Conn
-implementations (TCPConn, UDPConn, and TLSConn) use the netdev interface for
-device I/O access.
+Let's define some versions:
 
-Network drivers also (optionally) implement the Netlinker interface.  This
-interface is not used by TinyGo's "net" package, but rather provides the TinyGo
-application direct access to the network device for common settings and control
-that fall outside of netdev's socket interface.
+MIN = TinyGo minimum Go version supported (e.g. 1.15)
+CUR = TinyGo "net" current version (e.g. 1.20.5)
+UPSTREAM = Latest upstream Go version to upgrade to (e.g. 1.21)
+NEW = TinyGo "net" new version, after upgrade
 
-See the README-net.md in drivers repo for more details on netdev and netlink.
+In example, we'll upgrade from CUR (1.20.5) to UPSTREAM (1.21).
 
-## Using "net" and "net/http" Packages
+These are the steps to promote TinyGos "net" to latest Go upstream version.
+These steps should be done when:
 
-See README-net.md in drivers repo to more details on using "net" and "net/http"
-packages in a TinyGo application.
+- MIN moved forward
+- TinyGo major release
+- TinyGo minor release to pick up security fixes in UPSTREAM
+
+Step 1:
+
+Backport differences from Go UPSTREAM to Go CUR.  Since TinyGo CUR isn't the
+full Go "net" implementation, only backport differences, don't add new stuff
+from UPSTREAM (unless it's needed in the NEW release).
+
+	NEW = CUR + diff(CUR, UPSTREAM)
+
+If NEW contains updates not compatible with MIN, then NEW will need to revert
+just those updates back to the CUR version, and annotate with a TINYGO comment.
+If MIN moves forord, NEW can pull in the UPSTREAM changes.
+
+Step 2:
+
+As a double check, compare NEW against UPSTREAM.  The only differences at this
+point should be excluded (not ported) code from UPSTREAM that wasn't in CUR in
+the first place, and differences due to changes held back for MIN support.
+
+Step 3:
+
+Test NEW against example/net examples.  If everything checks out, then CUR
+becomes NEW, and we can push to TinyGo.
+
+	CUR = NEW

dial.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
+// TINYGO: The following is copied and modified from Go 1.20.5 official implementation.
 
 // TINYGO: Omit DualStack support
 // TINYGO: Omit Fast Fallback support

http/client.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
+// TINYGO: The following is copied and modified from Go 1.20.5 official implementation.
 
 // Copyright 2009 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/clone.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2019 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/cookie.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
+// TINYGO: The following is copied and modified from Go 1.20.5 official implementation.
 
 // Copyright 2009 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/fs.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2009 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style
@@ -71,6 +71,8 @@ func mapOpenError(originalErr error, name string, sep rune, stat func(string) (f
 // Open implements FileSystem using os.Open, opening files for reading rooted
 // and relative to the directory d.
 func (d Dir) Open(name string) (File, error) {
+	// TINYGO: internal/safefilepath isn't avail until 1.20, so keep pre 1.20
+	// TINYGO: code here until TinyGo min Go version is >= 1.20.
 	if filepath.Separator != '/' && strings.ContainsRune(name, filepath.Separator) {
 		return nil, errors.New("http: invalid character in file path")
 	}
@@ -256,81 +258,95 @@ func serveContent(w ResponseWriter, r *Request, name string, modtime time.Time,
 		Error(w, err.Error(), StatusInternalServerError)
 		return
 	}
+	if size < 0 {
+		// Should never happen but just to be sure
+		Error(w, "negative content size computed", StatusInternalServerError)
+		return
+	}
 
 	// handle Content-Range header.
 	sendSize := size
 	var sendContent io.Reader = content
-	if size >= 0 {
-		ranges, err := parseRange(rangeReq, size)
-		if err != nil {
-			if err == errNoOverlap {
-				w.Header().Set("Content-Range", fmt.Sprintf("bytes */%d", size))
-			}
+	ranges, err := parseRange(rangeReq, size)
+	switch err {
+	case nil:
+	case errNoOverlap:
+		if size == 0 {
+			// Some clients add a Range header to all requests to
+			// limit the size of the response. If the file is empty,
+			// ignore the range header and respond with a 200 rather
+			// than a 416.
+			ranges = nil
+			break
+		}
+		w.Header().Set("Content-Range", fmt.Sprintf("bytes */%d", size))
+		fallthrough
+	default:
+		Error(w, err.Error(), StatusRequestedRangeNotSatisfiable)
+		return
+	}
+
+	if sumRangesSize(ranges) > size {
+		// The total number of bytes in all the ranges
+		// is larger than the size of the file by
+		// itself, so this is probably an attack, or a
+		// dumb client. Ignore the range request.
+		ranges = nil
+	}
+	switch {
+	case len(ranges) == 1:
+		// RFC 7233, Section 4.1:
+		// "If a single part is being transferred, the server
+		// generating the 206 response MUST generate a
+		// Content-Range header field, describing what range
+		// of the selected representation is enclosed, and a
+		// payload consisting of the range.
+		// ...
+		// A server MUST NOT generate a multipart response to
+		// a request for a single range, since a client that
+		// does not request multiple parts might not support
+		// multipart responses."
+		ra := ranges[0]
+		if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
 			Error(w, err.Error(), StatusRequestedRangeNotSatisfiable)
 			return
 		}
-		if sumRangesSize(ranges) > size {
-			// The total number of bytes in all the ranges
-			// is larger than the size of the file by
-			// itself, so this is probably an attack, or a
-			// dumb client. Ignore the range request.
-			ranges = nil
-		}
-		switch {
-		case len(ranges) == 1:
-			// RFC 7233, Section 4.1:
-			// "If a single part is being transferred, the server
-			// generating the 206 response MUST generate a
-			// Content-Range header field, describing what range
-			// of the selected representation is enclosed, and a
-			// payload consisting of the range.
-			// ...
-			// A server MUST NOT generate a multipart response to
-			// a request for a single range, since a client that
-			// does not request multiple parts might not support
-			// multipart responses."
-			ra := ranges[0]
-			if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
-				Error(w, err.Error(), StatusRequestedRangeNotSatisfiable)
-				return
-			}
-			sendSize = ra.length
-			code = StatusPartialContent
-			w.Header().Set("Content-Range", ra.contentRange(size))
-		case len(ranges) > 1:
-			sendSize = rangesMIMESize(ranges, ctype, size)
-			code = StatusPartialContent
-
-			pr, pw := io.Pipe()
-			mw := multipart.NewWriter(pw)
-			w.Header().Set("Content-Type", "multipart/byteranges; boundary="+mw.Boundary())
-			sendContent = pr
-			defer pr.Close() // cause writing goroutine to fail and exit if CopyN doesn't finish.
-			go func() {
-				for _, ra := range ranges {
-					part, err := mw.CreatePart(ra.mimeHeader(ctype, size))
-					if err != nil {
-						pw.CloseWithError(err)
-						return
-					}
-					if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
-						pw.CloseWithError(err)
-						return
-					}
-					if _, err := io.CopyN(part, content, ra.length); err != nil {
-						pw.CloseWithError(err)
-						return
-					}
+		sendSize = ra.length
+		code = StatusPartialContent
+		w.Header().Set("Content-Range", ra.contentRange(size))
+	case len(ranges) > 1:
+		sendSize = rangesMIMESize(ranges, ctype, size)
+		code = StatusPartialContent
+
+		pr, pw := io.Pipe()
+		mw := multipart.NewWriter(pw)
+		w.Header().Set("Content-Type", "multipart/byteranges; boundary="+mw.Boundary())
+		sendContent = pr
+		defer pr.Close() // cause writing goroutine to fail and exit if CopyN doesn't finish.
+		go func() {
+			for _, ra := range ranges {
+				part, err := mw.CreatePart(ra.mimeHeader(ctype, size))
+				if err != nil {
+					pw.CloseWithError(err)
+					return
 				}
-				mw.Close()
-				pw.Close()
-			}()
-		}
+				if _, err := content.Seek(ra.start, io.SeekStart); err != nil {
+					pw.CloseWithError(err)
+					return
+				}
+				if _, err := io.CopyN(part, content, ra.length); err != nil {
+					pw.CloseWithError(err)
+					return
+				}
+			}
+			mw.Close()
+			pw.Close()
+		}()
+	}
 
-		w.Header().Set("Accept-Ranges", "bytes")
-		if w.Header().Get("Content-Encoding") == "" {
-			w.Header().Set("Content-Length", strconv.FormatInt(sendSize, 10))
-		}
+	w.Header().Set("Accept-Ranges", "bytes")
+	if w.Header().Get("Content-Encoding") == "" {
+		w.Header().Set("Content-Length", strconv.FormatInt(sendSize, 10))
 	}
 
 	w.WriteHeader(code)
@@ -433,6 +449,7 @@ func checkIfUnmodifiedSince(r *Request, modtime time.Time) condResult {
 	// The Last-Modified header truncates sub-second precision so
 	// the modtime needs to be truncated too.
 	modtime = modtime.Truncate(time.Second)
+	// TINYGO: time.Compare not until Go 1.20
 	if modtime.Before(t) || modtime.Equal(t) {
 		return condTrue
 	}
@@ -484,6 +501,7 @@ func checkIfModifiedSince(r *Request, modtime time.Time) condResult {
 	// The Last-Modified header truncates sub-second precision so
 	// the modtime needs to be truncated too.
 	modtime = modtime.Truncate(time.Second)
+	// TINYGO: time.Compare not until Go 1.20
 	if modtime.Before(t) || modtime.Equal(t) {
 		return condFalse
 	}
@@ -644,7 +662,6 @@ func serveFile(w ResponseWriter, r *Request, fs FileSystem, name string, redirec
 			defer ff.Close()
 			dd, err := ff.Stat()
 			if err == nil {
-				name = index
 				d = dd
 				f = ff
 			}
@@ -820,6 +837,7 @@ func (f ioFile) Readdir(count int) ([]fs.FileInfo, error) {
 
 // FS converts fsys to a FileSystem implementation,
 // for use with FileServer and NewFileTransport.
+// The files provided by fsys must implement io.Seeker.
 func FS(fsys fs.FS) FileSystem {
 	return ioFS{fsys}
 }

http/header.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
+// TINYGO: The following is copied and modified from Go 1.20.5 official implementation.
 
 // TINYGO: Removed trace stuff
 

http/http.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2016 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/internal/ascii/print.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2021 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/internal/ascii/print_test.go

@@ -1,3 +1,5 @@
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
+
 // Copyright 2021 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.

http/internal/chunked.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2009 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/internal/chunked_test.go

@@ -1,3 +1,5 @@
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
+
 // Copyright 2011 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.

http/jar.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied and modified from Go 1.19.3 official implementation.
+// TINYGO: The following is copied and modified from Go 1.20.5 official implementation.
 
 // Copyright 2011 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style

http/method.go

@@ -1,4 +1,4 @@
-// TINYGO: The following is copied from Go 1.19.3 official implementation.
+// TINYGO: The following is copied from Go 1.20.5 official implementation.
 
 // Copyright 2015 The Go Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style