Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Couldn't flush line :: Error: Failed to retrieve directory listing #1158

Open
mateors opened this issue Oct 19, 2023 · 8 comments
Open

Couldn't flush line :: Error: Failed to retrieve directory listing #1158

mateors opened this issue Oct 19, 2023 · 8 comments

Comments

@mateors
Copy link

mateors commented Oct 19, 2023

I am unable to connect with ftpserver what's wrong?
Server log:

root@mostain:~/ftps# ./ftpserver
level=info version= date= commit= event="FTP server"
level=info component=server address=[::]:2121 event=Listening...
level=info component=server event=Starting...
level=debug component=server clientId=1 clientIp=103.124.226.98:61928 event="Client connected"
level=info component=driver clientId=1 remoteAddr=103.124.226.98:61928 nbClients=1 event="Client connected"
level=info component=driver clientId=1 remoteAddr=103.124.226.98:61928 nbClients=0 event="Client disconnected"
level=debug component=server clientId=1 clientIp=103.124.226.98:61928 event="Client disconnected"
level=debug component=server clientId=2 clientIp=103.124.226.98:61940 event="Client connected"
level=info component=driver clientId=2 remoteAddr=103.124.226.98:61940 nbClients=1 event="Client connected"
level=warn component=server clientId=1 error="accept tcp [::]:2125: i/o timeout" event="Unable to open transfer"
level=warn component=server clientId=1 err="write tcp 209.126.12.223:2121->103.124.226.98:61928: use of closed network connection" event="Couldn't flush line"
level=info component=driver clientId=2 remoteAddr=103.124.226.98:61940 nbClients=0 event="Client disconnected"
level=debug component=server clientId=2 clientIp=103.124.226.98:61940 event="Client disconnected"
level=warn component=server clientId=2 error="accept tcp [::]:2126: i/o timeout" event="Unable to open transfer"
level=warn component=server clientId=2 err="write tcp 209.126.12.223:2121->103.124.226.98:61940: use of closed network connection" event="Couldn't flush line"

FileZilla client log:

Status:	Connecting to 209.126.12.223:2121...
Status:	Connection established, waiting for welcome message...
Response:	220 ftpserver
Command:	AUTH TLS
Response:	550 Cannot get a TLS config: not enabled
Command:	AUTH SSL
Response:	550 Cannot get a TLS config: not enabled
Status:	Insecure server, it does not support FTP over TLS.
Command:	USER test
Response:	331 OK
Command:	PASS ****
Response:	230 Password ok, continue
Command:	SYST
Response:	215 UNIX Type: L8
Command:	FEAT
Response:	211- These are my features
Response:	 CLNT
Response:	 UTF8
Response:	 SIZE
Response:	 MDTM
Response:	 REST STREAM
Response:	 EPRT
Response:	 EPSV
Response:	 MLSD
Response:	 MLST
Response:	 MFMT
Response:	211 end
Command:	CLNT FileZilla
Response:	200 Good to know
Command:	OPTS UTF8 ON
Response:	200 I'm in UTF8 only anyway
Status:	Logged in
Status:	Retrieving directory listing...
Command:	PWD
Response:	257 "/" is the current directory
Command:	TYPE I
Response:	200 Type set to binary
Command:	PASV
Response:	227 Entering Passive Mode (209,126,12,223,8,77)
Command:	MLSD
Error:	Connection timed out after 20 seconds of inactivity
Error:	Failed to retrieve directory listing
Status:	Disconnected from server
Status:	Connecting to 209.126.12.223:2121...
Status:	Connection established, waiting for welcome message...
Status:	Insecure server, it does not support FTP over TLS.
Status:	Logged in
Status:	Retrieving directory listing...
Command:	PWD
Response:	257 "/" is the current directory
Command:	TYPE I
Response:	200 Type set to binary
Command:	PASV
Response:	227 Entering Passive Mode (209,126,12,223,8,78)
Command:	MLSD
Error:	Connection timed out after 20 seconds of inactivity
Error:	Failed to retrieve directory listing
@mateors mateors changed the title Couldn't flush line Couldn't flush line :: Error: Failed to retrieve directory listing Oct 19, 2023
@jskorlol
Copy link

jskorlol commented Jan 5, 2024

I am also experiencing the same issue. Have you resolved it? You cannot connect through an external IP in Docker

@mateors
Copy link
Author

mateors commented Jan 5, 2024

I am also experiencing the same issue. Have you resolved it? You cannot connect through an external IP in Docker

Yes i have resolved it by myself.

@jskorlol
Copy link

jskorlol commented Jan 5, 2024

Would you be able to share the method with me?

@mateors
Copy link
Author

mateors commented Jan 6, 2024

Would you be able to share the method with me?

sure I will, let me check my code first, I already used it in one of my product lxroot.com

@mateors
Copy link
Author

mateors commented Jan 6, 2024

Would you be able to share the method with me?

package main

import (
	"encoding/json"
	"flag"
	"fmt"
	"io"
	"log"
	"os"
	"os/signal"
	"path/filepath"
	"strings"
	"syscall"
	"time"

	ftpserver "github.com/fclairamb/ftpserverlib"
	gkwrap "github.com/fclairamb/go-log/gokit"
	gokit "github.com/go-kit/log"
	_ "github.com/go-sql-driver/mysql"
	"github.com/joho/godotenv"
	"github.com/mateors/msql"
	"github.com/mateors/mtool"

	"github.com/fclairamb/ftpserver/config"
	"github.com/fclairamb/ftpserver/server"
)

var (
	ftpServer *ftpserver.FtpServer
	driver    *server.Server
)

func init() {

	err := godotenv.Load(envFilePath())
	if err != nil {
		log.Fatal("Error loading .env file", err)
	}
	HOST = os.Getenv("HOST")
	DBPORT = os.Getenv("DBPORT")
	DBNAME = os.Getenv("DATABASE")
	DBUSER = os.Getenv("DBUSER")
	DBPASS = os.Getenv("DBPASS")
	ENCDECPASS = os.Getenv("ENCDECPASS")
	dataSourceName := fmt.Sprintf("%s:%s@tcp(%s:%s)/%s", DBUSER, DBPASS, HOST, DBPORT, DBNAME)

	//database
	db, err := getDbConn(dataSourceName)
	if err != nil {
		log.Println(err)
		return
	}
	var nrows = make([]map[string]interface{}, 0)
	sql := fmt.Sprintln("SELECT id,domain,name,passwd,dirname FROM ftp_user WHERE protocol='ftp' AND status=1;")
	rows, err := msql.GetAllRowsByQuery(sql, db)
	if err != nil {
		log.Println(err)
		return
	}
	for _, row := range rows {
		pass := mtool.DecodeStr(row["passwd"].(string), ENCDECPASS)
		nrow := make(map[string]interface{})
		nrow["user"] = row["name"]
		nrow["pass"] = mtool.HashBcrypt(pass)
		nrow["fs"] = "os"
		domain := row["domain"].(string)
		basePath := fmt.Sprintf("/var/www/vhosts/%s/public_html", domain)
		dirName := row["dirname"].(string)
		if dirName != "" {
			basePath = filepath.Join(basePath, dirName)
			err = os.MkdirAll(basePath, 0755)
			log.Println("Mkdir:", err, basePath)
		}
		nrow["params"] = map[string]interface{}{"basePath": basePath}
		fmt.Println(row["name"], pass)
		nrows = append(nrows, nrow)
	}
	//generate json file
	var fmap = make(map[string]interface{})
	logging := ftpSettings(db)
	prange := logging["passive_transfer_port_range"].(string)
	delete(logging, "passive_transfer_port_range")
	fmap["version"] = 1
	fmap["listen_address"] = ":21"
	fmap["tls"] = tlsCert()
	fmap["accesses"] = nrows
	if logging != nil {
		fmap["logging"] = logging
	}
	var start, end int
	slc := strings.Split(prange, ":")
	if len(slc) == 2 {
		start = strToint(slc[0])
		end = strToint(slc[1])
	}
	fmap["passive_transfer_port_range"] = map[string]int{"start": start, "end": end}
	bs, err := json.Marshal(fmap)
	if err != nil {
		log.Println(err)
	}
	content := string(bs)
	err = FileCreate("lxftpconfig.json", content)
	if err != nil {
		log.Println(err)
	}
}

// ufw allow 2121:2130/tcp
func main() {

	// Arguments vars
	var confFile string
	var onlyConf bool

	// Parsing arguments
	flag.StringVar(&confFile, "conf", "", "Configuration file")
	flag.BoolVar(&onlyConf, "conf-only", false, "Only create the conf")
	flag.Parse()

	// Setting up the logger
	logger := gkwrap.New()
	logger.Info("FTP server", "version", BuildVersion, "date", BuildDate, "commit", Commit)
	autoCreate := onlyConf

	// The general idea here is that if you start it without any arg, you're probably doing a local quick&dirty run
	// possibly on a windows machine, so we're better of just using a default file name and create the file.
	if confFile == "" {
		confFile = "lxftpconfig.json" //ftpserver.json
		autoCreate = false
	}

	if autoCreate {

		if _, err := os.Stat(confFile); err != nil && os.IsNotExist(err) {
			logger.Warn("No conf file, creating one", "confFile", confFile)
			if err := os.WriteFile(confFile, confFileContent(), 0600); err != nil { //nolint: gomnd
				logger.Warn("Couldn't create conf file", "confFile", confFile)
			}
		}
	}

	conf, errConfig := config.NewConfig(confFile, logger)
	if errConfig != nil {
		logger.Error("Can't load conf", "err", errConfig)
		return
	}

	// Now is a good time to open a logging file
	if conf.Content.Logging.File != "" {

		writer, err := os.OpenFile(conf.Content.Logging.File, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0600) //nolint:gomnd
		if err != nil {
			logger.Error("Can't open log file", "err", err)
			return
		}
		logger = gkwrap.NewWrap(gokit.NewLogfmtLogger(io.MultiWriter(writer, os.Stdout))).With(
			"ts", gokit.DefaultTimestampUTC,
			"caller", gokit.DefaultCaller,
		)
	}

	// Loading the driver
	var errNewServer error
	driver, errNewServer = server.NewServer(conf, logger.With("component", "driver"))

	if errNewServer != nil {
		logger.Error("Could not load the driver", "err", errNewServer)
		return
	}

	// Instantiating the server by passing our driver implementation
	ftpServer = ftpserver.NewFtpServer(driver)

	// Overriding the server default silent logger by a sub-logger (component: server)
	ftpServer.Logger = logger.With("component", "server")

	// Preparing the SIGTERM handling
	go signalHandler()

	// Blocking call, behaving similarly to the http.ListenAndServe
	if onlyConf {
		logger.Warn("Only creating conf")
		return
	}

	//remove config file
	err := os.Remove("lxftpconfig.json")
	if err != nil {
		log.Println("unable to remove lxftpconfig.json", err)
	}

	if err = ftpServer.ListenAndServe(); err != nil {
		logger.Error("Problem listening", "err", err)
	}

	// We wait at most 1 minutes for all clients to disconnect
	if err := driver.WaitGracefully(time.Minute); err != nil {
		ftpServer.Logger.Warn("Problem stopping server", "err", err)
	}
}

func stop() {

	driver.Stop()
	if err := ftpServer.Stop(); err != nil {
		ftpServer.Logger.Error("Problem stopping server", "err", err)
	}
}

func signalHandler() {
	ch := make(chan os.Signal, 1)
	signal.Notify(ch, syscall.SIGTERM)

	for {
		sig := <-ch
		if sig == syscall.SIGTERM {
			stop()
			break
		}
	}
}

func confFileContent() []byte {

	str := `
	{
		"version": 1,
		"listen_address": ":21",
		"logging": {
		  "ftp_exchanges": true,
		  "file_accesses": true,
		  "file": "ftpserver.log"
		},
		"tls": {
		  "server_cert": {
			 "cert": "/mnt/e/project/mastrhost/frontend/cert/host.lxroot.local+2.pem",
			 "key": "/mnt/e/project/mastrhost/frontend/cert/host.lxroot.local+2-key.pem"
		  }
	   },
		"accesses": [
		  {
			"user": "mostainlocal",
			"pass": "$2a$14$SAUy.Gj2vVXxMvBf7WHFBOqEoQQ8KognVvJT50aFy740avFx9dDvO",
			"fs": "os",
			"params": {
			  "basePath": "/var/www/vhosts/mostain.local/public_html"
			}
		  },
		  {
			"user": "sftp",
			"pass": "sftp",
			"fs": "sftp",
			"params": {
			   "username": "mostainlocal",
			   "password": "=9oocWwiQPdv",
			   "hostname": "172.21.131.5:22"
			}
		 }
		],
		"passive_transfer_port_range": {
		  "start": 2122,
		  "end": 2130
		}
	}
	`
	return []byte(str)
}

@jskorlol
Copy link

jskorlol commented Jan 6, 2024

@mateors It's really amazing!! I'll give it a try, thank you so much for sharing!! Have a great New Year!

@mateors
Copy link
Author

mateors commented Jan 6, 2024

@mateors It's really amazing!! I'll give it a try, thank you so much for sharing!! Have a great New Year!
You are welcome bro, don't forget to reach out if you face any trouble, I am a passionate golang developer and making a developer friendly product, use my lxroot.com and give me your feedback

@terinjokes
Copy link

@mateors Are you able to share more details on what you had to change? You've pasted a bunch of Go code without a clue as to what you needed to debug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants