Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

支持对scan后rows.Err()检查 #144

Merged
merged 1 commit into from
Aug 10, 2023
Merged

Conversation

winston-ke
Copy link
Contributor

ctx超时或取消情况下,scanner.Scan()方法会把err吞掉,不返回err,导致读出的数据不全

@caibirdme
Copy link
Contributor

我没太理解这里的问题,在什么情况下for loop都执行完了,rows.Err()会报错?即使报错,这时候数据也读完了吧,怎么会不完整呢

@winston-ke
Copy link
Contributor Author

winston-ke commented Aug 9, 2023

版本:github.com/didi/gendry@v1.3.2
这个在线上环境是可以复现的,for loop执行完成后可能会出现数据的不完整

数据可能会不完整的原因:
数据读到mysqlConn.buffer后会进行for循环scan, 在读入buffer后ctx被cancel或timeout, row.Next()中会判断err, 如果err!=nil&&er!=io.EOF, 会返回false直接退出for循环,此时就会出现返回的result为空或不完整的情况

代码路径:
rows.Next()指定的函数为go官方源码go/src/datrabase/sql/sql.go #L2983
Next()方法中会对rs.lasterr进行判断,如果此时(1)执行完成后ctx被cancle, rs.lasterr !=nil && rs.laterr != io.EOF, Next()会在3021行返回false,for循环会直接跳出并返回空的result, 并且err为nil

func resolveDataFromRows(rows Rows) ([]map[string]interface{}, error) {
        (1)go-sql-driver请求数据库将数据放入buffer
        columns, err := rows.Columns()
        (2)for循环scan数据
	for rows.Next() {
		// fmt.Println(rows.)
		err = rows.Scan(values...)
		if nil != err {
			return nil, err
		}
	}
	return result, nil
}

两种复现方式:
1、在github.com/didi/gendry@v1.3.2/scanner/scanner.go的 319行后面加上time.Sleep(time.Second*10), 用postman发起请求后,点击cancle, 此时scanner.Scan()方法返回的err为空,且数据不全

2、接口设置一个比较小的超时时间,恰好在在请求数据后,for循环scan过程中ctx 被cancel, 此时也会出现数据返回不完整的情况

@caibirdme caibirdme merged commit b1ded9c into didi:master Aug 10, 2023
1 check passed
@caibirdme
Copy link
Contributor

确认,感谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants