Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在工单sql语句检查中增加对GO关键词的处理,使用GO作为关键字切分sql再执行通过 #263

Merged
merged 6 commits into from
Jun 13, 2019
Merged

在工单sql语句检查中增加对GO关键词的处理,使用GO作为关键字切分sql再执行通过 #263

merged 6 commits into from
Jun 13, 2019

Conversation

AceAttorney
Copy link
Contributor

在sqlserver2008和2016上测试通过。

@codecov
Copy link

codecov bot commented Jun 10, 2019

Codecov Report

Merging #263 into master will increase coverage by 0.1%.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master    #263     +/-   ##
========================================
+ Coverage    83.1%   83.2%   +0.1%     
========================================
  Files          65      65             
  Lines        8231    8251     +20     
========================================
+ Hits         6840    6865     +25     
+ Misses       1391    1386      -5
Impacted Files Coverage Δ
sql/engines/mssql.py 84.17% <100%> (+4.72%) ⬆️
sql/engines/tests.py 99.76% <100%> (ø) ⬆️
sql/sql_tuning.py 95.65% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4be082e...9865e6c. Read the comment docs.

@LeoQuote
Copy link
Collaborator

感谢贡献,因为单元测试覆盖率稍微有些降低,如果方便的话可不可以补充一下测试,看了下主要是execute_check 方法缺少测试,基本上是不需要mock,只需要构造对象,然后简单的用几个关于包含 go 的语句调用方法,之后再加上若干个断言即可。

如果暂时没空的话我近期会帮你补上测试。

@AceAttorney
Copy link
Contributor Author

@LeoQuote 好的,我试试看,我这个代码还有点问题,一会把go改成正则匹配大小写的

@LeoQuote LeoQuote added the enhancement New feature or request label Jun 11, 2019
@AceAttorney
Copy link
Contributor Author

@LeoQuote 咨询个的问题
单元测试写差不多了,但是断言的时候遇到些问题,就是我在单元测试外部直接调用execute_check的时候,可以把sql按我想法用go拆开,返回正确的结果。
当我在单元测试里调用execute_check时,就不能正确拆开了。导致断言时判断不过。

# 直接调用execute_check 时的代码

def execute_check(db_name=None, sql=''):
    """上线单执行前的检查, 返回Review set"""
    check_result = ReviewSet(full_sql=sql)
    # 切分语句,追加到检测结果中,默认全部检测通过
    t = re.compile('^GO$', re.I | re.M)
    sql = re.split(t, sql)
    sql = filter(None,sql)
    split_sql = [f"""use [dbname]"""]
    for i in sql:
        split_sql = split_sql + [i]
    rowid = 1
    for statement in split_sql:
        check_result.rows.append(ReviewResult(
            id=rowid,
            errlevel=0,
            stagestatus='Audit completed',
            errormessage='None',
            sql=statement,
            affected_rows=0,
            execute_time=0, ))
        rowid += 1
    return check_result



test_sql = '''use database
go

some sql1
GO
some sql2

Go
some sql3
gO
'''


check_result = execute_check(db_name=None, sql=test_sql)

num = 1
for i in check_result.rows:
    print ("this is sql %s" % num) 
    print (i.__dict__['sql'])
    num += 1

# 直接调用execute_check 时的返回

this is sql 1
use [dbname]
this is sql 2
use database

this is sql 3


some sql1

this is sql 4

some sql2


this is sql 5

some sql3

this is sql 6

然后我把test_sql当作测试数据放在单元测试里

# 单元测试调用的代码

class TestMssql(unittest.TestCase):
    def test_execute_check(self):
        test_sql = '''use database
        go
        
        some sql1
        GO
        some sql2

        Go
        some sql3
        gO
        '''
        check_result = execute_check(db_name=None, sql=test_sql)
        num = 1
        for i in check_result.rows:
            print ("this is sql %s" % num) 
            print (i.__dict__['sql'])
            num += 1
        self.assertIsInstance(check_result, ReviewSet)
        self.assertEqual(check_result.rows[1].__dict__['sql'], 'use database')



if __name__ == '__main__':
    unittest.main()




# 单元测试内调用的返回

this is sql 1
use [dbname]
this is sql 2
use database
        go
        
        some sql1
        GO
        some sql2

        Go
        some sql3
        gO
        
F
======================================================================
FAIL: test_execute_check (__main__.TestMssql)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\aozakiaoko\Desktop\tempCodeRunnerFile.py", line 187, in test_execute_check
    self.assertEqual(check_result.rows[1].__dict__['sql'], 'use database')
AssertionError: 'use database\n        go\n        \n        some sql1[82 chars]    ' != 'use database'
- use database
?             -
+ use database-         go
-         
-         some sql1
-         GO
-         some sql2
- 
-         Go
-         some sql3
-         gO
-         

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

测了很久,还是不能通过,麻烦能给看看么?

@AceAttorney
Copy link
Contributor Author

好吧 找到原因了。。。单元测试里必须用单行字符串配合\n \r才能使用

@hhyo
Copy link
Owner

hhyo commented Jun 13, 2019

上面导致断言失败是因为对比的是字符串,可以在对比时strip

@hhyo hhyo requested a review from LeoQuote June 13, 2019 11:51
@AceAttorney
Copy link
Contributor Author

上面导致断言失败是因为对比的是字符串,可以在对比时strip

其实是我自己字符串格式没写对 函数内多行字符串换行时候 需要顶格写。。。而不是遵循python的tab缩进。

# 多行正确写法

class TestMssql(unittest.TestCase):
    def test_execute_check(self):
        test_sql = '''use database
go
        
some sql1
GO
some sql2

Go
some sql3
gO'''
        check_result = execute_check(db_name=None, sql=test_sql)
        num = 1
        for i in check_result.rows:
            print ("this is sql %s" % num) 
            print (i.__dict__['sql'])
            num += 1
        self.assertIsInstance(check_result, ReviewSet)
        self.assertEqual(check_result.rows[1].__dict__['sql'], 'use database')

这样写其实可以过,就是太丑了,所以我换成单行的了

Copy link
Collaborator

@LeoQuote LeoQuote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

很棒!

sql/engines/mssql.py Outdated Show resolved Hide resolved
@LeoQuote
Copy link
Collaborator

稍等我帮你rebase调整一下commit信息就可以merge了

@LeoQuote LeoQuote changed the title 修改pyodbc不识别GO语句,使用GO作为关键字切分sql再执行通过. 在工单sql语句检查中增加对GO关键词的处理,使用GO作为关键字切分sql再执行通过 Jun 13, 2019
@LeoQuote LeoQuote merged commit a4ce8ed into hhyo:master Jun 13, 2019
@AceAttorney AceAttorney deleted the featrue/pyodbc-go-syntax branch June 14, 2019 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants