Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial import of CCL Unit Mocking. #7

Merged
merged 10 commits into from Feb 15, 2019
328 changes: 13 additions & 315 deletions cclunit-framework-source/doc/CCLUTGUIDANCE.md

Large diffs are not rendered by default.

22 changes: 17 additions & 5 deletions cclunit-framework-source/doc/CCLUTMOCKING.md
Expand Up @@ -171,15 +171,27 @@ call cclutRemoveAllMocks(null)

2. cclutRemoveAllMocks should be called as part of the teardown for all tests. The framework will attempt to clean up any outstanding mocks, but it is good practice to explicitly remove any mocks to ensure that no mocked tables remain in the Oracle instance.

3. Tables can be mocked even if the table does not exist in the domain where the test is run. The mocked version will be used when executing programs with executeProgramWithMocks. This can be useful to test tables that do not formally exist yet.
3. Namespaces are not supported when using cclutAddMockImplementation. An example of this would be:

4. The mocked items created through cclutCreateMockTable() and cclutAddMockImplementation will not be applied to child scripts called from the script-under-test. Some alternatives would be to mock the child script to return the appropriate data or to mock the child script to execute the real script applying the mocked tables and implementations.
```
call cclutAddMockImplementation("public::testSubroutine", "myNamespace::mockSubroutine")
```

5. The mocked items created through cclutCreateMockTable() and cclutAddMockImplementation will not be applied to statements executed through "call parser()" commands. One alternative would be to separate the parser string generation into separate subroutines and mock the subroutines to return parser strings using the mocked entity names. Another alternative would be to mock the parser() call to validate the correct information is supplied, then perform the appropriate mock versions of the actions the statement would normally perform.
Alternatives will depend on the specifics of the script-under-test, but one alternative for the example above would be to define the mock implementation under the namespace that the program uses (i.e. public::mockSubroutine) and exclude the namespaces when adding the mock:

```
call cclutAddMockImplementation("testSubroutine", "mockSubroutine")
```

6. The table mocking APIs are not supported when called from within a reportwriter section. It might be tempting to use a dummyt query to set up mock data from a record structure, but various mocking calls such as cclutCreateMockTable, cclutRemoveMockTable and cclutAddMockData cannot be executed within the context of a query (because the implementations execute queries). Use a for loop instead.
4. Tables can be mocked even if the table does not exist in the domain where the test is run. The mocked version will be used when executing programs with executeProgramWithMocks. This can be useful to test tables that do not formally exist yet.

5. The mocked items created through cclutCreateMockTable() and cclutAddMockImplementation will not be applied to child scripts called from the script-under-test. Some alternatives would be to mock the child script to return the appropriate data or to mock the child script to execute the real script applying the mocked tables and implementations.

6. The mocked items created through cclutCreateMockTable() and cclutAddMockImplementation will not be applied to statements executed through "call parser()" commands. One alternative would be to separate the parser string generation into separate subroutines and mock the subroutines to return parser strings using the mocked entity names. Another alternative would be to mock the parser() call to validate the correct information is supplied, then perform the appropriate mock versions of the actions the statement would normally perform.

7. The table mocking APIs are not supported when called from within a reportwriter section. It might be tempting to use a dummyt query to set up mock data from a record structure, but various mocking calls such as cclutCreateMockTable, cclutRemoveMockTable and cclutAddMockData cannot be executed within the context of a query (because the implementations execute queries). Use a for loop instead.

7. Mocking the tdbexecute "reply_to" entity is unsupported under certain conditions, specifically if a call to free the "reply_to" entity is made just prior to calling tdbexecute. If the scenario is truly necessary for a test, the best alternative is to define and use a subroutine for freeing the "reply_to" entity within the script and use a mock for that subroutine which does not actually perform the freeing of the "reply_to" entity.
8. Mocking the tdbexecute "reply_to" entity is unsupported under certain conditions, specifically if a call to free the "reply_to" entity is made just prior to calling tdbexecute. If the scenario is truly necessary for a test, the best alternative is to define and use a subroutine for freeing the "reply_to" entity within the script and use a mock for that subroutine which does not actually perform the freeing of the "reply_to" entity.

## Example
Below is an example of some of the APIs available in the CCL Unit Mocking framework along with some simple notes.
Expand Down
69 changes: 69 additions & 0 deletions cclunit-framework-source/doc/examples/basic_example.inc
@@ -0,0 +1,69 @@
;; this is the test
;;; public::mainForSubroutineB is the workhorse. It will be called instead of public::main because of "with replace"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move line 2 into (at the end of) the documentation for testSubroutineB.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing.


/**
* Confirms that subroutineB returns "subroutineBTest" when it is passed "Test"
* and "subroutineBSubroutine" when it is passed "Subroutine".
*/
subroutine (testSubroutineB(null) = null)

execute the_script:dba with replace("MAIN", MAINFORSUBROUTINEB)

end

subroutine (public::mainForSubroutineB(null) = null)
declare b_string = vc with protect, noconstant("")

set b_string = subroutineB("Test")
set stat = cclutAssertVCEqual(CURREF, "testSubroutineB Test", b_string, "subroutineBTest")

set b_string = subroutineB("Subroutine")
set stat = cclutAssertVCEqual(CURREF, "testSubroutineB Subroutine", b_string, "subroutineBSubroutine")
end;;;mainForSubroutineB

;; this is the script to be tested

drop program the_script:dba go
create program the_script:dba

record reply (
%i cclsource:status_block.inc
)

subroutine (PUBLIC::subroutineA(id = f8) = vc)
call echo("subroutineA")
;;; do stuff
return("subroutineA")
end

subroutine (PUBLIC::subroutineB(name = vc) = vc)
call echo("subroutineB")
;;; do stuff
return(concat("subroutineB", name))
end

subroutine (PUBLIC::subroutineC(null) = null)
call echo("subroutineC")
;;; do stuff
call subroutineD(null)
end

subroutine (PUBLIC::subroutineD(null) = null)
call echo("subroutineD")
end

subroutine(PUBLIC::main(null) = null)
declare a_string = vc with protect, noconstant("")
declare b_string = vc with protect, noconstant("")
set a_string = subroutineA(1.0)
set b_string = subroutineB(a_string)
call subroutineC(null)
end

call main(null)

#exit_script
;; script finalizer code can go here
;; a PUBLIC::exitScript(null) subroutine that encapsulates this work could be created and called
;; if it does something that should occur for some unit tests.
end go
70 changes: 70 additions & 0 deletions cclunit-framework-source/doc/examples/mocking_api.inc
@@ -0,0 +1,70 @@
;;; put the following script definition in a .prg file and compile it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the introductory example for demonstrating the mocking api.
Shouldn't it foremost include a lightweight demonstration of table and table data mocking which are the key benefits of the api? The introductory paragraph in CCLUTGUIDANCE.md should likewise be updated to mention table access.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, technically the CCLUTMOCKING.md is supposed to be the introduction for mocking. I realize that people can read things in any order and hop around as they please, but this document links to it first, and there's a full example in there using tables.

Would it be more helpful if I said:
"These unit tests demonstrate a basic example of how the CCL Unit Testing framework's mocking API can be used instead of "with replace"."

Or if that's not acceptable, should I just move the example from CCLUTMOCKING here (or create a separate file for it (forgot to do that on the last commit) and link to it from both spots)?

This section was more based on the introduction of what mocking is than the full suite, and I was reusing your example from your blog which was just mocking the script, so my preference would be to clearly state that it's a basic example, or just reuse the CCLUTMOCKING example rather than having a separate example.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it would make more sense if the stated purpose were "how to accomplish a basic 'with replace' while using the mocking framework" opposed to "demonstrate the mocking api".

It would also be good to modify "Details on the API can be found....." to say "Example usage and details...". Otherwise this still feels like the first offering of any example usage of the mocking api.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I'll make those changes.



drop program mock_other_script go
create program mock_other_script
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe document this as
/**
Mock implementation for other_script which sets reply equal to a copy of mock_other_script_reply.
*/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

free record reply
set stat = copyrec(mock_other_script_reply, reply, 1)
end go

;;; put the following functions in a test case (.inc)

/**
* Test myFunction when other_script returns zero items
*/
subroutine (testMyFunctionOtherScriptZero(null) = null)
record mock_other_script_reply (
1 qual [*]
2 id = f8
%i cclsource:status_block.inc
) with protect

set mock_other_script_reply->status_data->status = "Z"

call cclutAddMockImplementation("OTHER_SCRIPT", "mock_other_script")
call cclutExecuteProgramWithMocks("the_script", "")

;assert things here
end ;;;testMyFunctionZero

/**
* Test myFunction when other_script returns more than five items
*/
subroutine (testMyFunctionOtherScriptMoreThanTen(null) = null)
record mock_other_script_reply (
1 qual [*]
2 id = f8
%i cclsource:status_block.inc
) with protect

set mock_other_script_reply->status_data->status = "S"
set stat = alterlist(mock_other_script_reply->qual, 6)

declare idx = i4 with protect, noconstant(0)
for (idx = 1 to 6)
set mock_other_script_reply->qual[idx].id = idx
endfor

call cclutAddMockImplementation("OTHER_SCRIPT", "mock_other_script")
call cclutExecuteProgramWithMocks("the_script", "")

;assert things here
end ;;;testMyFunctionMoreThanTen

/**
* Test myFunction when other_script fails
*/
subroutine (testMyFunctionOtherScriptFail(null) = null)
record mock_other_script_reply (
1 qual [*]
2 id = f8
%i cclsource:status_block.inc
) with protect

set mock_other_script_reply->status_data->status = "F"

call cclutAddMockImplementation("OTHER_SCRIPT", "mock_other_script")
call cclutExecuteProgramWithMocks("the_script", "")

;assert things here
end ;;;testMyFunctionOtherScriptFail
13 changes: 13 additions & 0 deletions cclunit-framework-source/doc/examples/using_namespaces.inc
@@ -0,0 +1,13 @@
/**
* Executes a test knowing that every call to getPersonName(id) will return "Bob Marley".
*/
subroutine (testGetNameReturnsBobMarley(null) = null)
declare otherScriptCallCount = i4 with protect, noconstant(0)

call cclutExecuteProgramWithMocks("the_script", "", "testGetNameReturnsBobMarley")

; assert stuff here
end
subroutine (testGetNameReturnsBobMarley::getPersonName(id = f8) = vc)
return ("Bob Marley")
end ;;;testGetNameReturnsBobMarley
47 changes: 47 additions & 0 deletions cclunit-framework-source/doc/examples/validation_subroutine.inc
@@ -0,0 +1,47 @@
;;; here is the script

drop program the_script go
create program the_script
execute other_script 4, 5
execute other_script 3, 2
execute other_script 0, 1
end go


;;; put this definition in a .prg file and compile it

drop program mock_other_script go
create program mock_other_script
prompt "param 1", "param 2" with param1, param2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/**
Mock implementation for other_script which calls validateOtherScriptParams to validate the two input parameters
and increments a counter for the number of calls to other_script.
*/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change.


set otherScriptCallCount = otherScriptCallCount + 1
call validateOtherScriptParams($param1, $param2)
end go

;;; put this functions in a test case (.inc)

/**
* confirms that the script executes other_script exactly three times passing in (4, 5) then (3, 2) then (0, 1)
*/
subroutine (testOtherScriptCalledThreeTimes(null) = null)
declare otherScriptCallCount = i4 with protect, noconstant(0)

call cclutAddMockImplementation("OTHER_SCRIPT", "mock_other_script")
call cclutExecuteProgramWithMocks("the_script", "")

set stat = cclutAssertI4Equal(CURREF, "testMyFunction_6_9 a", otherScriptCallCount, 3)
end ;;;testMyFunctionZero

subroutine (validateOtherScriptParams(p1 = i4, p2 = i4) = null)
case (otherScriptCallCount)
of 1:
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 1 a", p1, 4)
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 1 b", p2, 5)
of 2:
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 2 a", p1, 3)
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 2 b", p2, 2)
of 3:
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 3 a", p1, 0)
set stat = cclutAssertI4Equal(CURREF, "validateOtherScriptParams 3 b", p2, 1)
endcase
end ;;;validateOtherScriptParams
2 changes: 2 additions & 0 deletions cclunit-framework-source/src/main/ccl/cclut_ff.prg
Expand Up @@ -110,6 +110,8 @@ with outdev, testCaseDirectory, testCaseFileName, testNamePattern, optimizerMode
cclut_ff::outputLine row+1
cclut_ff::outputLine = "%i cclsource:cclut_test_runner.inc"
cclut_ff::outputLine row+1
cclut_ff::outputLine = "%i cclsource:cclutmock.inc"
cclut_ff::outputLine row+1
cclut_ff::outputLine = concat("%i ", cclut_ff::testCaseFileLocation)
cclut_ff::outputLine row+1
cclut_ff::outputLine = \
Expand Down
Expand Up @@ -294,6 +294,7 @@ set cclutReply->status_data.status = "S"
call cclut::executeTestLogic(null)

#exit_script
;Clean up any mocks that were not cleaned up by the tests.
call cclutRemoveAllMocks(null)
;Record any dangling errors and asserts in case the last executed unit test called go to exit_script.
set cclutReply->resultInd = band(cclutReply->resultInd,
Expand Down
Expand Up @@ -48,6 +48,9 @@ if (checkfun("TEARDOWNONCE") = cclut_test_runner::FUN_TYPE_SUBROUTINE)
endif
endif

;Clean up any mocks that were not cleaned up by the tests.
call cclutRemoveAllMocks(null)

call parser(concat("drop program ", cclut_ff::testProgramName, " go"))

if (cclut_test_runner::assertSuccess = TRUE AND cclut_test_runner::testRunnerSuccessInd = TRUE)
Expand Down
Expand Up @@ -78,4 +78,7 @@ end

call internalSubroutine(null)

set internalVariable = 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose internalVariable and internalRecord->item are used to confirm that the SUT was actually executed.
Is there a situation where that is questionable?
Does it not suffice to just use one thing for this?

Note that the adjective "internal" doesn't really fit here. Maybe "external" or "provided".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose internalVariable and internalRecord->item are used to confirm that the SUT was actually executed.

Actually, I was doing it to comply with the statement from here:
https://github.com/cerner/cclunit-framework/pull/7/files#r250405913

Just for kicks, I would probably demonstrate something similar for record PUBLIC::rec and variable PUBLIC::var defined by SUT.

Although reading it again, it sounds like you wanted the declarations in SUT. Do you want me to just update these tests to move the declarations inside SUT? And, if so, should I leave it as "internal" or do you still want "external"/"provided"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct.
That comment intended for the_script to declare a record and a variable to be overridden by the test. I am not sure how to derive value from this capability, but I believe we do support it.

To do this, the _namespace test would declare competing mock::rec and mock::var that would be updated rather than the public::rec and pulic::var declared by SUT. If SUT declares the public:: versions 'with protect', they will fall out of scope after the script returns (i.e., they will fail a validate check) yet the values will be set on the mock:: versions declared by the test.

The 'with persistscript' check needs to be performed on yet some other thing declared within SUT and only needs to be checked by one test after SUT returns I imagine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I'll update the _namespace test and update the _happy test to include a persistscript check.

set internalRecord->item = 1

end go
Expand Up @@ -1076,3 +1076,38 @@ subroutine testSetupErrors(null)
call cclutAssertI4Equal(CURREF, "assert value",
ucetc_testCaseResults->tests[5].asserts[1].resultInd, TRUE)
end ;;;testSetupErrors

/**
Test that the framework removes any mock tables that test cases fail to do when calling cclut_execute_test_case.
*/
subroutine testMockTableCleanup(null)
declare cclutMockTableName = vc with protect, noconstant("")

set ucetc_request->testINCName = "ut_cclut_mock_table_misbehaving_cleanup"

execute cclut_execute_test_case with
replace("CCLUTREQUEST", ucetc_request),
replace("CCLUTREPLY", ucetc_reply),
replace("CCLUTTESTCASERESULTS", ucetc_testCaseResults)

call cclutAssertStartsWith(CURREF, "testMockTableCleanup 001", "CUST_CCLUT", cclutMockTableName)

declare mockId = f8 with protect, noconstant(0.0)
declare mockText = vc with protect, noconstant("")
declare mockDate = dq8 with protect, noconstant(0)
select into "nl:"
from (value(cclutMockTableName) m)
detail
mockId = m.SAMPLE_TABLE_ID
mockText = m.SAMPLE_TABLE_TEXT
mockDate = m.SAMPLE_TABLE_DATE
with nocounter

declare errorMessage = vc with protect, noconstant("")
declare errorCode = i4 with protect, noconstant(0)
set errorCode = error(errorMessage, 0)

call cclutAssertVcOperator(CURREF, "testMockTableCleanup 002", trim(errorMessage, 3),
"regexplike", concat("%CCL-E-18-PRG_[0-9]+_[0-9]+\([^)]+\)[0-9]+:[0-9]+\{\}Unable to add range, definition for",
" table \(CUST_CCLUT_[^)]*\) not found in dictionary."))
end ;;;testMockTableCleanup
Expand Up @@ -1577,4 +1577,39 @@ subroutine testCoverageExcludes(null)
replace("CCLUTTESTCASERESULTS", tce_testCaseResults)
call cclutAssertVcEqual(CURREF, "expected empty listingXml", tce_reply->programs[1].listingXml, "")
call cclutAssertVcEqual(CURREF, "expected same coverageXml", tce_reply->programs[1].coverageXml, coverageXml)
end ;;;testCoverageExcludes
end ;;;testCoverageExcludes

/**
Test that the framework removes any mock tables that test cases fail to do when calling cclut_execute_test_case_file.
*/
subroutine testMockTableCleanup(null)
declare cclutMockTableName = vc with protect, noconstant("")

set ucets_request->testCaseFileName = "ut_cclut_mock_table_misbehaving_cleanup"

execute cclut_execute_test_case_file with
replace("CCLUTREQUEST", ucets_request),
replace("CCLUTREPLY", ucets_reply),
replace("CCLUTTESTCASERESULTS", ucets_testCaseResults)

call cclutAssertStartsWith(CURREF, "testMockTableCleanup 001", "CUST_CCLUT", cclutMockTableName)

declare mockId = f8 with protect, noconstant(0.0)
declare mockText = vc with protect, noconstant("")
declare mockDate = dq8 with protect, noconstant(0)
select into "nl:"
from (value(cclutMockTableName) m)
detail
mockId = m.SAMPLE_TABLE_ID
mockText = m.SAMPLE_TABLE_TEXT
mockDate = m.SAMPLE_TABLE_DATE
with nocounter

declare errorMessage = vc with protect, noconstant("")
declare errorCode = i4 with protect, noconstant(0)
set errorCode = error(errorMessage, 0)

call cclutAssertVcOperator(CURREF, "testMockTableCleanup 002", trim(errorMessage, 3),
"regexplike", concat("%CCL-E-18-PRG_[0-9]+_[0-9]+\([^)]+\)[0-9]+:[0-9]+\{\}Unable to add range, definition for",
" table \(CUST_CCLUT_[^)]*\) not found in dictionary."))
end ;;;testMockTableCleanup
30 changes: 30 additions & 0 deletions cclunit-framework-source/src/test/ccl/ut_cclut_ff.inc
Expand Up @@ -718,3 +718,33 @@ subroutine testFailedCompile(null)
call testing::checkMessages(null)
call testing::checkMRS(_memory_reply_string)
end ;;;testFailedCompile

/**
Test that the framework removes any mock tables that test cases fail to do when calling cclut_ff.
*/
subroutine testMockTableCleanup(null)
declare cclutMockTableName = vc with protect, noconstant("")

execute cclut_ff "ut_cclut_output", "cclsource", "ut_cclut_mock_table_misbehaving_cleanup"

call cclutAssertStartsWith(CURREF, "testMockTableCleanup 001", "CUST_CCLUT", cclutMockTableName)

declare mockId = f8 with protect, noconstant(0.0)
declare mockText = vc with protect, noconstant("")
declare mockDate = dq8 with protect, noconstant(0)
select into "nl:"
from (value(cclutMockTableName) m)
detail
mockId = m.SAMPLE_TABLE_ID
mockText = m.SAMPLE_TABLE_TEXT
mockDate = m.SAMPLE_TABLE_DATE
with nocounter

declare errorMessage = vc with protect, noconstant("")
declare errorCode = i4 with protect, noconstant(0)
set errorCode = error(errorMessage, 0)

call cclutAssertVcOperator(CURREF, "testMockTableCleanup 002", trim(errorMessage, 3),
"regexplike", concat("%CCL-E-18-PRG_[0-9]+_[0-9]+\([^)]+\)[0-9]+:[0-9]+\{\}Unable to add range, definition for",
" table \(CUST_CCLUT_[^)]*\) not found in dictionary."))
end ;;;testMockTableCleanup