Updated readme with more information to better describe how TcUnit-Ve…

…rifier works.

Improved robustness of C#/.NET-part of the application.
Updated .gitignore-file to newest template.

.gitignore

@@ -1,5 +1,5 @@
-# gitignore template for TwinCAT
-# website: https://www.beckhoff.com/twincat/
+# gitignore template for TwinCAT3
+# website: https://www.beckhoff.com/twincat3/
 #
 # Recommended: VisualStudio.gitignore
 
@@ -362,4 +362,7 @@ ASALocalRun/
 healthchecksdb
 
 # Backup folder for Package Reference Convert tool in Visual Studio 2017
-MigrationBackup/
+MigrationBackup/
+
+# Ionide (cross platform F# VS Code tools) working folder
+.ionide/

README.md

@@ -1,32 +1,42 @@
 # TcUnit-Verifier
-This project is can be used to verify the different functions of [TcUnit](https://www.github.com/tcunit/TcUnit).
-It has various test suites to test different functions in TcUnit. It consists of two separate programs:
+This project is can be used to verify the different functions of
+[TcUnit](https://www.github.com/tcunit/TcUnit). It has various test suites to
+test different functions in TcUnit. It consists of two separate programs:
 - TcUnit-Verifier_TwinCAT
 - TcUnit-Verifier_DotNet
 
-When verifying that TcUnit works as expected a developer needs to add tests to both of these programs.
+When verifying that TcUnit works as expected a developer needs to add tests to
+both of these programs.
 
 ![TcUnit-Verifier concept](https://github.com/tcunit/TcUnit-Verifier/blob/master/img/TcUnit-Verifier_Concept_1280.png)
 
 ## TcUnit-Verifier_TwinCAT
-TcUnit-Verifier_TwinCAT (TcVT) is as the name implies a TwinCAT project. It holds a reference to and instantiates
-the TcUnit framework. It is defined by several test suites (function blocks) that make use of TcUnit and its different
-functionalitites. An example of a test is to verify that TcUnit prints a message if an assertion of an
-INT has failed (expected value not equals to asserted value). This means per definition that the verifier
-will have failed tests as results, because that is what we want to test. Everytime a test suite is created
-in TcVT that tests the functionality of TcUnit it is expected that either:
-- Some specific output is created by TcUnit (error-log), for example asserting a value that differs from the expected value
-- Some specific output is **not** created by TcUnit (error-log), for example asserting a value that is equal to the expected value
-
-Creating a test suite for TcUnit thus requires that the developer needs to know what TcUnit should print (or not print)
-This means that running this TwinCAT project is not enough to know whether TcUnit behaves as expected, as it would
-require to manually check all the output from TcVT. Because we don't manually want to check that the output provided by
-TcUnit-Verifier_TwinCAT everytime it's running, another program is necessary that verifies that the output of the TcVT
+TcUnit-Verifier_TwinCAT (TcVT) is as the name implies a TwinCAT project.
+It holds a reference to and instantiates the TcUnit framework. It is defined by
+several test suites (function blocks) that make use of TcUnit and its different
+functionalitites. An example of a test is to verify that TcUnit prints a message
+if an assertion of an INT has failed (expected value not equals to
+asserted value). This means per definition that the verifier will have failed
+tests as results, because that is what we want to test. Everytime a test suite
+is created in TcVT that tests the functionality of TcUnit it is expected
+that either:
+- Some specific output is created by TcUnit (error-log), for example asserting a
+  value that differs from the expected value
+- Some specific output is **not** created by TcUnit (error-log), for example
+  asserting a value that is equal to the expected value
+
+Creating a test suite for TcUnit thus requires that the developer needs to know
+what TcUnit should print (or not print). This means that running this TwinCAT
+project is not enough to know whether TcUnit behaves as expected, as it would
+require to manually check all the output from TcVT. Because we don't manually
+want to check that the output provided by TcUnit-Verifier_TwinCAT everytime it's
+running, another program is necessary that verifies that the output of the TcVT
 is as expected.
 
 ## TcUnit-Verifier_DotNet
-The TcUnit-Verifier_DotNet (TcVD) is a C# program that opens and runs the TcUnit-Verifier_TwinCAT project by the usage
-of the TwinCAT automation interface. It basically does the following:
+The TcUnit-Verifier_DotNet (TcVD) is a C# program that opens and runs the
+TcUnit-Verifier_TwinCAT project by the usage of the TwinCAT automation
+interface. It basically does the following:
 - Starts Visual Studio (using the same version that was used developing TcVT)
 - Opens TcVT
 - Does a clean/build of TcVT
@@ -37,4 +47,65 @@ of the TwinCAT automation interface. It basically does the following:
 - Collects all results from the error list in Visual Studio
 - Checks that the output is as defined/expected
 
+The different tests in TcVD follow the same naming-schema as for the TcVT tests
+so for example you can find a structured text (ST) function block (FB)
+`FB_PrimitiveTypes.TcPOU` in TcVT and the accompanying C#-class
+`FB_PrimitiveTypes.cs`. The C# classes verify and test that the output from the
+ST function blocks is correct. Thus a complete test of a specific function in
+TcUnit needs to be developed in pair, both a FB of tests needs to be added to
+TcVT as well a class of tests needs to be added to TcVD.
+
+All test classes are instantiated in the class `Program.cs` starting from the
+lines:
+```
+/* Insert the test classes here */
+FB_PrimitiveTypes primitiveTypes = new FB_PrimitiveTypes(errorItems, "PrimitiveTypes");
+FB_AssertTrueFalse assertTrueFalse = new FB_AssertTrueFalse(errorItems, "AssertTrueFalse");
+FB_AssertEveryFailedTestTwice assertEveryFailedTestTwice = new FB_AssertEveryFailedTestTwice(errorItems, "AssertEveryFailedTestTwice");
+...
+...
+...
+```
+
+To create a new test class and make sure that it will be running all that is
+necessary is to make sure to instantiate it with two arguments:
+1. A reference to errorItems (just as above)
+2. A string with the name of what the function block is called in TcVT in the
+   PRG_TEST-program
+
+For example, if we in the PRG_TEST-program of TcVT have a function block
+instantiated in this way:
+```
+PROGRAM PRG_TEST
+VAR
+    AssertEveryFailedTestTwiceArrayVersion : FB_AssertEveryFailedTestTwiceArrayVersion;
+END_VAR
+```
+The equivalent test class in TcVD needs to be instantiated with the second
+argument using the same name as in PRG_TEST, in this example:
+```
+FB_AssertEveryFailedTestTwiceArrayVersion assertEveryFailedTestTwiceArrayVersion = new FB_AssertEveryFailedTestTwiceArrayVersion(errorItems, "AssertEveryFailedTestTwiceArrayVersion");
+```
+
+This is an example of how it can look running the TcUnit-Verifier_DotNet:
 
+```
+C:\Code\TcUnit\TcUnit-Verifier\TcUnit-Verifier_DotNet\TcUnit-Verifier\bin\Debug>TcUnit-Verifier.exe -v "C:\Code\TcUnit\TcUnit-Verifier\TcUnit-Verifier_TwinCAT\TcUnit-Verifier_TwinCAT.sln"
+2019-07-31 14:35:05 - Starting TcUnit-Verifier...
+2019-07-31 14:35:05 - Loading the Visual Studio Development Tools Environment (DTE)...
+2019-07-31 14:35:38 - Cleaning and building TcUnit-Verifier_TwinCAT solution...
+2019-07-31 14:35:39 - Generating TcUnit-Verifier_TwinCAT boot project...
+2019-07-31 14:35:59 - Activating TcUnit-Verifier_TwinCAT configuration...
+2019-07-31 14:35:59 - Restaring TwinCAT...
+2019-07-31 14:35:59 - Waiting for TcUnit-Verifier_TwinCAT to finish running tests...
+2019-07-31 14:36:12 - Asserting results...
+2019-07-31 14:36:31 - Done.
+2019-07-31 14:36:31 - Closing the Visual Studio Development Tools Environment (DTE), please wait...
+2019-07-31 14:36:51 - Exiting application...
+```
+If there was an error in the TcUnit framework this would be shown between the
+lines `Asserting results...` and `Done.`. If nothing is shown between these
+lines (like in the example) it means that TcUnit behaves according to the tests
+defined in TcUnit-Verifier. As TcVD starts the TwinCAT runtime it is required 
+that a (trial) runtime license for TwinCAT is activated on the PC of where TcVD
+and TcVT will be running.

TcUnit-Verifier_DotNet/TcUnit-Verifier/AutomationInterface.cs

@@ -7,17 +7,22 @@
 
 namespace TcUnit.Verifier
 {
+    /// <summary>
+    /// This class provides the functionality to access the TwinCAT automation interface, which
+    /// is a complement to the VS DTE and that gives access to certain TwinCAT specific functions
+    /// integrated into Visual Studio
+    /// </summary>
     class AutomationInterface
     {
-        private ITcSysManager13 sysManager = null;
+        private ITcSysManager10 sysManager = null;
         private ITcConfigManager configManager = null;
         private ITcSmTreeItem plcTreeItem = null;
         private ITcSmTreeItem routesTreeItem = null;
 
 
         public AutomationInterface(EnvDTE.Project project)
         {
-            sysManager = project.Object;
+            sysManager = (ITcSysManager10)project.Object;
             configManager = (ITcConfigManager)sysManager.ConfigurationManager;
             plcTreeItem = sysManager.LookupTreeItem(Constants.PLC_CONFIGURATION_SHORTCUT);
             routesTreeItem = sysManager.LookupTreeItem(Constants.RT_CONFIG_ROUTE_SETTINGS_SHORTCUT);
@@ -26,7 +31,7 @@ public AutomationInterface(EnvDTE.Project project)
         public AutomationInterface(VisualStudioInstance vsInst) : this(vsInst.GetProject())
         { }
 
-        public ITcSysManager13 ITcSysManager
+        public ITcSysManager10 ITcSysManager
         {
             get
             {

TcUnit-Verifier_DotNet/TcUnit-Verifier/Program.cs

@@ -24,6 +24,7 @@ class Program
         static int Main(string[] args)
         {
             bool showHelp = false;
+            Console.CancelKeyPress += new ConsoleCancelEventHandler(CancelKeyPressHandler);
             log4net.GlobalContext.Properties["LogLocation"] = AppDomain.CurrentDomain.BaseDirectory + "\\logs";
             log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo(AppDomain.CurrentDomain.BaseDirectory + "log4net.config"));
 
@@ -62,6 +63,7 @@ static int Main(string[] args)
             try
             {
                 vsInstance = new VisualStudioInstance(@tcUnitVerifierPath);
+                vsInstance.Load();
             }
             catch
             {
@@ -169,13 +171,28 @@ private static void DisplayHelp(OptionSet p)
             p.WriteOptionDescriptions(Console.Out);
         }
 
+        /// <summary>
+        /// Executed if user interrups the program (i.e. CTRL+C)
+        /// </summary>
+        static void CancelKeyPressHandler(object sender, ConsoleCancelEventArgs args)
+        {
+            log.Info("Application interrupted by user");
+            CleanUp();
+            Environment.Exit(0);
+        }
+
+        /// <summary>
+        /// Cleans the system resources (the VS DTE)
+        /// </summary>
         private static void CleanUp()
         {
             try
             {
                 vsInstance.Close();
             }
             catch { }
+
+            log.Info("Exiting application...");
             MessageFilter.Revoke();
         }
     }

TcUnit-Verifier_DotNet/TcUnit-Verifier/VisualStudioInstance.cs

@@ -1,14 +1,20 @@
 //using log4net;
 using EnvDTE80;
+using log4net;
 using System;
 using System.Collections.Generic;
 using System.IO;
 using System.Linq;
 using System.Text;
+using System.Threading;
 using System.Threading.Tasks;
 
 namespace TcUnit.Verifier
 {
+    /// <summary>
+    /// This class is used to instantiate the Visual Studio Development Tools Environment (DTE)
+    /// which is used to programatically access all the functions in VS.
+    /// </summary>
     class VisualStudioInstance
     {
         private string @filePath = null;
@@ -17,30 +23,55 @@ class VisualStudioInstance
         private Type type = null;
         private EnvDTE.Solution visualStudioSolution = null;
         EnvDTE.Project pro = null;
-        //ILog log = LogManager.GetLogger("TcUnit-Verifier");
+        ILog log = LogManager.GetLogger("TcUnit-Verifier");
+        private bool loaded = false;
 
         public VisualStudioInstance(string @visualStudioSolutionFilePath)
         {
             this.filePath = visualStudioSolutionFilePath;
             string visualStudioVersion = FindVisualStudioVersion();
             this.vsVersion = visualStudioVersion;
-            LoadDevelopmentToolsEnvironment(visualStudioVersion);
-            LoadSolution(@visualStudioSolutionFilePath);
-            LoadProject();
         }
 
         public VisualStudioInstance(int vsVersionMajor, int vsVersionMinor)
         {
             string visualStudioVersion = vsVersionMajor.ToString() + "." + vsVersionMinor.ToString();
             this.vsVersion = visualStudioVersion;
-            LoadDevelopmentToolsEnvironment(visualStudioVersion);
         }
 
+        /// <summary>
+        /// Loads the development tools environment
+        /// </summary>
+        public void Load()
+        {
+            loaded = true;
+            LoadDevelopmentToolsEnvironment(vsVersion);
+            if (!String.IsNullOrEmpty(@filePath))
+            {
+                LoadSolution(@filePath);
+                LoadProject();
+            }
+        }
+
+        /// <summary>
+        /// Closes the DTE and makes sure the VS process is completely shutdown
+        /// </summary>
         public void Close()
         {
-            dte.Quit();
+            if (loaded)
+            {
+                log.Info("Closing the Visual Studio Development Tools Environment (DTE), please wait...");
+                Thread.Sleep(20000); // Avoid 'Application is busy'-problem (RPC_E_CALL_REJECTED 0x80010001 or RPC_E_SERVERCALL_RETRYLATER 0x8001010A)
+                dte.Quit();
+            }
+            loaded = false;
         }
 
+
+        /// <summary>
+        /// Opens the main *.sln-file and finds the version of VS used for creation of the solution
+        /// </summary>
+        /// <returns>The version of Visual Studio used to create the solution</returns>
         private string FindVisualStudioVersion()
         {
             /* Find visual studio version */
@@ -81,6 +112,7 @@ private void LoadDevelopmentToolsEnvironment(string visualStudioVersion)
              */
             string VisualStudioProgId = "VisualStudio.DTE." + visualStudioVersion;
             type = System.Type.GetTypeFromProgID(VisualStudioProgId);
+            log.Info("Loading the Visual Studio Development Tools Environment (DTE)...");
             dte = (EnvDTE80.DTE2) System.Activator.CreateInstance(type);
             dte.UserControl = false; // have devenv.exe automatically close when launched using automation
             dte.SuppressUI = true;
@@ -99,6 +131,7 @@ private void LoadProject()
             pro = visualStudioSolution.Projects.Item(1);
         }
 
+        /// <returns>Returns null if no version was found</returns>
         public string GetVisualStudioVersion()
         {
             return this.vsVersion;
@@ -139,6 +172,5 @@ public ErrorItems GetErrorItems()
 
             return returnErrorItems;*/
         }
-
     }
 }