Partial wrapper files generated in gen folder

[vijairaj]

With Comtypes 1.1.2 and Python 2.7.10

I have noticed that some times the generated wrappers have only the first two lines as below:

# -*- coding: mbcs -*-
typelib_path = u'C:\\XXXXXXXXXXXXXXXXXXXXX.tlb'

This manifests as issues such as:
AttributeError: 'module' object has no attribute 'XXXXX'

I had to delete the generated file and regenerate it to fix the issue whenever I see them. However I wasn't able to isolate the root cause and neither is the issue very frequent. After going through the github issues and searching over the internet, I don't see many complaining about such an issue. Only two instance that were discussed elsewhere seem relevant:

• https://geonet.esri.com/thread/122158
• http://gis.stackexchange.com/questions/135741/debugging-module-object-has-no-attribute-sdeworkspacefactory

so I guess this is not specific to my setup but a generic issue.

[vijairaj]

After some more analysis of the information collected during this issue, I believe that the issue could have been due to the user closing the application while the cache files are being generated. Looking at the code generator, I see that it creates a cache file and writes the first two lines before it proceeds with resolving the dependencies. The rest of the code is buffered in to StringIO before making it to the file and if there's an interruption in between, it would not make it to the file and result in an incomplete file. I guess we can make this a bit more robust by:

1. Not creating the file upfront and instead Buffering the header to StringIO as well. Then, after all the dependencies are resolved and the code is generated in the buffer, create the file and dump all the buffers to the file thereby reducing the chances of incomplete file.
2. Add a file valid check to the end of the file (a variable like _comtypes_cache_valid = True) to the end of the file which the GetModule logic can validate to detect if the file is valid, In case of invalid files the file can be automatically regenerated.

[junkmd]

I PRed #300 to fix this problem.

[junkmd]

Although #116 is closed without merging, the "Partial wrapper files generated in gen folder" can still occur.
This issue still needs to be considered about proper implementation and testing.

[vasily-v-ryabov]

Maybe wrapper generation should become a transaction to guarantee full file generation or fail.

[vijairaj]

Maybe wrapper generation should become a transaction to guarantee full file generation or fail.

I agree. The problem with the original implementation was that the first two lines of the generated code was immediately written to the file but the remaining lines were streamed to StringIO and later written to file. The partial file problem could occur when the execution is interrupted b/w these two steps.

In my original PRs - I employed two strategies to solve this problem

1. Reduce the chance of partial file generation by first using StringIO to cache all the generated content and use a single call to transfer the content to a file
2. Auto regenerate files during import if they are incomplete - this was done by using a marker statement to the generated file and checking for the valid marker during import.