Fix problem with excluding training ops #835
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've found a problem on a line 240 of tflearn/helpers/trainer.py where i've made changes.
The problem is that a cycle, where program removes excluded training operations from actual ones, it doesn't remove all training operations that must be excluded. I'll try to explain this cycle on similar task of removing numbers from an array. Let's see:
This is a similar cycle that removes number 2 from array instead of excluded trainops. You think that it'll remove all 2 and resulting array a will be empty. But if you run this code you'll see, that it six numbers 2 in your array left!
It's happen so, because, ii seems to me, Python's iterator have some attachements to index of element instead of element itself. How it's hapenning:
cycle takes zero element from an array. Element 'a' removed
second iteration
iterator takes element with index 1, that have a sense if array is still similar. But as first item was removed, indexes of elements has changed, and now element 1 is a an element 2 from previous iteration.
That way your cycle doesn't always pass over all items, while there are training operations to exclude. I've found this error when tried to use on my graph three different trainops, that should run separately in different training steps.