Debug mode output bug

[ejunprung]

In some models, debug mode will not capture information in time. In the below, debug mode shows that every agent is always skipped but agents are actually receiving actions just fine.

Example: BikePicking.zip

[SaharEs]

@ejunprung This happens because we print debug after executing the actions. If we print debug first it would solve the problem.

So DE_executeStep should look like this:

if(debugMode)
	printDebug();

for (int i = 0; i < numberOfAgents; i++) {
	if (!isSkip(i)) {
		// Be sure to call those here allowing users to fix errors before training
		double[] training = observationForTraining(i);
		double[] reward = observationForReward(i);
		// Throw error if infinity or NaN detected in observations or rewards array
		if (DoubleStream.of(training).anyMatch(x -> x == infinity || Double.isNaN(x))) {
			throw new RuntimeException("Invalid input. Observations cannot contain a value of infinity or NaN.");
		}
		if (DoubleStream.of(reward).anyMatch(x -> x == infinity || Double.isNaN(x))) {
			throw new RuntimeException("Invalid input. Reward cannot contain a value of infinity or NaN.");
		}
		
		if (usePolicy && policyHelper != null) {
			double[] a = policyHelper.computeActions(training);
			action[i] = doAction(a, true, i);
		} else if (!usePolicy) {
			double[] a = getActionRandom(getEngine().getDefaultRandomGenerator(), i);
			action[i] = doAction(a, false, i);
		}
	}
}

steps++;

I think printing debug first makes more sense for other models too.

[SaharEs]

Oh, but if we do this the "Last Action" will be really for the previous action, not the action taken at that step.

we have to first save the decided actions, then print debug, then execute the actions.