Implemented Optimization with additive scoring and RL for selection policy learning - #1
Conversation
| {"output", required_argument, NULL, 'o'}, | ||
| {"property", no_argument, NULL, 'p'}, | ||
| {"theory", no_argument, NULL, 't'} | ||
| {"theory", no_argument, NULL, 't'}, |
There was a problem hiding this comment.
Don't we need to remove the trailing comma for the code to compile?
There was a problem hiding this comment.
No i found out that it compile with or without the commas.
There was a problem hiding this comment.
And plus the original code has it so I figured I'll just keep them.
| heuristicPostProcessing(); | ||
| /* If RL episodic loop ran, postProcessing already executed inside the loop */ | ||
| if (!(doConjecturing && selectedHeuristic == RL_HEURISTIC)) { | ||
| if (heuristicPostProcessing) heuristicPostProcessing(); |
There was a problem hiding this comment.
If the selectedHeuristic is not RL_HEURISTIC, we still need to call heuristicPostProcessing. Please test the code to ensure that the Dalmatian heuristic works as before.
There was a problem hiding this comment.
The non-RL branch already calls heuristicPostProcessing() immediately after conjecture(), so the additional conditional block at the end was redundant and could result in duplicate execution.
I removed that extra block and confirmed that the Dalmatian heuristic behaves exactly as before (single initialization, single conjecture pass, single post-processing call).
jpbrooks
left a comment
jpbrooks
left a comment
There was a problem hiding this comment.
These are in addition to a few individual comments.
| } samConjecture; | ||
|
|
||
| /* Globals with defaults (you can wire CLI options later) */ | ||
| static int sam_top_k = 9; /* --sam-top-k <int> */ |
There was a problem hiding this comment.
Can these be removed now that they are command line options?
There was a problem hiding this comment.
yes it can, those are just default like the complexity_limit.
| double avgTightness = (n_feasible ? sumTight / (double)n_feasible : 0.0); | ||
|
|
||
| double complexityPenalty = (double)(targetUnary + 2 * targetBinary); | ||
| double score = avgViolation + sam_beta * avgTightness + sam_lambda * complexityPenalty; |
There was a problem hiding this comment.
Are we using alpha for the violation weight?
There was a problem hiding this comment.
yes I have included it now
| #endif | ||
|
|
||
| /*------------ Tunables -----------------------------------------------------*/ | ||
| static double rl_alpha_viol = 1.0; |
There was a problem hiding this comment.
Should these be command-line options?
There was a problem hiding this comment.
yes all parameters are command line options with the defaults as fall back incase they are not provided.
|
|
||
| /* ------------------------------- Utils ---------------------------------- */ | ||
| static inline int is_finite(double x){ return isfinite(x); } | ||
| static inline double clamp(double x, double lo, double hi){ |
There was a problem hiding this comment.
Can you please add a comment for each function with a brief description of what it does?
Removed unnecessary blank lines to improve code readability.
2 participants
This PR introduces an optimization framework using additive scoring, along with a reinforcement learning–based policy for expression selection.
Key updates:
Implemented additive scoring combining error, complexity, and constraint penalties.
Integrated an RL-driven selection mechanism to guide pool updates.
Refined reward structure to improve stability and reduce premature swap plateaus.
These changes aim to improve learning efficiency and expression quality within the conjecturing framework.