Trame hot reload, call align_system from decider.py, fill out some GUI#1
Trame hot reload, call align_system from decider.py, fill out some GUI#1
Conversation
Cleans some boilerplate and sets up trame's hot reload dev hack. Run align-app --hot-reload and a reload button appears for quick GUI tweaks.
PaulHax
requested review from
barry-ravichandran and
dmjoy
and removed request for
barry-ravichandran and
dmjoy
PaulHax
changed the title
|
@barry-ravichandran @dmjoy First pass here! |
| with html.Div(classes="text-h6 text-no-wrap text-truncate"): | ||
| html.Span("Scenario: ", classes="font-weight-bold") | ||
| html.Span( | ||
| f"{{{{{scenario}.scenario_id}}}} - " |
There was a problem hiding this comment.
@jourdain Help! I'm drowning in brackets.
There was a problem hiding this comment.
Just use the old "".format() syntax ;-)
PaulHax
changed the title
dmjoy
left a comment
dmjoy
left a comment
There was a problem hiding this comment.
Overall LGTM, don't have any requested changes really, just want to make a not of some things that we'll likely want to change in the future.
| log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def filter_actions(current_state, available_actions) -> list: |
There was a problem hiding this comment.
Hoping much longer term this function won't be necessary. This largely depends upon how the scenarios are designed. For now certainly fine to leave in just putting it on your radar.
align_app/adm/decider.py
Outdated
| deciders = ["outlines_transformers_structured", "outlines_comparative_regression"] | ||
|
|
||
|
|
||
| def load_llm(llm_backbone=LLM_BACKBONES[0], decider=deciders[0], aligned=True): |
There was a problem hiding this comment.
Nitpick here, but this is more like load_adm, the LLM is just one of the components of the algorithm
| def load_alignment_target(kdma=attributes[0], kdma_value=0): | ||
| kdma_split = kdma.split("_") | ||
| kdma_file = " ".join(kdma_split).capitalize() | ||
| if kdma_file in ["Moral deservingness", "Maximization"]: | ||
| binary_alignment = "high" if float(kdma_value) >= 0.5 else "low" | ||
| filename = f"{kdma}_{binary_alignment}.yaml" | ||
| elif kdma_file in ["Moral judgement", "Ingroup bias"]: | ||
| filename = (f"ADEPT-DryRun-{kdma_file}-{kdma_value}.yaml",) | ||
|
|
||
| return OmegaConf.load(alignment_configs / filename) |
There was a problem hiding this comment.
This all is assuming binary alignment targets, but the comparative regression ADM, among others, uses a more granular target (float values from 0 to 1). I think this "binarization" should be handled within the ADMs if that needs to happen.
Somewhat related, we can have a handful of alignment targets (like we have already) loaded into the app, and then if the user tweaks the slider (as in the older demo) it just changes the value in the already loaded target if that makes any sense.
|
|
||
| alignment_target = prompt["alignment_target"] | ||
|
|
||
| action_decision, *_ = decider.instance.top_level_choose_action( |
There was a problem hiding this comment.
Just a reminder (we discussed at the last meeting), the second returned argument here will be choice_info; for some ADMs it will be empty but for others (like the comparative regression ADM) it will have useful information that could be displayed in the app.
| demo_kwargs={ | ||
| "max_generator_tokens": 8092, | ||
| "generator_seed": 2, | ||
| "shuffle_choices": False, | ||
| }, |
There was a problem hiding this comment.
Just a heads up, when we move to using the main align-system code, this argument isn't supported. FWIW nothing really "breaks" if they're removed now (I think)
There was a problem hiding this comment.
Moving forward we shouldn't need this version of the file (and should just use the one supplied by the align-system code)
| @@ -0,0 +1,22 @@ | |||
| instance: | |||
| _target_: models.comparative_regression_adm.OutlinesTransformersComparativeRegressionADM | |||
There was a problem hiding this comment.
Once we move to using the align-system code provided version of this ADM (rather than the modified version that lives in this repo) we should point this at align_system.algorithms.outlines_regression_adm_comparative.OutlinesTransformersComparativeRegressionADM
3 participants
align-app --hot- reloadand a reload button apears for quick GUI tweeks.align_systemfrom decider.py