Add support for replaying data#2712
Draft
t-b wants to merge 6 commits into
Draft
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces an automated-testing-only “replay” mechanism that re-runs acquisition/analysis using previously acquired sweep data and stored LBN settings (intended to support debugging and reproducing analysis behavior from saved experiments).
Changes:
- Add new replay helper module (
UTF_ReplayData.ipf) that backs up MIES state, replays sweeps, and injects acquired AD data into the DAQ pipeline. - Hook replay behavior into core acquisition/config paths (set column/cycle count, SCI/RAC IDs, oscilloscope DAQ data) under
#ifdef AUTOMATED_TESTING. - Extend hardware test acquisition configuration (
DAQSettings+ parser +AcquireData_NG) to support additional settings (autobias, DA scale, async channels).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| Packages/tests/UTF_ReplayData.ipf | New replay/compare helpers and data injection logic for automated testing. |
| Packages/tests/UTF_HardwareHelperFunctions.ipf | Extend DAQSettings and acquisition setup to apply more controls (autobias, DA scaling, async channels). |
| Packages/tests/HardwareAnalysisFunctions/UTF_HardwareAnalysisFunctions_Includes.ipf | Include replay helpers for hardware analysis tests. |
| Packages/MIES/MIES_WaveDataFolderGetters.ipf | Documentation updates for async settings key naming/columns. |
| Packages/MIES/MIES_Oscilloscope.ipf | Inject replay data into DAQDataWave during DAQ in automated testing. |
| Packages/MIES/MIES_GlobalStringAndVariableAccess.ipf | Add global accessor for replay enable flag. |
| Packages/MIES/MIES_DataConfigurator.ipf | Override set column/cycle count and SCI ID from replay settings in automated testing. |
| Packages/MIES/MIES_DAEphys.ipf | Initialize DAQ counters/flags (count, RAC, PRE_SET_EVENT) from replay settings in automated testing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| WAVE numericalValues = GetLBNumericalValues(device) | ||
| WAVE/T textualValues = GetLBTextualValues(device) | ||
|
|
||
| WAVE sweeps = AFH_GetSweepsFromSameSCI(numericalValues, sweepNo, headstage) |
Comment on lines
+63
to
+71
| // now delete all acquired data with sweep <= refSweepNo | ||
| WAVE sweeps = AFH_GetSweeps(device) | ||
|
|
||
| DFREF deviceData = GetDeviceDataPath(device) | ||
|
|
||
| for(sweepNo : sweeps) | ||
| if(sweepNo < refSweepNo) | ||
| continue | ||
| endif |
Comment on lines
+121
to
+126
| WAVE numericalValues_replay = root:MIES:LabNoteBook:ITC18USB:Device0:numericalValues | ||
| WAVE/T textualValues_replay = root:MIES:LabNoteBook:ITC18USB:Device0:textualValues | ||
|
|
||
| WAVE numericalValues_orig = root:MIES_backup:LabNoteBook:ITC18USB:Device0:numericalValues | ||
| WAVE/T textualValues_orig = root:MIES_backup:LabNoteBook:ITC18USB:Device0:textualValues | ||
|
|
Comment on lines
+242
to
+245
|
|
||
| DFREF sweepDFR_orig = root:MIES_backup:HardwareDevices:ITC18USB:Device0:Data:$("X_" + num2str(sweepNo)): | ||
| DFREF sweepDFR_replay = root:MIES:HardwareDevices:ITC18USB:Device0:Data:$("X_" + num2str(sweepNo)): | ||
|
|
Comment on lines
+401
to
+406
|
|
||
| // todo | ||
| WAVE numericalValues_orig = root:MIES_backup:LabNoteBook:ITC18USB:Device0:numericalValues | ||
|
|
||
| settings[%RAC] = GetLastSettingIndep(numericalValues_orig, sweepNo, RA_ACQ_CYCLE_ID_KEY, DATA_ACQUISITION_MODE) | ||
|
|
Comment on lines
+397
to
+416
| // todo | ||
| Make/FREE/N=(5)/D settings | ||
|
|
||
| SetDimensionLabels(settings, "RAC;SCI;SetColumn;SetCycleCount;Count", ROWS) | ||
|
|
||
| // todo | ||
| WAVE numericalValues_orig = root:MIES_backup:LabNoteBook:ITC18USB:Device0:numericalValues | ||
|
|
||
| settings[%RAC] = GetLastSettingIndep(numericalValues_orig, sweepNo, RA_ACQ_CYCLE_ID_KEY, DATA_ACQUISITION_MODE) | ||
|
|
||
| WAVE stimsetCycleIDs = GetLastSetting(numericalValues_orig, sweepNo, STIMSET_ACQ_CYCLE_ID_KEY, DATA_ACQUISITION_MODE) | ||
| settings[%SCI] = ExtractSingelValue(stimsetCycleIDs) | ||
|
|
||
| WAVE setSweepCount = GetLastSetting(numericalValues_orig, sweepNo, "Set Sweep Count", DATA_ACQUISITION_MODE) | ||
| settings[%SetColumn] = ExtractSingelValue(setSweepCount) | ||
|
|
||
| WAVE setCycleCount = GetLastSetting(numericalValues_orig, sweepNo, "Set Cycle Count", DATA_ACQUISITION_MODE) | ||
| settings[%SetCycleCount] = ExtractSingelValue(setCycleCount) | ||
|
|
||
| return settings |
Comment on lines
1299
to
1302
| Make/FREE/N=(NUM_HEADSTAGES) s.asyncAlarm = NaN | ||
| Make/FREE/N=(NUM_HEADSTAGES)/D s.da_scale = NaN | ||
|
|
||
| Make/FREE/T/N=(NUM_HEADSTAGES) s.st, s.ist, s.af, s.st_ttl, s.iaf |
Comment on lines
+1355
to
+1357
| s.ab[headstage] = ParseNumber(str, "_AB", defValue = NaN) | ||
| s.abv[headstage] = ParseNumber(str, "_ABV", defValue = NaN) | ||
| s.abvv[headstage] = ParseNumber(str, "_ABVV", defValue = NaN) |
Comment on lines
+1574
to
+1576
| PGC_SetAndActivateControl(device, "check_DataAcq_AutoBias", val = s.ab[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasV", val = s.abv[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasVrange", val = s.abvv[i]) |
| if(RoVar(GetReplayDataEnable())) | ||
| WAVE replaySettings = RD_GetReplaySettings(device) | ||
|
|
||
| stimsetAcqIDHelper[][%id] = replaySettings[%SCI] |
t-b
force-pushed
the
feature/2712-replay-data-support
branch
from
3ac64b0 to
ff700f7
Compare
Comment on lines
2015
to
2016
| for(device : devices) | ||
| Make/FREE/WAVE valuesWave = {GetLBNumericalValues(device), GetLBTextualValues(device)} | ||
|
|
||
| InvalidateLBIndexAndRowCaches_Impl(valuesWave) | ||
| endfor |
|
|
||
| s.tbp = ParseNumber(str, "_TBP", defValue = NaN) | ||
|
|
||
| s.tad = ParseNumber(str, "_TAD", defValue = 0) |
Comment on lines
+1308
to
+1310
| Make/FREE/N=(NUM_HEADSTAGES) s.asyncAD = NaN | ||
| Make/FREE/N=(NUM_HEADSTAGES) s.asyncAlarm = NaN | ||
| Make/FREE/N=(NUM_HEADSTAGES)/D s.da_scale = NaN |
Comment on lines
+1586
to
+1590
| if(s.cm[i] == I_CLAMP_MODE) | ||
| PGC_SetAndActivateControl(device, "check_DataAcq_AutoBias", val = s.ab[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasV", val = s.abv[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasVrange", val = s.abvv[i]) | ||
| endif |
Comment on lines
+1365
to
+1367
| s.ab[headstage] = ParseNumber(str, "_AB", defValue = NaN) | ||
| s.abv[headstage] = ParseNumber(str, "_ABV", defValue = NaN) | ||
| s.abvv[headstage] = ParseNumber(str, "_ABVV", defValue = NaN) |
t-b
force-pushed
the
feature/2712-replay-data-support
branch
from
ff700f7 to
8efb3ab
Compare
Collaborator
Author
|
Gad2-IRES-Cre;Ai14-863395.03.06.01_replay.pxp: The differences in the RAMP stimset is expected. The "Clamp Mode" difference is known, see #2712. Adaptive works exactly the same in this experiment. |
t-b
force-pushed
the
feature/2712-replay-data-support
branch
from
8efb3ab to
0dcf361
Compare
Comment on lines
+245
to
+249
| #ifdef AUTOMATED_TESTING | ||
| if(ROvar(GetReplayDataEnable())) | ||
| maxITI = 0.5 | ||
| endif | ||
| #endif // AUTOMATED_TESTING |
Comment on lines
+4917
to
+4921
| #ifdef AUTOMATED_TESTING | ||
| if(RoVar(GetReplayDataEnable())) | ||
| index = 0 | ||
| endif | ||
| #endif // AUTOMATED_TESTING |
Comment on lines
+242
to
+247
| elseif(GrepString(elem, "^ASYNC")) | ||
| async = ACD_ParseNumber(elem, "ASYNC") | ||
| s.asyncAD[async] = 1 | ||
|
|
||
| s.asyncAlarm[async] = ACD_ParseNumber(elem, "_ALARM") | ||
| continue |
Comment on lines
+227
to
+228
| Make/FREE/N=(NUM_HEADSTAGES) s.asyncAD = NaN | ||
| Make/FREE/N=(NUM_HEADSTAGES) s.asyncAlarm = NaN |
Comment on lines
+500
to
+504
| if(s.cm[i] == I_CLAMP_MODE) | ||
| PGC_SetAndActivateControl(device, "check_DataAcq_AutoBias", val = s.ab[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasV", val = s.abv[i]) | ||
| PGC_SetAndActivateControl(device, "setvar_DataAcq_AutoBiasVrange", val = s.abvv[i]) | ||
| endif |
|
|
||
| s.tbp = ACD_ParseNumber(str, "_TBP", defValue = NaN) | ||
|
|
||
| s.tad = ACD_ParseNumber(str, "_TAD", defValue = 0) |
Comment on lines
+1250
to
+1252
| WAVE replaySettings = RD_GetReplaySettings(device, RD_MODE_ONCE, RD_SWEEP_SELECTOR_NEXT) | ||
|
|
||
| count = replaySettings[%SetColumn] |
Comment on lines
+1278
to
+1283
| WAVE replaySettings = RD_GetReplaySettings(device, RD_MODE_ONCE, RD_SWEEP_SELECTOR_NEXT) | ||
|
|
||
| raCycleID = replaySettings[%RAC] | ||
| activeSetCount -= replaySettings[%SetColumn] | ||
|
|
||
| setEventFlag[][%PRE_SET_EVENT] = (replaySettings[%SetColumn] == 0) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current limitations:
Steps:
#define REPLAY_DATARD_PrepareMIES()RD_ReplayData("ITC18USB_DEV_0", 0)// 0 is the sweepNoRD_CompareSCI("ITC18USB_DEV_0", 0, 0)// sweepNo, HSToDos:
Async AD X [XXX]) from the LBN and return that inASD_ReadChannelOpen issues:
Clamp Modeis different fromOperating Modebut both should be the same??Close #1123