Skip to content

Add helper function GetWaveDimensionality and use it throughout codebase#2686

Merged
t-b merged 5 commits intomainfrom
copilot/add-get-wave-dimensionality-function
Apr 7, 2026
Merged

Add helper function GetWaveDimensionality and use it throughout codebase#2686
t-b merged 5 commits intomainfrom
copilot/add-get-wave-dimensionality-function

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 28, 2026 •
edited by t-b
Loading

Thanks for opening a PR in MIES ✨!

  • Code can only be merged if the continous integration tests pass
  • Please ensure that the branch is named correctly. See
    here for the detailed explanation.

Description

WaveDims returns dimension count regardless of trailing size-1 dimensions:

Make/N=(5, 1)/o data
print WaveDims(data)  // Returns 2, but effectively 1D

Adds GetWaveDimensionality(wv) — a threadsafe function that returns the highest dimension with size > 1:

Make/N=(5, 1) wv
print GetWaveDimensionality(wv) == ROWS  // true

Changes

  • MIES_Utilities_WaveHandling.ipf: Added GetWaveDimensionality function with WAVE/Z parameter to allow null wave references
  • UTF_Utils_WaveHandling.ipf: Added 14 tests covering 1D-4D waves, trailing-1 dimensions, mixed cases, empty waves, and threadsafe operation
  • Codebase-wide adoption: Updated 14 source files and 1 documentation file to use the new function:
    • MIES_SweepFormula_Operations.ipf: Updated avg, max, min, rms, stdev, variance operations (WaveDims(input) <= 2 → GetWaveDimensionality(input) <= COLS)
    • MIES_SweepFormula.ipf: Updated text wave checks (WaveDims(wv) > 1 → GetWaveDimensionality(wv) > ROWS)
    • MIES_DataConfigurator.ipf, MIES_ExperimentDocumentation.ipf, MIES_JSONWaveNotes.ipf, MIES_MiesUtilities_Algorithm.ipf, MIES_SweepFormula_Executor.ipf, MIES_SweepFormula_Helpers.ipf, MIES_SweepFormula_Operations_Select.ipf, MIES_SweepFormula_PSX.ipf, MIES_ThreadsafeUtilities.ipf, MIES_Utilities_Algorithm.ipf, MIES_Utilities_Conversions.ipf, MIES_Utilities_WaveHandling.ipf: Replaced various DimSize-based 1D checks with GetWaveDimensionality(wv) == ROWS
    • SweepFormula.rst: Updated example code in documentation

📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

Close #2138

Copilot AI assigned Copilot and t-b Mar 28, 2026
Copilot AI linked an issue Mar 28, 2026 that may be closed by this pull request
Add helper function to return the wave dimensionality #2138
Closed
Copilot AI changed the title [WIP] Add helper function to return the wave dimensionality Add helper function GetWaveDimensionality to return wave dimensionality Mar 28, 2026
Copilot finished work on behalf of t-b March 28, 2026 22:25
Copilot AI requested a review from t-b March 28, 2026 22:25
@t-b
Copy link
Copy Markdown
Collaborator

t-b commented Mar 29, 2026

@copilot Looks good so far. Now use the new function in the appropriate places all over the code base. A first try would be to have a look at:

$git grep -iIP --show-function  'ASSERT.*[1-4]D.*'
Packages/MIES/MIES_DataConfigurator.ipf=static Function DC_FillDAQDataWaveForTP(string device, STRUCT DataConfigurationResult &s)
Packages/MIES/MIES_DataConfigurator.ipf:        ASSERT(DimSize(s.testPulse, COLS) <= 1, "Expected a 1D testpulse wave")
Packages/MIES/MIES_DataConfigurator.ipf=static Function DC_FillDAQDataWaveForDAQ(string device, STRUCT DataConfigurationResult &s)
Packages/MIES/MIES_DataConfigurator.ipf:                        ASSERT(DimSize(s.testPulse, COLS) <= 1, "Expected a 1D testpulse wave")
Packages/MIES/MIES_ExperimentDocumentation.ipf=Function ED_AddEntryToLabnotebook(string device, string key, WAVE values, [string unit, variable tolerance, variable overrideSweepNo])
Packages/MIES/MIES_ExperimentDocumentation.ipf: ASSERT(DimSize(values, COLS) == 0, "wv must be 1D")
Packages/MIES/MIES_JSONWaveNotes.ipf=threadsafe Function JWN_SetWaveInWaveNote(WAVE wv, string jsonPath, WAVE noteWave)
Packages/MIES/MIES_JSONWaveNotes.ipf:           ASSERT_TS(DimSize(noteWave, COLS) <= 1, "Expected only a 1D wave reference wave")
Packages/MIES/MIES_MiesUtilities_Algorithm.ipf=threadsafe Function GetNumberOfUsefulThreads(WAVE dims)
Packages/MIES/MIES_MiesUtilities_Algorithm.ipf: ASSERT_TS(numCols <= 1, "Expected a 1D wave")
Packages/MIES/MIES_SweepFormula.ipf=static Function [variable dataCnt, variable gdIndex, string annotation, variable formulaAddedOncePerDataset] SF_CreateTracesForResultsImpl(STRUCT SF_PlotterGraphStruct &pg, WAVE wvResultY, WAVE/Z wvResultX, variable dataNum, variable showInTable, WAVE plotFormData)
Packages/MIES/MIES_SweepFormula.ipf:    SFH_ASSERT(!(IsTextWave(wvResultY) && WaveDims(wvResultY) > 1), "Plotter got 2d+ text wave as y data.")
Packages/MIES/MIES_SweepFormula.ipf:            SFH_ASSERT(!(IsTextWave(wvResultX) && WaveDims(wvResultX) > 1), "Plotter got 2d+ text wave as x data.")
Packages/MIES/MIES_SweepFormula_Executor.ipf=Function/WAVE SFE_FormulaExecutor(STRUCT SF_ExecutionData &exd, [variable srcLocId])
Packages/MIES/MIES_SweepFormula_Executor.ipf:                   SFH_ASSERT(WaveDims(subArray) < MAX_DIMENSION_COUNT, "Encountered 4d sub array at " + exd.jsonPath)
Packages/MIES/MIES_SweepFormula_Helpers.ipf=Function [WAVE adaptedRange, WAVE/T epochRangeNames] SFH_GetNumericRangeFromEpoch(string graph, WAVE numericalValues, WAVE textualValues, WAVE range, variable sweepNo, variable chanType, variable chanNr, variable mapIndex)
Packages/MIES/MIES_SweepFormula_Helpers.ipf:    SFH_ASSERT(IsTextWave(epochPatterns) && !DimSize(epochPatterns, COLS), "Expected 1d text wave for epoch specification")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationAvgImplIn(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "avg accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationMaxImpl(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "max accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationMinImpl(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "min accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationRMSImpl(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "rms accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationStdevImpl(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "stdev accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations.ipf=static Function/WAVE SFO_OperationVarianceImpl(WAVE/Z input)
Packages/MIES/MIES_SweepFormula_Operations.ipf: SFH_ASSERT(WaveDims(input) <= 2, "variance accepts only upto 2d data")
Packages/MIES/MIES_SweepFormula_Operations_Select.ipf=Function/WAVE SFOS_OperationSelectSweeps(STRUCT SF_ExecutionData &exd)
Packages/MIES/MIES_SweepFormula_Operations_Select.ipf:                  SFH_ASSERT(!DimSize(data, COLS), "Argument of selsweeps must be a number or a 1d numeric array")
Packages/MIES/MIES_SweepFormula_PSX.ipf=static Function PSX_UpdateEventWaves(string win, [variable val, variable index, variable toggle, WAVE/Z indizes, variable writeState, variable stateType, variable comboIndex])
Packages/MIES/MIES_SweepFormula_PSX.ipf:                ASSERT(DimSize(indizes, COLS) <= 1, "Expected 1D wave")
Packages/MIES/MIES_ThreadsafeUtilities.ipf=Function/WAVE TS_GetNewestFromThreadQueueMult(variable tgID, WAVE/T varNames, [variable timeout_default, variable timeout_tries])
Packages/MIES/MIES_ThreadsafeUtilities.ipf:     ASSERT_TS(DimSize(varNames, COLS) == 0, "Expected a 1D wave")
Packages/MIES/MIES_Utilities_Algorithm.ipf=threadsafe Function/WAVE GetUniqueEntries(WAVE wv, [variable caseSensitive, variable dontDuplicate])
Packages/MIES/MIES_Utilities_Algorithm.ipf:     ASSERT_TS(numRows == numpnts(wv), "Wave must be 1D")
Packages/MIES/MIES_Utilities_Algorithm.ipf=threadsafe static Function/WAVE GetUniqueTextEntries(WAVE/T wv, [variable caseSensitive, variable dontDuplicate])
Packages/MIES/MIES_Utilities_Algorithm.ipf:     ASSERT_TS(numEntries == numpnts(wv), "Wave must be 1D.")
Packages/MIES/MIES_Utilities_Algorithm.ipf=threadsafe Function/WAVE GetSetDifference(WAVE wave1, WAVE wave2, [variable getIndices])
Packages/MIES/MIES_Utilities_Algorithm.ipf:     ASSERT_TS(!DimSize(wave1, COLS), "input wave1 must be 1d")
Packages/MIES/MIES_Utilities_Algorithm.ipf=threadsafe Function/WAVE GetSetIntersection(WAVE wave1, WAVE wave2, [variable getIndices])
Packages/MIES/MIES_Utilities_Algorithm.ipf:     ASSERT_TS(!DimSize(wave1, COLS) && !DimSize(wave2, COLS), "input waves must be 1d")
Packages/MIES/MIES_Utilities_Algorithm.ipf=threadsafe Function BinarySearchText(WAVE/T theWave, string theText, [variable caseSensitive, variable startPos, variable endPos])
Packages/MIES/MIES_Utilities_Algorithm.ipf:     ASSERT_TS(DimSize(theWave, COLS) <= 1, "Only works with 1D waves")
Packages/MIES/MIES_Utilities_Conversions.ipf=Function/S GetCodeForWaveContents(WAVE/T wv)
Packages/MIES/MIES_Utilities_Conversions.ipf:   ASSERT(DimSize(wv, COLS) <= 1, "Does only support 1D waves")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=Function/WAVE DuplicateSubRange(WAVE wv, variable first, variable last)
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT(DimSize(wv, COLS) == 0, "Requires 1D wave")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=Function/WAVE MergeTwoWaves(WAVE wv1, WAVE wv2)
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT(DimSize(wv1, COLS) <= 1, "Expected 1D wave")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=threadsafe Function ChangeWaveLock(WAVE/WAVE wv, variable val)
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT_TS(DimSize(wv, ROWS) == numpnts(wv), "Expected a 1D wave")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=Function/WAVE ZapNullRefs(WAVE/WAVE input)
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT(Dimsize(input, COLS) == 0, "input must be 1D")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=threadsafe Function SetDimensionLabelsFromWaveContents(WAVE wv, [string prefix, string suffix, variable strict])
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT_TS(!DimSize(wv, COLS), "Wave must be 1d")
Packages/MIES/MIES_Utilities_WaveHandling.ipf=threadsafe Function/WAVE FindNeighbourWithPredicate(WAVE wv, FUNCREF FindNeighbourPrototype pred)
Packages/MIES/MIES_Utilities_WaveHandling.ipf:  ASSERT_TS(numPoints == numpnts(wv), "Wave must be 1D")
Packages/doc/SweepFormula.rst=Example code for a typical operation taking three arguments, the first argument is some kind of input data.
Packages/doc/SweepFormula.rst:    SF_ASSERT(WaveDims(input) <= 2, "opname accepts only upto 2d data")

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 29, 2026

@copilot Looks good so far. Now use the new function in the appropriate places all over the code base. A first try would be to have a look at:

...

Applied GetWaveDimensionality across the codebase in commits 12e8eb5 and 38a4ab6. Updated 14 source files and the SweepFormula.rst documentation:

  • SweepFormula operations (avg, max, min, rms, stdev, variance): WaveDims(input) <= 2 → GetWaveDimensionality(input) <= COLS
  • SweepFormula plotter: WaveDims(wv) > 1 → GetWaveDimensionality(wv) > ROWS
  • Various 1D assertions in DataConfigurator, ExperimentDocumentation, JSONWaveNotes, ThreadsafeUtilities, Utilities_Algorithm, Utilities_Conversions, Utilities_WaveHandling, and others: replaced DimSize-based checks with GetWaveDimensionality(wv) == ROWS

Copilot AI changed the title Add helper function GetWaveDimensionality to return wave dimensionality Add helper function GetWaveDimensionality and use it throughout codebase Mar 29, 2026
Copilot finished work on behalf of t-b March 29, 2026 14:36
@t-b
Copy link
Copy Markdown
Collaborator

t-b commented Mar 30, 2026

@copilot The tests for GetWaveDimensionality are missing the case where the input wave is completely empty.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 30, 2026

@copilot The tests for GetWaveDimensionality are missing the case where the input wave is completely empty.

Added test case GWD_EmptyWave() for a completely empty wave (0 points) in commit 49af1ac.

@t-b
Copy link
Copy Markdown
Collaborator

t-b commented Mar 30, 2026

@copilot The function GetWaveDimensionality should use WAVE/Z wv for the input parameter.

Copilot finished work on behalf of t-b March 30, 2026 13:09
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 30, 2026

@copilot The function GetWaveDimensionality should use WAVE/Z wv for the input parameter.

Changed to WAVE/Z wv in commit f6a5e1c.

Copilot finished work on behalf of t-b March 30, 2026 13:13
@t-b t-b force-pushed the copilot/add-get-wave-dimensionality-function branch from c42ca7b to 4a5b932 Compare March 31, 2026 18:49
Copilot AI and others added 5 commits March 31, 2026 20:53
@t-b
Add GetWaveDimensionality function and tests
f9bf320 
Agent-Logs-Url: https://github.com/AllenInstitute/MIES/sessions/b10fcbd6-09e7-4c61-a617-c02789769fb6
Agent-Logs-Url: https://github.com/AllenInstitute/MIES/sessions/ce13e3dd-2eb2-460b-8efc-8e9532fafd24
Agent-Logs-Url: https://github.com/AllenInstitute/MIES/sessions/c0d50d5b-2fe0-4218-8f24-d38ba2f0cd46

Co-authored-by: t-b <1185677+t-b@users.noreply.github.com>
@t-b
Use GetWaveDimensionality throughout codebase
bd23899 
Agent-Logs-Url: https://github.com/AllenInstitute/MIES/sessions/c9c9143a-f661-4379-8487-0c45d58a5f7e

Co-authored-by: t-b <1185677+t-b@users.noreply.github.com>
@t-b
Simplify redundant checks in SweepFormula_Helpers
02f246f 
Agent-Logs-Url: https://github.com/AllenInstitute/MIES/sessions/c9c9143a-f661-4379-8487-0c45d58a5f7e

Co-authored-by: t-b <1185677+t-b@users.noreply.github.com>
@t-b
test/TestZapNullRefs: Fix it
e380658 
We now use GetWaveDimensionality for checking N-D ness and that accepts
single column waves. So let's remove that check.
@t-b
TP_AutoFitBaseline: Prefer GetWaveDimensionality
585501f
@t-b t-b force-pushed the copilot/add-get-wave-dimensionality-function branch from 4a5b932 to 585501f Compare March 31, 2026 18:53
@t-b
Copy link
Copy Markdown
Collaborator

t-b commented Mar 31, 2026

Polished commits a bit and also added minor fixes. Will merge once CI passes.

@t-b t-b marked this pull request as ready for review March 31, 2026 18:59
@t-b t-b requested a review from timjarsky as a code owner March 31, 2026 18:59
Copilot AI review requested due to automatic review settings March 31, 2026 18:59
@t-b t-b requested a review from MichaelHuth as a code owner March 31, 2026 18:59
Copilot AI reviewed Mar 31, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a threadsafe helper (GetWaveDimensionality) to treat waves with trailing size-1 dimensions as effectively lower-dimensional, and adopts it across MIES to replace WaveDims/DimSize-based dimensionality checks that were producing incorrect results.

Changes:

  • Added GetWaveDimensionality(WAVE/Z wv) to determine the highest dimension with DimSize > 1 (else ROWS).
  • Added a dedicated unit-test block covering 1D–4D cases, trailing-1 dimensions, empty waves, and threadsafe execution.
  • Replaced many “is 1D?” assertions and several SweepFormula dimensionality guards throughout the codebase and updated the SweepFormula docs example.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Packages/tests/Basic/UTF_Utils_WaveHandling.ipf Removes an outdated ZapNullRefs expectation and adds comprehensive GetWaveDimensionality tests.
Packages/MIES/MIES_Utilities_WaveHandling.ipf Adds GetWaveDimensionality and updates multiple 1D precondition checks to use it.
Packages/MIES/MIES_Utilities_Conversions.ipf Updates 1D-only assertion to use GetWaveDimensionality.
Packages/MIES/MIES_Utilities_Algorithm.ipf Updates several 1D assertions (set ops, search helpers) to use GetWaveDimensionality.
Packages/MIES/MIES_ThreadsafeUtilities.ipf Updates 1D assertion for varNames to use GetWaveDimensionality.
Packages/MIES/MIES_TestPulse.ipf Updates data dimensionality assertion to use GetWaveDimensionality.
Packages/MIES/MIES_SweepFormula.ipf Updates plotter text-wave dimensionality assertions to use GetWaveDimensionality.
Packages/MIES/MIES_SweepFormula_PSX.ipf Updates “Expected 1D wave” assertion to use GetWaveDimensionality.
Packages/MIES/MIES_SweepFormula_Operations.ipf Updates several operations’ “up to 2D” guards to use GetWaveDimensionality <= COLS.
Packages/MIES/MIES_SweepFormula_Operations_Select.ipf Updates selsweeps argument dimensionality assertion to use GetWaveDimensionality.
Packages/MIES/MIES_SweepFormula_Helpers.ipf Adjusts scalar extraction checks (now using numpnts==1 in two places) and updates an epoch-pattern 1D check to use GetWaveDimensionality.
Packages/MIES/MIES_SweepFormula_Executor.ipf Updates the 4D subarray guard to use GetWaveDimensionality.
Packages/MIES/MIES_MiesUtilities_Algorithm.ipf Updates dims-wave 1D assertion in GetNumberOfUsefulThreads to use GetWaveDimensionality.
Packages/MIES/MIES_JSONWaveNotes.ipf Updates 1D wave-ref-wave assertion to use GetWaveDimensionality.
Packages/MIES/MIES_ExperimentDocumentation.ipf Updates “values must be 1D” assertion to use GetWaveDimensionality.
Packages/MIES/MIES_DataConfigurator.ipf Updates testpulse dimensionality assertions to use GetWaveDimensionality.
Packages/doc/SweepFormula.rst Updates documentation example to use GetWaveDimensionality for the 2D limit check.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Packages/MIES/MIES_SweepFormula_Executor.ipf
@t-b t-b merged commit 5e7e638 into main Apr 7, 2026
67 of 70 checks passed
@t-b t-b deleted the copilot/add-get-wave-dimensionality-function branch April 7, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@timjarsky timjarsky Awaiting requested review from timjarsky timjarsky is a code owner

@MichaelHuth MichaelHuth Awaiting requested review from MichaelHuth MichaelHuth is a code owner

+1 more reviewer

@t-b t-b t-b approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@t-b t-b

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add helper function to return the wave dimensionality

3 participants

@t-b