fix(common): emit per-type assetCount in DataSet representation#146
Draft
ottobolyos wants to merge 49 commits intoTrakHound:masterfrom
Draft
fix(common): emit per-type assetCount in DataSet representation#146ottobolyos wants to merge 49 commits intoTrakHound:masterfrom
ottobolyos wants to merge 49 commits intoTrakHound:masterfrom
Conversation
ottobolyos
force-pushed
the
fix/issue-132
branch
2 times, most recently
from
30467aa to
c4350ec
Compare
ottobolyos
force-pushed
the
fix/issue-132
branch
from
c4350ec to
7d2a65e
Compare
ottobolyos
force-pushed
the
fix/issue-132
branch
from
57816a3 to
0057cb9
Compare
The docs/testing/issue-130-131/ subtree carried phase-by-phase campaign writeups that referenced internal tooling (CONVENTIONS rule-book, internal section numbers, extra-files.user/ paths, internal tracker terminology). Those writeups belong in the campaign's gitignored planning area, not in the maintainer-facing public docs tree.
Adds three RED regression fixtures (Assets/Streams/Devices) that pin the cppagent v2.7.0.7 Header.validation wire shape: ctor copy, serialized property, and round-trip through To*Header(). Tests fail until the Validation property is added to the JSON-cppagent header DTOs. Source authority: cppagent v2.7.0.7 emits Header.validation on every envelope. Public defect tracker: TrakHound#130 TrakHound#131
Adds the `validation` JSON property to JsonAssetsHeader, JsonStreamsHeader,
and JsonDevicesHeader, mapped from `IMTConnect{Assets,Streams,Devices}Header.Validation`
in the constructor and copied back through `To{Assets,Streams,Devices}Header()`.
Mirrors the cppagent v2.7.0.7 wire shape that emits `Header.validation` on
every envelope. Greens the F-C17 regression tests.
Source authority: cppagent v2.7.0.7 emits Header.validation on every
envelope. Public defect tracker:
TrakHound#130
TrakHound#131
Adds RED fixture asserting that an internal static
JsonHeaderWireShapeMatrix in TestHelpers exposes a TestCaseSource-shaped
compliance matrix for the (Assets|Streams|Devices) x (2.0|2.3|2.5)
combinations consumed by per-envelope and cross-envelope E2E tests.
Tagged [Category("ComplianceMatrix")].
Centralizes the (Assets|Streams|Devices) x (2.0|2.3|2.5) matrix into
TestHelpers/JsonHeaderWireShapeMatrix as an internal static class with
TestCaseSource-shaped Cases (cross-envelope) and SchemaVersionCases
(per-envelope) generators. Per-envelope SchemaVersion fixtures and the
cross-envelope E2E fixture now pull from this single source so adding a
new schema version automatically extends both layers. Tests are tagged
[Category("ComplianceMatrix")].
Greens the F-Si-M5 RED matrix fixture; full suite passes (60/60).
Adds regression fixture asserting NormalizeDevice/AddDevice backfills the four required Device-level DataItems (Availability, AssetChanged, AssetRemoved, AssetCount) exactly once each across four starting states (null, empty, with one required, with all required) and preserves user-provided DataItems. Lets the F-P-H5 perf optimization (cast DataItems once + HashSet for type checks) refactor the inner loop without regressing behavior.
Hoists the per-required-type LINQ scan out of the four required-DataItem backfill blocks in MTConnectAgent.NormalizeDevice. The DataItems enumerable is now materialized once into a List<IDataItem> and the existing types are projected into a HashSet<string>, dropping per-block work from O(n) lookups + four ToList() allocations to O(1) lookups + one materialization. Behavior preserved per NormalizeDeviceRequiredDataItemsTests.
Adds a RED file-content fixture asserting the SchemaVersion XML-doc example on the six MTConnect.NET-Common Header types does not pin to the stale "2.5" string. Drives the F-D15 doc fix that updates the example to "2.7" or a neutral phrasing.
Updates the SchemaVersion XML-doc example on the six MTConnect.NET-Common Header interfaces and classes from the stale "2.5" snapshot to "2.7" so IntelliSense matches the current MTConnect Standard release the agent targets. Greens the F-D15 regression pin.
Adds RED file-content fixture asserting the SchemaVersion property in the three JSON-cppagent header DTOs (Assets/Streams/Devices) is preceded by an XML-doc <summary> block. Drives the F-D16 doc fix that documents the new property as the cppagent v2 wire-shape mirror.
Adds an XML-doc <summary> block above the SchemaVersion property in the
JsonAssetsHeader and JsonStreamsHeader DTOs describing the property as the
cppagent v2 wire-shape mirror. JsonDevicesHeader's SchemaVersion summary is
contributed by fix/issue-128 (envelope-vs-Header semantics) which lands
ahead of this branch per the documented merge order; the F-D16 test only
requires that a <summary> block precede [JsonPropertyName("schemaVersion")]
on each of the three DTOs and is greened by either author. Greens the
F-D16 doc pin.
The F-P-H5 refactor (cast DataItems to List<IDataItem> once + HashSet for type checks) inadvertently dropped the spec-required line that overrides the auto-injected AssetCount's Representation from VALUE to DATA_SET. Restore the override + the explanatory comment citing the MTConnect Part 2 UML id. Cross-PR interaction: per the campaign convention, the consuming PR fixes the test that depends on the refactored default — here the consuming surface is the fix/issue-132 regression suite, but the fix must land on this branch since that's where F-P-H5 dropped the line.
`new HashSet<T>(int capacity)` was added in .NET Framework 4.7.2 + netstandard 2.1 + .NET Core 2.0; net461 / net47 / net471 / netstandard2.0 only know `(IEqualityComparer<T>)` and reject the int with CS1503. The NormalizeDevice DataItem-type cache landed in 355b74e used the capacity ctor unconditionally, which broke `dotnet pack -c Release` on those TFMs. Wrap the capacity-aware allocation in a conditional so: - net472 + / netstandard2.1 + / netcoreapp2.0 + → keep the capacity hint (the perf optimization the F-P-H5 commit was after). - net461 / net471 / net47 / netstandard2.0 → fall back to the parameterless ctor; behaviour identical apart from one extra rehash on the first hot-path call. Surfaced via `dotnet pack -c Release` from the integration branch.
Regression pin for TrakHound#132. Every entry point that auto-injects an AssetCountDataItem (single AddDevice, batched AddDevices, agent constructed via the IAgentConfiguration overload) must produce a DataItem whose Representation is DATA_SET. Defends against a regenerated AssetCountDataItem.g.cs reintroducing the DefaultRepresentation = VALUE bug or a refactor of NormalizeDevice forgetting the override.
The docs/testing/issue-132/ subtree carried phase-by-phase campaign writeups that referenced internal tooling (CONVENTIONS rule-book, internal section numbers, extra-files.user/ paths, internal tracker terminology). Those writeups belong in the campaign's gitignored planning area, not in the maintainer-facing public docs tree.
The class-level summary on NormalizeDeviceRequiredDataItemsTests cited an internal audit-finding code when describing the inner-loop perf optimisation it leaves room for; rewrite the summary so the same point is made inline.
ottobolyos
force-pushed
the
fix/issue-132
branch
from
0057cb9 to
0111e87
Compare
The importer's Render* methods loaded Scriban templates from lowercase `csharp/templates/` and similar paths, and joined output paths via the literal Windows separator `\\`. Linux and macOS filesystems are case-sensitive and use `/`, so the importer silently no-op'd on every Render call (the path-existence check returned false) and emitted no .g.cs files. Surface the same case-sensitive paths the on-disk template directories actually use (`CSharp/Templates`, `Json-cppagent/Templates`, `Xml/Templates`) and replace the literal `\\` with `Path.DirectorySeparatorChar` so the output path resolves on every platform. Also accept `--xmi <path>` and `--output <repo-root>` CLI flags so the importer can run against any pinned XMI from any working tree. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…lization
The SysML importer determines a DataItem's `DefaultRepresentation` from two
sources, both of which need to land:
1. **EventEnum prose marker.** Each EventEnum literal carries a Description
block with marker tokens; `{{term(data set)}}` flags the literal as
DATA_SET. The importer parses the prose and emits `DefaultRepresentation
= DATA_SET` accordingly.
2. **Result-class generalization walk.** Some DataItems anchor on a result
*class* (not an enum literal) whose generalization chain ends at the
`DataSet` parent class. Walking the generalization chain and matching
the parent name lets the importer emit DATA_SET on the resulting
class-level metadata, which is the surface the runtime actually reads.
Both paths converge on the same property; without both, the asset-count
and the 9 result-class DataItems each lost the DATA_SET classification on
re-import — the prose-marker path covered the enum-literal-anchored set
and the generalization-walk path covered the result-class-anchored set.
EOF
…tion Regen output from the importer fixes in the parent commit. Two effects: - `AssetCountDataItem.g.cs` now sets `DefaultRepresentation = DATA_SET` via the EventEnum prose marker path. Per the HIGH-1 audit decision this matches the spec's auto-emission contract for `assetCount` (DATA_SET, keyed by AssetType, mirrored from `AssetCounts`). - The 9 result-class DataItems whose SysML class generalizes to `DataSet` now carry `DefaultRepresentation = DATA_SET` at the class level: their `.g.cs` files reflect the importer walking the generalization chain and matching the `DataSet` parent. No hand-edits in this commit — pure regen output of the importer fix.
Single fixture covering both the AssetCount entry-point pin and the 9 result-class types' class-level `DefaultRepresentation = DATA_SET` pins — plus the `MinimumVersion = v2.0` pin on AssetCountDataItem (spec adds the assetCount auto-emit at v2.0; no earlier). Source authority cited inline on each fixture: SysML model + cppagent reference printer behavior + the relevant XSD `representation` attribute, per CONVENTIONS §15.
The Interface DataItem template emitted no SubTypes block even though every Interface DataItem class in the SysML XMI declares two subtype classes — `<Name>.Request` and `<Name>.Response`. Mirror the regular DataItem template so each Interface DataItem gets a nested SubTypes enum, a constructor accepting a typed SubTypes value, and the matching GetSubTypeId / GetSubTypeDescription helpers. Use the `new` keyword on each member so the derived enum and statics shadow the base InterfaceDataItem.SubTypes pair without a compile-time warning. Sources: - SysML XMI: https://github.com/mtconnect/mtconnect_sysml_model v2.7. Each interface event class (CloseChuck, CloseDoor, OpenChuck, OpenDoor, MaterialChange, MaterialFeed, MaterialLoad, MaterialRetract, MaterialUnload, PartChange) carries two immediate sub-classes named `<Name>.Request` and `<Name>.Response` (e.g. CloseChuck.Request at XMI line 48715, CloseChuck.Response at line 48736). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Regenerate the ten interface DataItem types so each carries a nested `SubTypes` enum mirroring the SysML-declared `<Name>.Request` and `<Name>.Response` subtype classes: - CloseChuckDataItem / CloseDoorDataItem - OpenChuckDataItem / OpenDoorDataItem - MaterialChangeDataItem / MaterialFeedDataItem - MaterialLoadDataItem / MaterialRetractDataItem - MaterialUnloadDataItem / PartChangeDataItem Each derived class now exposes a typed constructor `(string parentId, SubTypes subType)`, a matching GetSubTypeId / GetSubTypeDescription pair, and an override of SubTypeDescription so consumers can construct a fully-spec-conformant Interface DataItem without resorting to the base InterfaceDataItem.SubTypes literals. Sources: - SysML XMI: https://github.com/mtconnect/mtconnect_sysml_model v2.7. Each interface event class declares two immediate sub-classes named `<Name>.Request` and `<Name>.Response`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cover the ten Interface DataItem types that the SysML XMI marks with `<Name>.Request` and `<Name>.Response` subtype classes (CloseChuck, CloseDoor, OpenChuck, OpenDoor, MaterialChange, MaterialFeed, MaterialLoad, MaterialRetract, MaterialUnload, PartChange). Two parametric assertions per type pin (a) that the derived class exposes a nested SubTypes enum, and (b) that its members include both REQUEST and RESPONSE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The SysML model declares `introduced='2.0'` on the Controllers organizer. The importer's hardcoded-version table for component organizers carried a stale `v1.0` override that won out over the SysML metadata, so the regen kept emitting `MinimumVersion = V1_0` on `ControllersComponent.g.cs`. Remove the override row + regen the component metadata. The TDD pin asserts the class-level `MinimumVersion = MTConnectVersions.Version20` so the value can't drift back during a future regen.
…mentModel pipeline
Lift the ten Pallet measurement subclasses (Weight, Height, Width,
Length, Swing plus their Loaded* counterparts) into the rich
measurement-model pipeline so they regenerate with the same
TypeId / three-constructor scaffolding the CuttingTools measurement
DTOs already enjoy.
Three importer wires need to land together:
- `MTConnectAssetInformationModel.ParsePallets` now calls
`MTConnectMeasurementModel.Parse(..., "PhysicalAsset",
"Assets.Pallet", umlClasses)` over the Measurements sub-package
instead of the generic `MTConnectClassModel.Parse`. The abstract
`Measurement` class is filtered out of the rich parse (the
measurement-model constructor auto-suffixes `Measurement`, which
would otherwise produce `MeasurementMeasurement`) and re-parsed
separately via the regular class pipeline so it can carry the
partial-class scaffolding the rich subclasses chain to.
- `MeasurementModel.RenderModel` now loads the correctly-named
`Pallets.Measurement.scriban` template instead of the
`Assets.Measurement.scriban` placeholder that never existed on
disk. The dead-code condition (`File.Exists` returns false →
RenderModel returns null) silently dropped every Pallet
measurement render attempt; the rename makes the renderer
emit through the template that ships with the importer.
- `TemplateRenderer.Render` now routes
`MTConnectMeasurementModel` instances whose Id starts with
`Assets.Pallet.` through `MeasurementModel.Create`, mirroring
the existing CuttingTools branch. A renderer override pins
`Assets.Pallet.Measurement` as `partial` + concrete so the
hand-written partial in MTConnect.NET-Common can supply
`Type` and the `Measurement(IMeasurement)` ctor the per-subtype
template's `: base(measurement)` chain calls into.
The `Pallets.Measurement.scriban` template itself is updated to
use `{{namespace}}` (instead of a hard-coded
`MTConnect.Assets.Pallet.Measurements`) so the rendered files
sit in the same `MTConnect.Assets.Pallet` namespace as the rest
of the Pallet asset model, and to drop the `Code = CodeId`
assignments — the SysML's Pallet Measurement abstract class
declares no `code` property, so per-subclass `CodeId` is always
the empty string and the rich template no longer pretends
otherwise.
Sources:
- SysML XMI: https://github.com/mtconnect/mtconnect_sysml_model
v2.7. Pallet Measurement abstract base UML ID
`_2024x_68e0225_1727793846441_986747_23754` (no `code`
property), with ten concrete subclasses
generalizing from it under the `Pallet > Measurements`
package.
- Reference implementation: cppagent
`src/mtconnect/asset/physical_asset.cpp`
`getMeasurementsFactory()` registers a single generic
measurement factory under regex `.+`; the per-type DTO
scaffolding is a .NET-side ergonomics convenience that
mirrors what CuttingTools already gets.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…olding
Regen output from the importer fixes in the parent commit. The ten
Pallet measurement subclasses now carry the same shape the
CuttingTools measurement DTOs have used since v1.x:
- a `public const string TypeId = "<SysMLClassName>"` discriminator;
- a `public const string CodeId = ""` (Pallet measurements bind to
no MeasurementCodeEnum, per the SysML Pallet `Measurement`
abstract class which carries no `code` property);
- a default constructor that stamps `Type = TypeId`;
- a `(double value)` constructor that stamps `Type` and `Value`;
- a `(IMeasurement measurement)` constructor that chains to the
base partial via `: base(measurement)` and re-stamps `Type`.
Subclasses covered: WeightMeasurement, HeightMeasurement,
WidthMeasurement, LengthMeasurement, SwingMeasurement,
LoadedWeightMeasurement, LoadedHeightMeasurement,
LoadedWidthMeasurement, LoadedLengthMeasurement,
LoadedSwingMeasurement.
The abstract `Measurement.g.cs` is regenerated as a `partial class`
(no longer `abstract`) so the hand-written partial at
`Assets/Pallet/Measurement.cs` can supply the `Type` setter and
the `Measurement(IMeasurement)` copy constructor that the rich
subclasses' `: base(measurement)` ctor chain depends on. This
mirrors the long-standing CuttingTools partial pattern at
`Assets/CuttingTools/Measurement.{g.cs,cs}`. The `IMeasurement`
interface in the same package picks up `partial` for symmetry —
again matching the CuttingTools precedent.
The hand-written `Assets/Pallet/Measurement.cs` partial is
new (Pallet had no equivalent before) and contributes:
- `string Type { get; set; }` — the wire-side type discriminator
the subclass constructors stamp.
- `Measurement()` default ctor.
- `Measurement(IMeasurement measurement)` copy ctor that mirrors
the field-by-field copy `CuttingTools.Measurement.cs` does
(Value, Nominal, Minimum, Maximum, SignificantDigits,
NativeUnits, Units).
No hand-edits in this commit beyond the new partial — the .g.cs
files are pure regen output of the importer fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cover the ten Pallet measurement subtypes (Weight, Height, Width, Length, Swing plus their Loaded* counterparts) with a parametric NUnit fixture pinning the rich-template contract: - `TypeId` const equals the SysML class name verbatim. - `CodeId` const is the empty string (Pallet measurements have no MeasurementCode binding, in contrast to the CuttingTools ToolingMeasurement subclasses which carry an ISO-13399 code). - The default constructor stamps `Type = TypeId`. - The `(double)` constructor stamps `Type` + `Value`. - The `(IMeasurement)` constructor copies all transferable fields from a source measurement and re-stamps `Type`. - Each subclass directly derives from `MTConnect.Assets.Pallet.Measurement` so the `: base(measurement)` chain wired by the per-subtype template terminates on the hand-written partial. The fixture is parametric on the (Type, ExpectedTypeId) pair so a future regen accidentally renaming a TypeId, dropping a constructor, or re-rooting a subclass under a different base trips a single targeted assertion rather than producing an opaque downstream serialiser failure. Sources: - SysML XMI: https://github.com/mtconnect/mtconnect_sysml_model v2.7. Pallet Measurement abstract base UML ID `_2024x_68e0225_1727793846441_986747_23754` with ten concrete subclasses generalizing from it. - Reference implementation: cppagent generic `getMeasurementsFactory()` in `physical_asset.cpp` registered under regex `.+`; the per-type DTO scaffolding is .NET-side ergonomics, exactly the contract this fixture pins. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #147.
Summary
Fixes #132 — the agent auto-injects an
AssetCountDataItemfor every device whoseassetBufferSize > 0, but the generated defaultDefaultRepresentationwasVALUE, producing a scalar EVENT on the Probe instead of the cppagent-reference DATA_SET shape that strict JSON-CPPAGENT-MQTT consumers expect. The MTConnect prose standard (Part 4 — Assets Information Model) and the cppagent reference implementation both specify ASSET_COUNT as aDATA_SETrepresentation.The PR also extends to a class of related generator gaps surfaced by a deep XMI-vs-XSD-vs-
.g.csaudit run during this branch's lifetime — every fix below shares the same root cause (a generator hardcoded constant where the SysML model carries the authoritative value).Generator + regenerated
.g.csfixesAssetCountDataItem.DefaultRepresentationflipped fromVALUEtoDATA_SET. Importer now reads the EventEnum-literal description's{{term(data set)}}marker as the canonical-default signal for primitive-result types whose representation is encoded only in prose. Applies cleanly with no other DataItem affected.MTConnectDataItemType.cspreviously hardcodedRepresentation = "TABLE"wheneverresultwas auml:Class, regardless of which class. The importer now walks the result class'sgeneralizationchain and picksTABLE(parent namedTable) /DATA_SET(parent namedDataSet) /TIME_SERIES(parent namedTimeSeries); falls back toTABLEfor template-binding cases (WORK_OFFSETS,TOOL_OFFSETS). 9.g.csfiles re-emitted with the correct representation:AlarmLimit(s),ControlLimit(s),LocationAddress,LocationSpatialGeographic,SpecificationLimit(s),SensorAttachment— all flipTABLE→DATA_SET..g.csfiles inInterfaces/were missing theenum SubTypes { REQUEST, RESPONSE }block that the SysML defines as<TypeName>.Request/<TypeName>.Responseper Interface action. Importer template now emits the same SubTypes shape regular DataItems carry. Files re-emitted:CloseChuck,CloseDoor,OpenChuck,OpenDoor,MaterialChange,MaterialFeed,MaterialLoad,MaterialRetract,MaterialUnload,PartChange.MinimumVersionwas hand-codedVersion10via a renderer override that pre-empted the SysML'sintroduced='2.0'. Removed the override;.g.csregenerates withVersion20per the model.Cross-platform importer fix (prerequisite for the regen on Linux)
\\separators, lowercasecsharp/templates/paths against case-correctCSharp/Templates/, hardcodedC:\temp\mtconnect-model.jsonoutput path). Fixed viaPath.DirectorySeparatorChar+ case-correct paths + a CLI-driven--xmi/--outputflag. Without these, every regen on Linux silently produced no output.Tests
tests/MTConnect.NET-Common-Tests/adds:Devices/DataItems/AssetCountDataItemDefaultRepresentationTests.cs— pinsAssetCountDataItem.DefaultRepresentation == DATA_SETat the class-level + parametric ctors + reflection signature.Devices/DataItems/StructuredRepresentationClassifierTests.cs— parametric pins for the 9 Result-Class types'DefaultRepresentation.Interfaces/InterfaceDataItemSubTypesTests.cs— parametric pins for REQUEST/RESPONSE SubTypes presence on each of the 10 Interface DataItems.Devices/Components/ControllersComponentMinimumVersionTests.cs— pinsVersion20.Breaking change
Downstream consumers parsing
AssetCountas a scalar EVENT must update to consume the DATA_SET representation. The wire shape now matches the cppagent reference and the MTConnect prose standard. (See the cross-source mismatch writeup atextra-files.user/upstream/issues/01-…— gitignored, available on request — for the SysML-vs-prose-vs-cppagent disagreement on this specific type's representation classification, where the prose + cppagent agree on DATA_SET and the SysML XMI implies VALUE; cppagent + prose are authoritative for the JSON-CPPAGENT-MQTT format the JSON-cppagent library targets.)The
Depends on #147line above marks the cross-PR dependency:fix/issue-132is rebased on top offix/issue-130-131, so its diff againstupstream/mastertransitively includes #147's commits. Land #147 first.