Skip to content

202. Metadata

Jump to bottom Edit New page
Ovsep Avakian edited this page Jun 1, 2026 · 2 revisions

Overview

Labels and scalars are a flexible metadata system in F-Mesh, inspired by Kubernetes labels. They provide a common API for attaching metadata to signals, components, ports, and collections. Labels are string key-value pairs; scalars are numeric key-value pairs (string → float64).

Both types live in the meta package. All labeled entities share the same query API, making it easy to work with metadata consistently throughout your mesh. Metadata is particularly powerful when combined with F-Mesh's collection APIs for filtering and querying.

For complete API documentation, see the meta package.

Core Concepts

Label-Value Pairs

Labels are simple string key-value pairs:

  • Label: A unique identifier (e.g., "priority", "source", "type")
  • Value: A string value (e.g., "high", "sensor1", "temperature")

Scalar Pairs

Scalars are numeric metadata:

  • Name: A unique identifier (e.g., "weight", "confidence", "latency_ms")
  • Value: A float64 value

Two API styles: CoW vs mutating

F-Mesh uses two different naming conventions depending on whether the entity is immutable or mutable:

Entity Style Add one Add many Replace all Remove
signal.Signal, signal.Group Copy-on-write (With*) WithLabel(k,v) WithLabels(map) WithOnlyLabels(map) WithoutLabels(...)
component, port, fmesh, collections Mutating (Add*/Set*) AddLabel(k,v) AddLabels(map) SetLabels(map) RemoveLabels(...)

Signals and signal groups are immutable — With* methods return a new value. Components, ports, and meshes are mutable — Add*/Set* methods modify the receiver in place.

Using Labels with Signals

Signals can carry labels to provide metadata about the data they're transporting.

Adding Labels to Signals

// Create signal with labels (copy-on-write)
sig := signal.New("temperature: 25°C").WithLabels(map[string]string{
    "source":   "sensor1",
    "type":     "temperature",
    "priority": "normal",
})

// Or chain WithLabel calls — assign the result!
sig = sig.WithLabel("source", "sensor1").
    WithLabel("type", "temperature").
    WithLabel("priority", "normal")

// Add labels to existing signal
sig = sig.WithLabel("processed", "true").WithLabel("timestamp", "2024-01-01")

Querying Signal Labels

// Check if signal has specific label
if sig.Labels().Has("priority") {
    // Handle labeled signal
}

// Check label value
if sig.Labels().ValueIs("priority", "high") {
    // Handle high priority signal
}

// Get label value
priority := sig.Labels().ValueOrDefault("priority", "normal")

Filtering Signals by Labels

// Filter signals in a port by labels
highPriority := port.Signals().Filter(signal.LabelEquals("priority", "high"))

// Filter signals from specific source
fromSensor1 := port.Signals().Filter(signal.And(
    signal.HasLabel("source"),
    signal.LabelEquals("source", "sensor1"),
))

// Filter signals with any of multiple labels
urgent := port.Signals().Filter(signal.HasAnyLabel("urgent", "critical", "emergency"))

Signal Label Preservation

Labels are preserved when signals flow through pipes, making them reliable for tracking data throughout your mesh.

Using Scalars with Signals

sig := signal.New("measurement").
    WithScalar("weight", 0.85).
    WithScalar("confidence", 0.97)

weight := sig.Scalars().GetOrDefault("weight", 0)

Signal groups support cross-entity scalar aggregation: MinScalar, MaxScalar, AvgScalar, SumScalar.

Using Labels with Components

Components can be labeled to categorize and organize them within the mesh.

Adding Labels to Components

// Create component with labels (mutating API)
c, _ := component.New("processor",
    component.WithInputs("in"),
    component.WithOutputs("out"),
    component.WithActivationFunc(myFunc),
)
c.SetLabels(map[string]string{
    "layer":       "business-logic",
    "category":    "data-processing",
    "criticality": "high",
})

// Or chain AddLabel calls
c.AddLabel("layer", "business-logic").
    AddLabel("category", "data-processing").
    AddLabel("criticality", "high")

// Add labels after creation
c.AddLabel("version", "2.0").AddLabel("owner", "team-a")

Labels can also be set via constructor options:

c, _ := component.New("processor",
    component.WithLabelOption("layer", "business-logic"),
    component.WithInputs("in"),
    component.WithOutputs("out"),
    component.WithActivationFunc(myFunc),
)

Querying Component Labels

if c.Labels().ValueIs("criticality", "high") {
    // Apply special monitoring
}

layer := c.Labels().ValueOrDefault("layer", "unknown")

Filtering Components by Labels

critical := fm.Components().Filter(func(c *component.Component) bool {
    return c.Labels().ValueIs("criticality", "high")
})

businessLogic := fm.Components().Filter(func(c *component.Component) bool {
    return c.Labels().ValueIs("layer", "business-logic")
})

monitored := fm.Components().Filter(func(c *component.Component) bool {
    return c.Labels().HasAll("owner", "environment")
})

Using Labels with Ports

Ports can also be labeled for categorization and filtering.

Adding Labels to Ports

c, _ := component.New("processor",
    component.WithInputs("data", "config", "control"),
    component.WithOutputs("result", "logs"),
    component.WithActivationFunc(myFunc),
)

c.InputByName("data").
    AddLabel("type", "primary").
    AddLabel("required", "true")

c.InputByName("config").
    AddLabel("type", "configuration").
    AddLabel("optional", "true")

c.OutputByName("logs").
    AddLabel("type", "diagnostic")

// Or use SetLabels
c.InputByName("data").SetLabels(map[string]string{
    "type":     "primary",
    "required": "true",
})

Filtering Ports by Labels

required := c.Inputs().Filter(func(p *port.Port) bool {
    return p.Labels().ValueIs("required", "true")
})

diagnostic := c.Outputs().Filter(func(p *port.Port) bool {
    return p.Labels().Has("type") &&
        p.Labels().ValueOrDefault("type", "") == "diagnostic"
})

meta.Labels API

The meta.Labels type provides a rich set of methods for working with labels.

Basic Operations

lc := meta.NewLabels()

// Add single label
lc.Set("status", "active")

// Add multiple labels
lc.SetMany(map[string]string{
    "env":    "production",
    "region": "us-west",
    "tier":   "backend",
})

// Remove labels
lc.Remove("old-status", "deprecated")

// Clear all labels
lc.Clear()

// Get label count
count := lc.Len()

// Check if empty
if lc.IsEmpty() {
    // No labels
}

Querying Labels

if lc.Has("environment") { /* ... */ }
if lc.HasAll("env", "region", "tier") { /* ... */ }
if lc.HasAny("dev", "staging", "prod") { /* ... */ }
if lc.ValueIs("env", "production") { /* ... */ }

value, err := lc.Value("region") // error if not found
region := lc.ValueOrDefault("region", "unknown")

Advanced Operations

filtered := lc.Filter(func(label, value string) bool {
    return strings.HasPrefix(label, "app.")
})

transformed := lc.Map(func(label, value string) (string, string) {
    return strings.ToUpper(label), strings.ToLower(value)
})

lc.ForEach(func(label, value string) error {
    fmt.Printf("%s=%s\n", label, value)
    return nil
})

count := lc.Count(func(label, value string) bool {
    return strings.Contains(value, "prod")
})

allProd := lc.Every(func(label, value string) bool {
    return strings.Contains(value, "prod")
})

anyProd := lc.Any(func(label, value string) bool {
    return strings.Contains(value, "prod")
})

Comparing Label Collections

collection1 := meta.NewLabels().Set("a", "1").Set("b", "2")
collection2 := meta.NewLabels().Set("a", "1")

if collection1.HasAllFrom(collection2) { /* ... */ }
if collection1.HasAnyFrom(collection2) { /* ... */ }

merged := collection1.Merge(collection2) // returns new Labels

// Sorted accessors
keys := lc.Keys()
values := lc.Values() // sorted by key

Batch metadata on groups/collections

Groups and collections support batch metadata operations on their contents:

// Set a label on every signal in a group
tagged := port.Signals().WithLabelOnEach("batch", "2024-01-01")

// Set metadata on the group/collection itself
tagged = tagged.WithLabel("source", "pipeline")

// Set a label on every component in a collection
fm.Components().WithLabelOnEach("processed", "true")

Common Use Cases

Priority-Based Routing

_, _ = component.New("router",
    component.WithInputs("in"),
    component.WithOutputs("high-priority", "low-priority", "normal"),
    component.WithActivationFunc(func(c *component.Component) error {
        _, err := c.InputByName("in").Signals().ForEach(func(sig *signal.Signal) error {
            priority := sig.Labels().ValueOrDefault("priority", "normal")
            switch priority {
            case "high":
                return c.OutputByName("high-priority").PutSignals(sig)
            case "low":
                return c.OutputByName("low-priority").PutSignals(sig)
            default:
                return c.OutputByName("normal").PutSignals(sig)
            }
        })
        return err
    }),
)

Source Tracking

sig := signal.New(data).WithLabels(map[string]string{
    "source":     "api-gateway",
    "request-id": "abc-123",
    "user-id":    "user-456",
})

Debugging and Observability

sig := signal.New(data).WithLabels(map[string]string{
    "trace-id":  generateTraceID(),
    "timestamp": time.Now().Format(time.RFC3339),
    "component": c.Name(),
})

Best Practices

Use Consistent Naming

lc.Set("app.name", "my-mesh")
lc.Set("app.version", "2.0")
lc.Set("app.environment", "production")

Keep Labels Simple

Labels are strings — keep them simple. Use payload for actual data, labels for metadata, scalars for numeric metadata.

Document Label Semantics

Document what labels mean in your mesh so all components agree on semantics.

meta.Scalars API

meta.Scalars is a mutable string → float64 store used on signals (CoW via WithScalar), components, ports, meshes, and groups/collections.

Basic operations

sc := meta.NewScalars()
sc.Set("weight", 0.85).Set("confidence", 0.97)
sc.SetMany(map[string]float64{"latency_ms": 42, "retries": 0})
sc.Get("weight")              // (0.85, true)
sc.GetOrDefault("missing", 0) // 0
sc.Has("weight")              // true
sc.Remove("retries")
sc.Clear()
sc.Len() / sc.IsEmpty()
sc.All()                      // defensive copy
sc.Keys()                     // sorted names

Aggregation

sc.Min()           // (name, value, ok) — smallest entry
sc.Max()           // largest entry
sc.Sum("a", "b")   // sum of named scalars (missing = 0); no args = sum all
sc.Average()       // mean of all scalars
sc.Scale("weight", 2.0)  // multiply in place
sc.Merge(other)    // returns new Scalars (neither input modified)

Query and transform

sc.Every(pred)  // all match (vacuous true on empty)
sc.Any(pred)    // any match
sc.Count(pred)  // count matching
sc.Filter(pred) // returns new Scalars
sc.ForEach(func(name string, value float64) error { ... })

ScalarPredicate is func(name string, value float64) bool.

Signal and port groups additionally expose cross-entity aggregation: SumScalar, MinScalar, MaxScalar, AvgScalar — aggregating a named scalar across all contained signals or ports.

Labels vs Other Mechanisms

Feature Labels Scalars Component State Signal Payload
Purpose String metadata Numeric metadata Persistent component data Actual data transfer
Scope Signal/Component/Port Signal/Component/Port Single component Single signal
Type string → string string → float64 Any type Any type
Persistence Travels with entity Travels with entity Persists across cycles Consumed in cycle
Best for Routing, debugging Weights, scores Counters, state machines Business data

Labels and scalars provide a powerful and flexible way to add metadata throughout your F-Mesh without cluttering your core logic. They're particularly useful when combined with filtering and collection APIs to implement dynamic, metadata-driven behavior.

Add a custom footer
Add a custom sidebar

Clone this wiki locally