From bb9a1fc46d616a8297d046b82407c4759ae28b9d Mon Sep 17 00:00:00 2001 From: Ben Gribaudo Date: Mon, 22 May 2023 14:17:37 -0400 Subject: [PATCH 1/4] Expounding on streaming --- query-languages/m/m-spec-basic-concepts.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/query-languages/m/m-spec-basic-concepts.md b/query-languages/m/m-spec-basic-concepts.md index 2b24d72c6..0c8525dd9 100644 --- a/query-languages/m/m-spec-basic-concepts.md +++ b/query-languages/m/m-spec-basic-concepts.md @@ -2,7 +2,7 @@ title: M Language basic concepts description: Discusses basic concepts that appear throughout the subsequent sections ms.topic: conceptual -ms.date: 8/2/2022 +ms.date: 5/22/2023 ms.custom: intro-internal --- @@ -228,7 +228,7 @@ It does, however, rely on being able to reorder computations. Since expressions Once a value has been calculated, it is _immutable_, meaning it can no longer be changed. This simplifies the model for evaluating an expression and makes it easier to reason about the result since it is not possible to change a value once it has been used to evaluate a subsequent part of the expression. For instance, a record field is only computed when needed. However, once computed, it remains fixed for the lifetime of the record. Even if the attempt to compute the field raised an error, that same error will be raised again on every attempt to access that record field. -An important exception to the immutable-once-calculated rule applies to list and table values. Both have _streaming semantics_. That is, repeated enumeration of the items in a list or the rows in a table can produce varying results. Streaming semantics enables the construction of M expressions that transform data sets that would not fit in memory at once. +An important exception to the immutable-once-calculated rule applies to list, table and binary values, each of which has _streaming semantics_. M's representation of a list, table or binary value, while immutable, does not contain materialized list items, table rows or binary bytes (respectively). Instead, these are produced as needed each time the value is enumerated. Since a streamed value's items, rows or bytes are produced on demand, repeated enumerations of the value can produce varying results unless the source and all intermediate transforms are guaranteed to be deterministic. Streaming semantics enables the construction of M expressions that transform data sets that would not fit in memory at once. Also, note that function application is _not_ the same as value construction. Library functions may expose external state (such as the current time or the results of a query against a database that evolves over time), rendering them _non-deterministic_. While functions defined in M will not, as such, expose any such non-deterministic behavior, they can if they are defined to invoke other functions that are non-deterministic. From bfe44d694411e8a4522769b9c0906fa64baee9eb Mon Sep 17 00:00:00 2001 From: Ben Gribaudo Date: Fri, 14 Jul 2023 14:01:25 -0400 Subject: [PATCH 2/4] Update m-spec-basic-concepts.md --- query-languages/m/m-spec-basic-concepts.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/query-languages/m/m-spec-basic-concepts.md b/query-languages/m/m-spec-basic-concepts.md index 0c8525dd9..0a16edf12 100644 --- a/query-languages/m/m-spec-basic-concepts.md +++ b/query-languages/m/m-spec-basic-concepts.md @@ -2,7 +2,7 @@ title: M Language basic concepts description: Discusses basic concepts that appear throughout the subsequent sections ms.topic: conceptual -ms.date: 5/22/2023 +ms.date: 7/14/2023 ms.custom: intro-internal --- @@ -228,7 +228,7 @@ It does, however, rely on being able to reorder computations. Since expressions Once a value has been calculated, it is _immutable_, meaning it can no longer be changed. This simplifies the model for evaluating an expression and makes it easier to reason about the result since it is not possible to change a value once it has been used to evaluate a subsequent part of the expression. For instance, a record field is only computed when needed. However, once computed, it remains fixed for the lifetime of the record. Even if the attempt to compute the field raised an error, that same error will be raised again on every attempt to access that record field. -An important exception to the immutable-once-calculated rule applies to list, table and binary values, each of which has _streaming semantics_. M's representation of a list, table or binary value, while immutable, does not contain materialized list items, table rows or binary bytes (respectively). Instead, these are produced as needed each time the value is enumerated. Since a streamed value's items, rows or bytes are produced on demand, repeated enumerations of the value can produce varying results unless the source and all intermediate transforms are guaranteed to be deterministic. Streaming semantics enables the construction of M expressions that transform data sets that would not fit in memory at once. +An important exception to the immutable-once-calculated rule applies to list, table, and binary values, which have _streaming semantics_. Streaming semantics allow M to transform data sets that do not fit into memory all at once. With streaming, the values returned when enumerating a given table, list, or binary value are produced as needed each time they're requested. Since the enumerated values are not fixed and immutable but instead are produced dynamically on demand, they can be different across multiple enumerations. This does not mean that multiple enumerations will always result in different values, just that they can be different if the data source or M logic being used is non-deterministic. Also, note that function application is _not_ the same as value construction. Library functions may expose external state (such as the current time or the results of a query against a database that evolves over time), rendering them _non-deterministic_. While functions defined in M will not, as such, expose any such non-deterministic behavior, they can if they are defined to invoke other functions that are non-deterministic. From c272bac427f7d39862038f6b21e1b6f47219257a Mon Sep 17 00:00:00 2001 From: Ben Gribaudo Date: Mon, 31 Jul 2023 12:30:48 -0400 Subject: [PATCH 3/4] Tweaks --- query-languages/m/m-spec-basic-concepts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/query-languages/m/m-spec-basic-concepts.md b/query-languages/m/m-spec-basic-concepts.md index 0a16edf12..1c178bdc8 100644 --- a/query-languages/m/m-spec-basic-concepts.md +++ b/query-languages/m/m-spec-basic-concepts.md @@ -228,7 +228,7 @@ It does, however, rely on being able to reorder computations. Since expressions Once a value has been calculated, it is _immutable_, meaning it can no longer be changed. This simplifies the model for evaluating an expression and makes it easier to reason about the result since it is not possible to change a value once it has been used to evaluate a subsequent part of the expression. For instance, a record field is only computed when needed. However, once computed, it remains fixed for the lifetime of the record. Even if the attempt to compute the field raised an error, that same error will be raised again on every attempt to access that record field. -An important exception to the immutable-once-calculated rule applies to list, table, and binary values, which have _streaming semantics_. Streaming semantics allow M to transform data sets that do not fit into memory all at once. With streaming, the values returned when enumerating a given table, list, or binary value are produced as needed each time they're requested. Since the enumerated values are not fixed and immutable but instead are produced dynamically on demand, they can be different across multiple enumerations. This does not mean that multiple enumerations will always result in different values, just that they can be different if the data source or M logic being used is non-deterministic. +An important exception to the immutable-once-calculated rule applies to list, table, and binary values, which have _streaming semantics_. Streaming semantics allow M to transform data sets that do not fit into memory all at once. With streaming, the values returned when enumerating a given table, list, or binary value are produced on demand each time they're requested. Since the expressions defining the enumerated values are evaluated each time they're enumerated, the output they produce can be different across multiple enumerations. This does not mean that multiple enumerations will always result in different values, just that they can be different if the data source or M logic being used is non-deterministic. Also, note that function application is _not_ the same as value construction. Library functions may expose external state (such as the current time or the results of a query against a database that evolves over time), rendering them _non-deterministic_. While functions defined in M will not, as such, expose any such non-deterministic behavior, they can if they are defined to invoke other functions that are non-deterministic. From 1a0ce6422ac1470fd6b851ff0036e330c9cc08aa Mon Sep 17 00:00:00 2001 From: Doug Klopfenstein Date: Wed, 9 Aug 2023 11:27:28 -0700 Subject: [PATCH 4/4] Minor changes for MS style --- query-languages/m/m-spec-basic-concepts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/query-languages/m/m-spec-basic-concepts.md b/query-languages/m/m-spec-basic-concepts.md index 1c178bdc8..ab858b4c8 100644 --- a/query-languages/m/m-spec-basic-concepts.md +++ b/query-languages/m/m-spec-basic-concepts.md @@ -228,7 +228,7 @@ It does, however, rely on being able to reorder computations. Since expressions Once a value has been calculated, it is _immutable_, meaning it can no longer be changed. This simplifies the model for evaluating an expression and makes it easier to reason about the result since it is not possible to change a value once it has been used to evaluate a subsequent part of the expression. For instance, a record field is only computed when needed. However, once computed, it remains fixed for the lifetime of the record. Even if the attempt to compute the field raised an error, that same error will be raised again on every attempt to access that record field. -An important exception to the immutable-once-calculated rule applies to list, table, and binary values, which have _streaming semantics_. Streaming semantics allow M to transform data sets that do not fit into memory all at once. With streaming, the values returned when enumerating a given table, list, or binary value are produced on demand each time they're requested. Since the expressions defining the enumerated values are evaluated each time they're enumerated, the output they produce can be different across multiple enumerations. This does not mean that multiple enumerations will always result in different values, just that they can be different if the data source or M logic being used is non-deterministic. +An important exception to the immutable-once-calculated rule applies to list, table, and binary values, which have _streaming semantics_. Streaming semantics allow M to transform data sets that don't fit into memory all at once. With streaming, the values returned when enumerating a given table, list, or binary value are produced on demand each time they're requested. Since the expressions defining the enumerated values are evaluated each time they're enumerated, the output they produce can be different across multiple enumerations. This doesn't mean that multiple enumerations always results in different values, just that they can be different if the data source or M logic being used is non-deterministic. Also, note that function application is _not_ the same as value construction. Library functions may expose external state (such as the current time or the results of a query against a database that evolves over time), rendering them _non-deterministic_. While functions defined in M will not, as such, expose any such non-deterministic behavior, they can if they are defined to invoke other functions that are non-deterministic.