Repository navigation
About the $now function and the TOSCA timestamp native type #384
Replies: 17 comments 6 replies
|
Adding a bit to this discussion. I made a quick test on Puccini, I share the result hereafter. I confirm that Puccini accepts the following syntax: Given this, probably the issue I report above is not really a portability issue, rather it is just a matter of presentation of a Honestly, I don't know how Ubicity reports it on stdout. In any case, I would like to hear for your comments about. |
|
Generally speaking, I think single implementations of custom functions in Python should not be included in the core profile because they would be implemented differently for different orchestrators. Right now we are mostly using Ubicity and Puccini, but who knows how other orchestrators would implement it? One implementation would not work for all of them. The TOSCA function signature specifies that the return value is a TOSCA timestamp. The Python function returns a string. This mismatch is already a problem that may be handled differently by implementations. I can't comment on how Ubicity handles this. Is it an error? Or is there an implicit attempt at conversion? (And will it error if the string cannot be parsed into a timestamp?) As for Puccini, I can comment. :) The format you show for the timestamp is internal to Floria, not Puccini per se. It appears when converting to JSON, because ... JSON has no such type. This format will allow you to convert it to whatever you need. Similar JSON conversions exist for scalar and version special types. However, when writing custom functions you don't have to worry about this internal format. If you create custom functions using the Puccini SDK (either Rust or Python) then it would always appear as a native type: a def now() -> datetime:
return datetime.now()Whether Ubicity will accept this, I cannot say. Puccini is currently missing function signature validation. However, it will validate the values for properties, attributes, and typed parameters. If you try to put a string into a timestamp-typed value, it will return an error. There are no implicit conversions by design, because TOSCA is supposed to be strictly typed. That said, Puccini does include a |
|
Thank you @tliron. First, concerning your first claim above (i.e. whether implementing or not custom functions in core profile), there is another discussion that already raised the problem (see here). This issue was also discussed during community meetings (see e.g. Second, the rationale behind my request above was based on the observation that the functions/now.py implementation in the community core profile actually provides a string, which is not what I expected. Third, I think that actually the TOSCA timestamp type is a strange beast. On one hand, the standard states that: "TOSCA timestamps are represented as strings following [RFC 3339]" (sec. 9.1.2.1) On the other hand, "represented as strings" does not mean "subtype of " or "derived from" a string; further, they are both TOSCA native types, but belong to different categories (string --> primitive type; timestamp --> special type). This raises an interoperability question for me. While the specification defines the representation of a TOSCA timestamp as an RFC 3339 string, does it also intend to prescribe that a timestamp value is exposed as a string across implementation boundaries, for example when invoking a custom function? Or should timestamp remain a distinct TOSCA type whose mapping to a language-native value is implementation-specific? I think clarifying this point would be useful. Otherwise, two compliant processors could potentially make different choices at the TOSCA/implementation boundary, which could in turn affect function signatures, result-type validation, and portability of function implementations. This ambiguity is what initially made me question the string returned by now.py. |
|
I hope the custom function implementation files in the community profile are clearly marked as for Ubicity or "for demonstration purposes". Otherwise they will give users the impression that TOSCA has a generally supported FFI, which it does not. I have to say, it's strange to me that we would pick Ubicity as the "community" standard, because it is not free or open source. There is no "download" button users can use to even test these functions. As for timestamps (and other special types), I think you are making it more complicated than it needs to be. The spec only mentions "representation", which I take to mean: In the TOSCA YAML source files, which is the only part of the orchestration implementation that the TOSCA standard can speak to. Even then, specifically for timestamps, we added a note that we will not support the old YAML 1.1 draft, which some parsers choose to support. "Representation" is different from format, structure, etc. For a similar usage of the word see Python's. TOSCA doesn't tell you how values are stored in databases, serialized over the network, etc. TOSCA doesn't even specify how service template inputs come in or outputs come out. It definitely doesn't dictate how arguments and return values are passed to custom function implementations or to interface operations. Actually, TOSCA doesn't even say that users can provide their own custom function implementations. The (I'll just add that timestamps, specifically, are a thorny rabbit hole. RFC 3339 might seem well-defined, but just like the TOSCA spec there are aspects that are open to interpretation. Different parsers might parse the same string differently or represent an internal time instance in a different way. The Gregorian calendar is anything but clear or "settled". This is not just an academic issue as there have been serious and hard-to-detect bugs in software that relies on it. Leap years and even leap seconds can mess up calculations in exciting ways! On that note, even the simple "float" type works differently on different hardware, even on slightly different versions of the same CPU hardware.) |
|
Here are my comments on the exchange above based on my reading of the TOSCA v2.0 specification:
|
|
The useful observation here is that when a profile uses incompatible conventions, there is nothing in the profile that states which convention is used. Roberto's proposal for supporting multiple alternative implementations for functions and operations could potentially address that. |
I don't like that the community profile includes this convention, and I also don't like that it includes Python scripts. It could give readers an impression that this is a best or even common practice of some sort. I also don't see the point: The functions are defined in TOSCA, so that should be enough for any implementor to know what needs to be implemented if they want to support the profile. Why must they adhere to a calling convention if their orchestrator looks nothing like that? And if nobody can run these Python scripts right now, why provide them? What positive purpose does it serve? |
|
These are fair questions:
|
|
The problem with compliance is that must cover so many aspects of the spec. Many orchestrators would not be able to support 100% of it. Consider any of the most popular orchestrators that exist right now. Kubernetes, Ansible, TerraForm, etc. We would very much love for them to have native TOSCA support. But many TOSCA features would not make sense there. There is a feature and often paradigm mismatch. Instead, I imagine them having a built-in profile of sorts exposing the features they already have to TOSCA, with some TOSCA features being left out entirely. Not because they aren't useful, but because they are irrelevant. Would these orchestrators be compliant, then? Partially compliant? This is an open question for which I don't have an answer. To your second point -- imagine if Kubernetes had native TOSCA support as I described above. Could you imagine the community profile's Python artifacts working there? Running arbitrary executables? (Where? In the kubelet?) If the expectation is they build a whole execution subsystem, then it would be reasonable for the Kubernetes community to conclude that TOSCA is not for them. I don't believe that's true and I want that misconception to disappear. Unfortunately the community profile is not helping. It's nice that these artifacts run on one orchestrator, but I strongly object to it being an example of how other orchestrators should implement functions, and strongly recommend against implementing the requirement that must be able to run arbitrary executables in order to support it. Portability, defined in this way, is a race to the bottom. |
|
I would put it the other way around. If a technology implements only those TOSCA features that map exactly onto what it already does, then TOSCA adds only a new dialect without providing additional capabilities. This by itself doesn't give anybody a reason to adopt TOSCA. What makes TOSCA worth adopting are the unique features that are missing from the three technologies you mentioned. The most important TOSCA feature in my opinion is that TOSCA uses a graph of typed nodes and typed relationships as its only structural abstraction. Everything else in the language either adds types to elements of that graph or annotates one. All domain-specific knowledge is provided through types, and the types are themselves defined using the TOSCA language. As a result, an orchestrator does not need new code to support a new application domain, because domain specific knowledge is captured in the profile rather than as code in the orchestrator. That is a main property the three technologies you referred to do not have. HCL for example does not define resource types: resource schemas live in provider binaries and the dependency graph is derived from references rather than defined explicitly. This means that edges don't use a type, can't define properties and don't have any behavior associated with it. Kubernetes can add custom types through CRDs, but it has no relationship abstraction, no derivation between types, and the behavior of a type lives in a controller. In each case the domain knowledge must be defined in compiled code rather than in the model. Once we establish the graph as the main abstraction, the other unique TOSCA features become clear:
Those are the reasons to adopt TOSCA, and I would argue that without support for these feature nobody would. I agree with you that augmenting Kubernetes, Terraform, Ansible, etc. with these unique features requires a place for the TOSCA processor to run. With Kubernetes specifically, that execution environment is likely not Kubernetes itself. However, in my experience most modern uses cases can't be implemented using only Kubernetes, and for some of them Kubernetes is not a good fit. For example, there is a lot of focus these days on edge deployments where deploying Kubernetes on an edge device just so you can deploy a single edge application is much more of a hassle than just deploying that edge application in the first place. And industrial automation use cases may have requirements that cannot be implemented using containerized applications (for example to network interfaces that connect to field devices into specific namespaces). If non-Kubernetes functionality is required anyway, then I would rather provide that functionality through TOSCA profiles rather than requiring custom code in an orchestrator. |
|
I see TOSCA's attraction as being a widely used standard, and a decent one. Yes, it is "just another dialect", but it's one uniquely suitable for the task. I think it's a very big advantage in a world where every product invents its own language and often does it poorly, because as we know it's hard to create a language specification. Graphs exist everywhere in the cloud, so it's not hard to see a fit for TOSCA in many orchestrators. My goal has always been to keep TOSCA open enough for broad suitability. I always fight against it dictating an opinion on behavior. As for features -- None of the features you mention are special. They can all be implemented without TOSCA should the orchestrator want them. In fact some orchestrators do these things much better than TOSCA in some ways. Policies and workflows are both anemic in TOSCA and I think unsuitable for real-world clouds. And you already know my opinion about substitution mapping as a solution to service composition. Of course I agree that Kubernetes is not a good fit for everything. I am, after all, creating an alternative. But it would be great in a hybrid cloud scenario (all large clouds will involve a variety of orchestration tools) to be able to use one language and set of tools for all blueprint designs, from applications to networks to infrastructure. |
|
Hi all, this discussion has touched on several topics of rather different nature, ranging from the representation of TOSCA special types and implementation artifacts to portability, conformance, abstraction, and even the broader value proposition of TOSCA itself. I will try to contribute my perspective on several of them, so apologies in advance for the length of this post. As far as I can tell, my initial observation, which was ultimately about a very specific aspect of the language, opened a Pandora's box and sparked a very engaging debate. Let me say that this is definitely a good thing: different points of view generally lead to better solutions. About the specific aspect related to the TOSCA Of course, each processor/orchestrator is free to use whatever structure it prefers internally, provided that there is a common representation, defined by the standard, for values expressed in TOSCA YAML files. Somewhere in between, there is necessarily a boundary at which conversion between the two representations takes place. What initially caught my attention is that two different implementations of the same function appear to place that boundary in two different places: the community Let me be clear on this: I'm not arguing that either of these two choices is inherently better than the other. My initial concern, which I probably wasn't able to express clearly because it wasn't completely clear to me either, was mainly that service-template portability across orchestrators might be affected by the precise location of the boundary mentioned above. I now understand that this isn't really the case, at least as far as service-template portability itself is concerned, and that I was probably conflating the portability of the TOSCA model with the portability of its implementation artifacts. However, the discussion raised some broader issues, mainly related to the handling of custom functions and, more generally, artifacts. These topics come up repeatedly in several discussions and concern that gray area between the standard language and its implementation. A few observations about this:
There is another aspect of profile portability that I think deserves consideration: portability can also reduce platform lock-in. Even if perfect portability of executable artifacts is difficult to achieve, making profiles usable across multiple processors, or at least making their adaptation reasonably straightforward, has considerable practical value. It lowers the cost of replacing the TOSCA processor or orchestration platform without having to redesign the entire modeling layer around the new implementation. From a customer's point of view, I consider that a significant advantage. A technology that does not unnecessarily lock its models and domain knowledge into a specific implementation provides considerably more freedom in the long term. I have been involved a couple of times in projects where an existing platform had to be replaced, and in practice these migrations can become extremely painful. The cost is rarely limited to replacing the platform itself: models, integrations, scripts, operational procedures, and embedded domain knowledge may all have accumulated implementation-specific assumptions over time. The more of that knowledge can remain in portable declarative models and profiles, the smaller the portion that needs to be reconsidered when the underlying platform changes. So, even if "write once, run on every orchestrator" may be an unrealistic objective for implementation artifacts, I would still consider portability a direction worth pursuing. There is a large spectrum between perfect binary portability and complete platform lock-in, and moving toward the former has tangible value. About abstraction and the related topics, such as the layered-model-continuum approach and substitution mapping, I would also like to share my perspective.
A couple of years ago I was involved in a project concerning an IoT platform operated by a large Italian telco. It managed approximately 25,000 remote gateways connected to sensors distributed across the country. We had equipment from five different vendors, seven different hardware configurations, and each gateway could have a variable number of sensors, belonging to roughly ten different types, connected in different combinations to the available ports. The result was hundreds of possible configurations. Those gateways had to be replaced quite frequently, often with a gateway from a different vendor depending on what happened to be available in the warehouse. Gateway setup, including the sequence of operations required for first boot and configuration, was largely manual and extremely difficult to manage. An orchestrator able to represent each gateway as an abstract node that can be substituted by the appropriate topology depending on vendor, hardware configuration, attached sensors, and other constraints could help enormously in handling that complexity. The higher-level model could remain stable while the concrete realization behind that abstraction changes. Similarly, I find the idea of an abstract template acting as a "placeholder" in a system, whose concrete realization can be changed by selecting a different substituting topology, quite compelling. Of course, nobody is likely to replace the But consider a real service initially deployed on Kubernetes in a central cloud. At some point it may need to move to the edge, for example because of latency requirements. The edge environment may have limited resources and support only lightweight execution technologies such as With a sufficiently abstract model, moving that service from the core to the edge while also changing its underlying implementation technology can be expressed by changing the realization behind the abstract node rather than redesigning the higher-level topology around the technology selected at each location. To me, this leads to one of TOSCA's most interesting properties when compared with more technology-specific IaC approaches: its potential ability to model and orchestrate heterogeneous environments through a common abstraction layer. A complex system may span very different technological domains: VMs, Kubernetes clusters, lightweight containers, edge devices, network functions, physical devices, and potentially other execution environments. Rather than requiring the top-level service model to adopt the abstractions of one specific underlying technology, TOSCA can act as an overlay model describing the system in terms of typed nodes, relationships, requirements, capabilities, and policies, while profiles provide the domain-specific semantics beneath that common model. The analogy is obviously not perfect, but I tend to think of this in a way similar to IP in networking. IP provides a common layer across heterogeneous lower-layer technologies without requiring them to share the same MAC-level mechanisms. Likewise, TOSCA can potentially provide a common modeling layer across heterogeneous execution and orchestration domains without pretending that those domains are internally identical. For me, this is one of TOSCA's strongest potential advantages. The goal should not necessarily be to replace Kubernetes, Terraform, Ansible, or any other technology-specific orchestrator. It can instead provide a higher-level model capable of describing a system that spans several of them, while delegating domain-specific realization to the appropriate profiles and execution environments. On this specific point, I find myself very close to Chris's view. To me, one of TOSCA's most distinctive properties is precisely the possibility of having a general-purpose orchestration engine operate across very different technology domains, while concentrating most of the domain-specific heavy lifting in profiles and in the artifacts or plugins implementing their semantics. In other words, I see considerable value in keeping the orchestration core as generic as possible. The processor should understand TOSCA concepts and manipulate a TOSCA topology, while knowledge about how to provision, configure, operate, or remove a particular technology should, as much as reasonably possible, live outside that generic core. Interestingly, at least from my perspective, Tal's own ecosystem can also be seen as broadly consistent with this separation of concerns, even though Tal and Chris clearly differ on where some responsibilities should lie and on what portability should imply. Puccini provides the TOSCA processing/compiler layer, Floria deals with the executable graph model, and technology-specific plugins or integrations can take responsibility for interacting with the actual target environments. What I find attractive in that architectural principle is that the generic layers should primarily need to evolve when the language or their generic execution semantics evolve, rather than every time a new technology domain needs to be supported. The domain-specific complexity can instead be pushed toward extensible components. This is, in my opinion, more than an implementation detail. It is one of the architectural properties that can allow TOSCA to sit above heterogeneous technologies rather than becoming yet another technology-specific IaC language. Finally, a few thoughts about compliance and standards. Standards come in very different forms, ranging from highly formalized specifications with explicit conformance processes to de facto standards whose adoption is almost entirely market-driven. In telecommunications, for example, there are standardization environments where conformance is verified through formal test specifications and well-defined certification processes. At the opposite end of the spectrum, a product or service API may become a de facto standard simply because it is widely adopted, without any formal certification mechanism. An approach that I personally find interesting as a middle ground is the one used for TM Forum Open APIs: along with the API specifications, TM Forum defines conformance requirements and provides tooling to assess implementations against them. The important principle, in my opinion, is the separation between specifying the language or API and defining a concrete, testable set of conformance expectations. From this perspective, I think the work already being done by Paul Jordan around the TOSCA test suite and conformance clauses is an excellent starting point. It could eventually provide the basis for a more structured notion of conformance without necessarily requiring a binary "fully compliant / non-compliant" classification. This last point seems particularly important considering Tal's observation about technologies such as Kubernetes, Terraform, and Ansible. Requiring every processor to implement every TOSCA feature before it can meaningfully claim TOSCA support may be unrealistic. A capability- or clause-based conformance model could perhaps provide a more useful vocabulary: an implementation could state precisely which portions of the specification it supports, and users or profiles could state which capabilities they require. That would make the notion of interoperability more concrete and testable while still leaving room for very different kinds of TOSCA processors. |
|
There's a lot here, so I'll touch on just a few points. "Reduce the cost of switching implementations" is indeed a goal. The question is how much reduction is possible before the inherent costs of your abstractions are higher than the cost of switching. And who is paying those costs? In fact, answering that second question is critical, because it's usually the end user who pays, and they have the misfortune of usually not being part of the decision process. This is a pervasive problem in software engineering. Compare, for example, with developing a mobile app that must work on both Android and iOS. There are many different approaches, from: 1) having two completely separate codebases, to 2) separating the backend (shared code between the two) from the UI (separate codebases), to 3) using a UI library that can work on both, such as an HTML-based UI. Option 3 is the worst. Users get an experience that is less-than-native, that requires more compute power and memory, reduces battery life, and might even lead to a costly phone upgrade. It hurts our planet. Engineers might be unhappy, too. The cross-platform UI libraries are lowest-common-denominator monstrosities. They make hard things easy but make easy things hard. For example, everything is great until you must support a native feature on iOS. It would take a few lines of code in the native iOS SDK, but with this abstraction you have to punch through layers of HTML and web browser widgets and maybe do unsupported things that would break in the next release. The best option is somewhere between 2 and 3, a bit closer to 2. Yes, it's very subtle! You must develop the UI natively to each platform. I will die on this hill. It is a red line you should not cross because beyond it is the Land of Self-Inflicted and Unnecessary Costs for Everyone. But perhaps there are some aspects of the UI that can be shared between Android and iOS? UI data and state validation, for example. It's not easy to find the right balance. That's where approaches like MVC, MVP, etc., can help, but even then the boundaries are hard to get right, and you will likely need to continuously refactor them. Here general rules of thumb won't work and you need to consider your project and team's capabilities. The best option might end up being just 2, period. I think you guys are dreaming of an option 3 for cloud. In the place of HTML you would put TOSCA. And in the place of a web browser widget you would put a library of scripts. Substitution mapping fulfills the role of an "MVC-like". It's all just as bad as option 3 for mobile apps, perhaps worse, because we don't actually have a dependable web browser equivalent. The closest thing might actually be Ansible, but we would probably not want to standardize on Ansible playbooks, even if they work well. Roberto, I think you are close to my thinking here in separating the "core" functions from actual cloud interactions, and your analogy to CPU platforms is useful. Could that be our balancing point between option 2 and option 3? Just allow for portable core functions while keeping cloud integration out of scope for TOSCA? Unfortunately, I don't think so. A function like I would put the balancing point a bit closer to option 2. Going back to the CPU analogy, let's consider C. C is a great solution because you can compile it to any CPU architecture. But what about different operating systems? If you want the same library to work on Linux and Windows, it's not just a CPU story. You need different C code for each. But C can help with this, too, because it allows us to separate our code between definitions in I see TOSCA in the role of header files: Define what the I understand that this falls short of your option 3 dream. But I think it sits squarely in option 2 territory, and is still immensely beneficial. It's not "write once, run everywhere", but it is "write one, do some integration work, and then run everywhere", which is still a huge step forward that I think it strikes exactly the right balance between the practical and the ideal. But even this approach has limitations and some big challenges. Back to C: Linux and Windows don't just require different So, this is why I get very concerned about TOSCA dictating opinions on policies, workflows, event models, etc., and why I'm concerned about what "conformance" could mean. I want to keep TOSCA a language, like C, not a library, like libc. TOSCA can't address all our challenges and I'll insist that it shouldn't. It still does a lot for us. |
|
Thank you @tliron for this clarification. I think there may be a small misunderstanding here, because I actually believe our positions on this specific point are much closer than they might appear. I am definitely not proposing that all TOSCA processors should support Python integration, or any other common runtime or calling convention. Actually, I think your What I would like to make portable is primarily the contract, not necessarily its implementation. Different processors may provide completely different implementations of the same function, using Python, WASM, native code, built-in functionality, or anything else appropriate to their architecture. This is precisely why I mentioned separating the function signature from its implementation. Ideally, a profile could define something analogous to the So I am not advocating "write once, run everywhere" for artifacts. Actually, your formulation, "write one, do some integration work, and then run everywhere", is very close to what I have in mind. What I would like to avoid is having to rewrite not only the integration layer, which I fully expect may need to be processor-specific, but also the higher-level models and contracts simply because the underlying processor changes. In other words, I see the integration work as the processor-specific Seen this way, I think we are actually drawing the boundary in a very similar place. Where we may still differ is perhaps on how much can usefully be placed on the portable side of that boundary, and how far TOSCA profiles should go in defining reusable behavior. But I fully agree that portability should not be achieved by forcing every processor into a common execution architecture. |
|
Roberto, I can support the goal as you stated it, but it's not what we currently have. As it stands, the contract requires 1) support for bash and Python, and 2) that the orchestrator must be able to consume independent script files (as artifacts), and 3) the use of environment variables. (The link seems to be to a proposal from Chris, I'm not sure if it's been accepted.) All of this is very extreme. Independent environment variables means forking. Running bash and Python means a containerized runtime environment for these forked processes. Even Ansible, which is eminently extensible via Python, doesn't support or allow for this. And I would argue strongly that it never should. I'm sorry, but nothing in this conversation has convinced me that I'm wrong. You are trying to make things easier for users by including a few ready-to-use core functions, such as We don't need an additional "contract" for the community profiles. TOSCA already defines a contract: the I'll repeat: I strongly recommend that the community profile does not include implementation artifacts for functions and does not specify a function calling mechanism of any kind. What you are doing right now is a promotion for bad practices that should not be implemented by TOSCA orchestrators, or any other orchestrator for that matter. If you really want to include implementations, then I recommend including specific ones. Add artifacts for Ubicity and a plugin for Puccini. The current "runs everywhere" solution is detrimental to TOSCA. |
|
Hi Tal, I think we actually agree on the fundamental point, and your Let me clarify what I have in mind in a few points:
So I think our positions are much closer than they initially appeared. I am not aiming for "write once, run everywhere" at the artifact level. Your formulation, "write one, do some integration work, and then run everywhere", is actually a very good description of what I would like to achieve. I would also be very interested in your view on Discussion #365, precisely because I think your |
|
Roberto, there is a disconnect between what you are saying here and what is actually in the community profile. The community profile includes implementations, not just definitions. And it includes an artifact calling convention (this link is in the actual published profiles; it's not a "proposal"). I am strongly advocating for removing both. (Remember, this issue was opened about the The community profile should showcase best practices for publishing profiles. I think right now it teaches the wrong lesson. |
|
@tliron, your last comment, and in particular your suggestion to provide specific implementations for different processors, made me reconsider the issue from a slightly different perspective. Perhaps part of the problem is not whether the community repository should contain implementations at all, but rather how profiles and implementations are organized and presented, so that their portability scope is immediately clear. Today, very roughly, we have something like: The The situation is different for What if we made these distinctions explicit in the organization of the community repository? Conceptually, we could distinguish between:
I am not suggesting a specific directory layout yet. The important point would be that a user looking at the repository could immediately understand what is intended to be portable and what explicitly targets a particular execution environment. This would also seem consistent with your suggestion to provide, for example, an Ubicity-specific artifact and a Puccini/Floria-specific plugin instead of presenting one artifact convention as universally applicable. Personally, I still see value in keeping working implementations and examples in the community repository. The TOSCA specification explicitly allows profiles to contain implementation artifacts, and practical, executable examples can significantly lower the barrier to adopting TOSCA. But I agree that they should not give the impression that a specific execution mechanism represents the TOSCA way of implementing functions or operations. So perhaps a useful community best practice could be not "community profiles must never contain implementations", but rather "portable definitions and processor-specific implementations must be clearly separated and identified as such." Would that address your main concern about what lesson the community repository should teach? |
|
What value do you see in providing Python implementations that don't work with any orchestrator? Moreover, ones that prescribe an execution mechanism that no orchestrator should ever use? I think we might have a different issue here, in that TOSCA function definitions don't work exactly like C header files. When you define a TOSCA function, you must specify the The good thing is that it's easy to solve the import problem. Because we are importing by profile names, the orchestrator can register a different TOSCA file for the same profile name depending on which implementation it wants to use. |
|
A couple of comments and corrections:
I do agree with you that we want the orchestrator to decide which implementations it has. That is also why I think Roberto's separation principle is right, but only if it can be supported using the proper grammar. Roberto articulates the right goal when he says: "Portable definitions and processor-specific implementations must be clearly separated and identified as such". However, without proper grammar, connecting the two still requires manual copying or duplicate versions, both of which are undesirable. |
|
This discussion has covered a lot of ground as Roberto has pointed out. I'll try to get back to the core of what needs to be decided:
I do not think any of us would disagree that this second contract exists, even if it is only implied in code somewhere rather than explicitly defined in a document. The real question then is whether such contracts belong in the community profiles. As Tal points out, when a community profile includes an artifact definition it creates an expectation that "every orchestrator can execute this implementation artifact". However, some orchestrators may be unable or unwilling to do so for a variety of reasons. Tal argues this from a cost to the
The right answer is one where users don't incur cost at all: orchestrators should not have to depend on deploying execution environments that are not native to the orchestrator, and profiles should work out-of-the-box without requiring orchestrator-specific software development work. This argues for the following approach:
The second part is the hard part. One could obviously hard-code implementations for all functions and operations defined in the community profiles, but that isn't a very scalable approach. In the Ubicity approach, implementations are always bundled with the Unfortunately, TOSCA currently doesn't provide a mechanism for dynamically attaching implementations to types defined somewhereelse. I guess one could use derivation, but that defines new types rather than augmenting the existing types. This approach could also Functions are a partial exception. §10.4 already defines refinement of a function definition in which only the implementation may be changed, with signatures matched positionally. Unfortunately, the rule is written for a service template refining a function by name. It does In my opinion, then, the right approach is to explore new grammar that supports attaching implementations dynamically to the types already defined in a profile, without deriving anything. This is by the way how the Model Continuum methodology describes translating the Device View to the Instance View, where the instance view is precisely the level that "adds interface implementations based on implementation artifacts that can be used by an Orchestrator". It includes the line: "Unfortunately, the TOSCA language currently does not have any constructs to support such dynamic behavior." The core idea is that implementations should sit at the lowest layer in the Model Continuum hierarchy, not the highest layer. This is not a new idea. Miles Stötzner and colleagues at Stuttgart have published on exactly this problem: Managing the Variability of I'm not proposing we should adopt their mechanism, just that we investigate something similar. What I have in mind is something that I |
|
I did not say that TOSCA is the only contract needed, just that it is the only contract that should be published in community profiles. Execution should be out of scope for a publicly available profile that purports to be usable with all TOSCA orchestrators. See my previous post -- I suggest something similar to you, that there should be a way to inject implementations. I don't see why it needs to be substitution. It could just be a reference, the same way we use relationship templates. Here's a very simple description of the solution: For the Again, see my post above -- if that imported file uses a profile name then you can control which implementation file gets imported without having to modify the source code, because the profile mappings are managed by the orchestrator. Just import "core-implementation:0.1" or similar, and it would import the right ones. Example: tosca_definitions_version: tosca_2_0
imports:
- profile: implementations:0.1
namespace: imp
functions:
hello:
signatures:
- arguments:
- type: string
result:
type: map
key_schema: string
entry_schema: string
implementation: imp:functions # name instead of definitionAnd then the profile "implementations:0.1" can be mapped to this file for a Puccini implementation: tosca_definitions_version: tosca_2_0
imports:
- profile: puccini
namespace: puccini
implementations:
functions:
type: puccini:WasmPlugin
file: artifacts/wasm/functions.wasmI'll return to a feature I've proposed many times: TOSCA needs an "implementation type" instead of reusing artifact types for implementations. While both Puccini and Ubicity currently support implementations stored in files, this is a requirement that might not always be relevant to all execution environments. For example, I can imagine an orchestrator based on an interpreted language (Python, etc.) that specifies an implementation by the name of the library, namespace, or similar. Or it can be a networked API call to some kind of "execution service" that manages just executions. And even if you are using files, perhaps you would want several implementation definitions to refer to the same artifact name, instead of copy-pasting the same path between all implementation definitions. A new type can offer a higher level of reference. Different implementation definitions can refer to the same artifact. (On that note, we also need better artifact management, as we've discussed in the past. Having to attach artifacts to nodes or functions is weird.) EDIT:
|
Uh oh!
There was an error while loading. Please reload this page.
I’m opening a new discussion because I’ve been going around in circles with an issue and something still doesn’t feel right.
Let's consider the following custom definition, taken from the community core profile:
The signature states that the function returns a TOSCA
timestamp, which is a built-in TOSCA type. However, the implementation provided in repository (functions/now.py) is the following:It returns a string, not a timestamp. As a matter of fact, the following property:
is evaluated as follows according to this definition (I guess this is what Ubicity provides):
2026-10-12T12:53:44.374402076ZIn other words, it manages the timestamp as a string, even if in TOSCA, the built-in timestamp is not derived from string.
On the other hand, Puccini implements this function natively in its built-in Puccini profiles. However, the same property above, evaluated on Puccini using the native
$nowimplementation provides:which is quite different.
In my opinion, this raises some portability issues. Suppose that I need to define a timestamp that represents the Unix epoch. According to the discussion above, I expect that Ubicity accepts the following syntax:
while Puccini accepts a different layout:
Further, I suspect that the one that works on either orchestrator does not work on the other.
Am I right? @lauwers, @tliron could you please provide your opinion about?
Thank you
Roberto
All reactions