Community support/help chanel?

[al-darmonski]

I'm glad to find out that RD is open source. But there is almost no documentation. Can you give your advice on what is the best way to get some more info and/or help?
Is there some sort of community support to encourage adoption of RD? Is there a means like Slack channel, forum etc?

I want to evaluate JetBrains RD as a potential basis for integrating JetBrains MPS with our in-house created .NET modelling environment. Our use case seems quite similar to Rider, the context n which RD was created.

[ForNeVeR]

@al-darmonski, hello! We're glad you found the project interesting.

We have a community Slack channel for .NET IDE plugin writers. Please drop me a email to [REDACTED]@jetbrains.com, and I'll send you an invite.

Also, we're working on an improved documentation; thanks for raising this issue.

[al-darmonski]

Hi @ForNeVeR ,

Thanks for your reply. I sent you an email.

Do you have expectations about when the improved documentation will be available?

I have some technical questions, that hopefully you or someone else from the RD team can answer:

• Can we use RD in one server and multiple clients setup? If yes, can you provide an example?
• Does the RD modelling DSL (Kotlin based) supports references? We want to implement hierarchy (tree) where some tree nodes will refer to another node in the same or another tree. Can we set the RdId’s ourselves or is it meant to be internally set and used?
• How does the conflict resolution work? What is the algorithm behind it?
• Does RD allows for security measures (authentication and authorization) in case client and server run on different machines over the network?

[ForNeVeR]

Hello!

First, I have sent the invites to the addresses you've sent me by email. Now, about the questions.

Do you have expectations about when the improved documentation will be available?

I think maybe a week or two. Though I'm not sure the documentation will be very useful from the beginning: right now, I'm listing the main concepts and adding some simple examples for the main Rd features.

Can we use RD in one server and multiple clients setup? If yes, can you provide an example?

Not that I'm aware of. Usually, a separate Rd session is established for every client-server pair.

Does the RD modelling DSL (Kotlin based) supports references? We want to implement hierarchy (tree) where some tree nodes will refer to another node in the same or another tree. Can we set the RdId’s ourselves or is it meant to be internally set and used?

You better not use RdId for your purposes. The usual scheme is to add a map or a list, and then use the element indices from these collections as item identifiers in further structures, where you would usually like to use a reference.

How does the conflict resolution work? What is the algorithm behind it?

For certain structures (say, RdProperty) there's IsMaster property, and the "master" version is supposed to "win" in concurrent modification scenarios.

Though I should mention that the usual way to design a protocol for most purposes is to not rely on concurrent modifications at all. If there's only one side that has rights to modify a property at any moment of time (and there's a proper synchronization between the sides of the protocol), it will be much easier to reason about the whole interaction. In some cases, this is impossible, and more complex entities like RdTextBuffer are introduced.

Does RD allows for security measures (authentication and authorization) in case client and server run on different machines over the network?

Currently, Rd offers no embedded security measures. There're internal cases when this is necessary. For such cases, we could recommend implementing your own Wire.

P.S. As a side note (not directly related to any of the above), I'v found this excellent talk from @maartenba, which includes a more or less detailed explanation of some protocol entities and conventions: https://youtu.be/ujcp2LiPds8.

[al-darmonski]

Hi @ForNeVeR ,
Your answers are quite useful. Thanks!

I was wondering of you can provide some explanation about the RD primitives (or concepts is the better term) like classdef, structdef, agregatedef but also property vs field, signal vs call/callback. It's also useful to have the full list of primitives.

I've got some intuitive understanding and some insight from the generated codes but it will be useful to have the intended meaning and maybe some examples demonstrating use cases. For instance field it is not synchronized (bound), what's it useful for?

[ForNeVeR]

• classdef vs structdef should be more or less intuitive for a .NET developer: a classdef may own properties and other reactive values (and be bound to the procotol), while a structdef is just a collection of values without any behavior.

• property vs field is exactly what you've written. Fields are useful for structs, rarely for classes I'd say (probably for readonly constructor props though?)

• signal is a "fire and forget"-sort of thing, while a call allows to return a result to the caller.

• For signals and calls, there's a difference between symmetric and asymmetric models. When modeling a signal or a call, there's often a difference between the called and callee side. For such cases, you may choose to generate two different sets of source files: on one side, signal may only be fired, and on the other one, it may only be subscribed to. To discriminate these cases, use signal (only fire) / sink (only subscribe), or call (only call) / callback (only subscribe / be called).

  "Asymmetric" generation is an opt-in feature; by default, I think the symmetric model will be generated (where both sides are free to both call and subscribe to a signal or a call, and you control the correctness yourself).

Hope I answered your questions. Feel free to ask more!

[al-darmonski]

Hi @ForNeVeR ,
Yes, you have answered my questions. Thank you for your support.
I have another question: I want to specify an interface with a method that has a nullable return type.

var IMyInterface = interfacedef("IMyInterface") {
     method("get", PredefinedType.string.nullable)
}

But it generates to:

interface IMyInterface
{
    fun get() : StringNullable
}

I expected fun get() : String?
Is what I expect possible? How can I do that?

[al-darmonski]

Another question: what's the intended use case of interfacedef?
I've seen what it generates to in Kotlin and C#. If the class C1 implementing the interface I1 is not abstract (classdef or openclass), the interface methods are overridden and throw UnsupportedOperationException. That gives the opportunity to create a non generated derived class DC1 that can override the methods in a meaningful way.
Consider this scenario:

1. DC1 object is created on server side
2. DC1 object is handed over to RD model, it will be cast to C1 and will be transferred to client as C1 object
3. At the client side we can get notification that C1 is added. But we're not able to call the DC1 methods.

If the class C1 is abstract (baseclass) than the I1 methods are abstract. We still have opportunity to create derived class DC1 overriding the I1 methods. But RD model doesn't know about them and also can't transfer DC1 objects since such can't be created (DC1 is abstract).

So it seems I'm missing the point of having interfacedef. Please enlighten me.

[ForNeVeR]

@al-darmonski

I want to specify an interface with a method that has a nullable return type.

var IMyInterface = interfacedef("IMyInterface") {
     method("get", PredefinedType.string.nullable)
}

But it generates to:

interface IMyInterface
{
    fun get() : StringNullable
}

I expected fun get() : String?
Is what I expect possible? How can I do that?

I believe this is a bug. Extracted to #232.

Another question: what's the intended use case of interfacedef?

In our code, we only use it for marker interfaces with no methods at all, so it's hard to answer this question properly now. I'll ask my colleagues.

[al-darmonski]

Thanks @ForNeVeR! I'm looking forward to the follow-up on interfacedef use cases (other than marker interfaces).

[ForNeVeR]

I have asked around and got confirmation: you're supposed to register custom serializers for your custom interface implementations.

You're passing a ISerializers object when creating a protocol, and may call register (or registerSerializersOwnerOnce, which is usually called from generated models) on it to add your custom serializers.

Maybe you even could register a serializer for your generated model, so that it always creates your own model inheritor instead of the generated one.

[al-darmonski]

That seems quite useful for my use case. Can you point me to (public) examples that I can look at?

[ForNeVeR]

Unfortunately, there're no known public examples of this technique.

[al-darmonski]

Hello @ForNeVeR,
I've tried custom serializers. They work fine when used with baseclass. But not when used with openclass.

Here is what I tried for both baseclass and openclass:

1. Create inheritors of generated classes (baseclass and openclass).
2. The inheritors' companion object implements IMarshaller and thus its read() and write() methods.
3. Register my inheritors with ISerializers object before passing it to Protocol constructor.
4. At the server side create inheritors objects and adds them to a list in the top-level RD model.
   • write() of baseclass inheritor is engaged
   • write() of openclass inheritor is NOT engaged but the write() of the openclass itself
5. At the client side corresponding objects are created/synced.
   • read() of baseclass inheritor is engaged
   • read() of openclass inheritor is NOT engaged but the read() of the openclass itself

It seems like an inconsistent/undesirable/unintended behavior to me. Or do I miss something?

Thank you for providing information and support!
I was wondering what is the development cycle of RD? What is a time line that we can expect a recognized issue to be addressed? I'm asking so that we can set our expectation right and potentially look for (temporary) workaround(s).

[ForNeVeR]

@al-darmonski, could you please share the sources? It's very hard to understand what's going on without them.

I was wondering what is the development cycle of RD?

We develop new features and fixes on the basis of necessity for our main use cases (we accept community pull requests, though!).

There are no immediate plans for fixing that issue. I'd say that the models should be very simple, and there shouldn't be any need for custom serializers. If you write these, then you're mostly in the uncharted territory, since there's little use of this feature in Rider and other products we develop.

[micfort]

I work with @al-darmonski at Sioux.

We found that it even is even happens with non-custom serializers. I tried to create a simple example with openclass in the RD model, but I got an Fatal error. Internal CLR error.

I used the following RD model

@file:Suppress("unused")

package model

import com.jetbrains.rd.generator.nova.*

const val folder = "demo"

object DemoModel : Root() {

    val a = openclass("a") {
        property("prop1", PredefinedType.string)
    }

    init {
        property("a", a)
    }
}

And used the following C# code with that. (this is as well the server as client, based on a command line parameter)

using System;
using System.Net;
using JetBrains.Collections.Viewable;
using JetBrains.Lifetimes;
using JetBrains.Rd;
using JetBrains.Rd.Impl;
using rdTest2Implementation.model;

namespace rdTest2Implementation
{
    class Program
    {
        private static ApplicationType _applicationType = ApplicationType.Client;
        
        private static int port = 10001;
        private static LifetimeDefinition ModelLifetimeDef { get; } = Lifetime.Eternal.CreateNested();
        private static LifetimeDefinition SocketLifetimeDef { get; } = Lifetime.Eternal.CreateNested();
        private static IProtocol Protocol { get; set; }
        private static IScheduler Scheduler { get; set; }

        private static DemoModel Model { get; set; } = null;
        
        private static Lifetime ModelLifetime { get; set; }
        private static Lifetime SocketLifetime { get; set; }
        
        static void Main(string[] args)
        {
            if (args.Length != 1) throw new ArgumentException("needs one parameter");
            if (args[0].ToLowerInvariant() == "server")
                _applicationType = ApplicationType.Server;

            ModelLifetime = ModelLifetimeDef.Lifetime;
            SocketLifetime = SocketLifetimeDef.Lifetime;
            
            Scheduler = SingleThreadScheduler.RunOnSeparateThread(SocketLifetime, "Worker", scheduler =>
            {
                IWire client;
                IdKind idKind;
                switch (_applicationType)
                {
                    case ApplicationType.Server:
                        client = new SocketWire.Server(ModelLifetime, scheduler, new IPEndPoint(IPAddress.Loopback, port),
                            "server");
                        idKind = IdKind.Server;
                        break;
                    case ApplicationType.Client:
                        client = new SocketWire.Client(ModelLifetime, scheduler, new IPEndPoint(IPAddress.Loopback, port), 
                            $"client");
                        idKind = IdKind.Client;
                        break;
                    default:
                        throw new ArgumentOutOfRangeException();
                }
                var serializers = new Serializers(ModelLifetime, scheduler, null);
                Protocol = new Protocol(_applicationType == ApplicationType.Server?"server":"client", serializers, 
                    new Identities(idKind), scheduler, client, SocketLifetime);
            });
            
            Scheduler.Queue(() =>
            {
                Model = new DemoModel(ModelLifetime, Protocol);
            });

            while (true)
            {
                Console.Out.WriteLine($"Press any to check type and close");
                switch (Console.ReadKey().Key)
                {
                    case ConsoleKey.A:
                        Scheduler.Queue(() =>
                        {
                            Model.A.Value = new A();
                        });
                        break;
                    case ConsoleKey.B:
                        Scheduler.Queue(() =>
                        {
                            Console.Out.WriteLine($"result: {(Model.A.Maybe.HasValue?Model.A.Maybe.Value.GetType():"Nothing")}");
                        });
                        break;
                }
                
            }
            
            SocketLifetimeDef.Terminate();
            ModelLifetimeDef.Terminate();
        }
    }


    enum ApplicationType
    {
        Server,
        Client
    }
}

And the log is:

14:08:45.988 |V| JetBrains.Threading.ByteBufferAsyncProcessor.ClientSocket-client-Sender | :1                             | PAUSE ('Disconnected') :: state=Initialized
14:08:46.126 |V| JetBrains.Rd.Impl.SocketWire+Client | ClientSocket-client-Receiver:8 | ClientSocket-client : connecting
14:08:46.129 |V| JetBrains.Rd.Impl.SocketWire+Client | ClientSocket-client-Receiver:8 | ClientSocket-client : connected
14:08:46.164 |V| JetBrains.Threading.ByteBufferAsyncProcessor.ClientSocket-client-Sender | ClientSocket-client-Receiver:8 | RESUME :: state=AsyncProcessing
Press any to check type and close
Fatal error. Internal CLR error. (0x80131506)
   at System.Buffer._Memmove(Byte ByRef, Byte ByRef, UIntPtr)
   at System.Buffer.Memmove(Byte ByRef, Byte ByRef, UIntPtr)
   at System.String.Ctor(Char*, Int32, Int32)
   at JetBrains.Serialization.UnsafeReader.ReadString()
   at JetBrains.Rd.Impl.Serializers+<>c.<.cctor>b__82_9(JetBrains.Rd.SerializationCtx, JetBrains.Serialization.UnsafeReader)
   at JetBrains.Rd.Impl.RdProperty`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Read(JetBrains.Rd.SerializationCtx, JetBrains.Serialization.UnsafeReader, Je
tBrains.Rd.CtxReadDelegate`1<System.__Canon>, JetBrains.Rd.CtxWriteDelegate`1<System.__Canon>)
   at rdTest2Implementation.model.A+<>c.<.cctor>b__9_0(JetBrains.Rd.SerializationCtx, JetBrains.Serialization.UnsafeReader)
   at JetBrains.Rd.Impl.RdProperty`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].OnWireReceived(JetBrains.Serialization.UnsafeReader)
   at JetBrains.Rd.Impl.MessageBroker.Execute(JetBrains.Rd.Base.IRdWireable, Byte[])
   at JetBrains.Rd.Impl.MessageBroker+<>c__DisplayClass8_0.<Invoke>b__0()
   at JetBrains.Collections.Viewable.SingleThreadScheduler.ExecuteOneAction(Boolean)
   at JetBrains.Collections.Viewable.SingleThreadScheduler.Run()
   at JetBrains.Collections.Viewable.SingleThreadScheduler+<>c__DisplayClass15_0.<RunOnSeparateThread>b__0()
   at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

For completion the versions I have used:

• RD-gen: 0.213.389
• RD Framework: 2021.1.1
• dotnet: 5.0.206

[ForNeVeR]

I'm sorry, but I wasn't able to reproduce the issue. Here's a test project I used (copy-pasted your code and added build/run infrasturcture). Could you please help me with that?