Avoid code repetition across tiers in full-stack app

[yakir12]

When building a full-stack app, I find it frustrating to define basically 3 nearly identical types:

1. What I consider the main struct: defining the logic of the type disregarding any GUI/DB needs. This is the struct you'd define normally. This struct would contain only the most basic fields needed to define it "in real life" (e.g. a User will have name, email, and password). Methods associated with it would only deal with actions that have to do with what it actually is.
2. The back-end struct: same as in # 1 but for the database, with the obligatory id and maybe public id for associations with other entities. Methods with this one would be CRUD.
3. The front-end struct: Again, same as # 1 but for the GUI. User-facing with some additional checks and validations.

As suggested in this Discourse topic, one way forward is to wrap the business struct (type # 1 above) with a wrapper struct that includes all the missing fields needed for the DB or the GUI. Currently, I think the pipeline to make this convenient is missing. For instance, in issue GenieFramework/Stipple.jl#47 a convenience wrapper for custom types is missing on the GUI end of things. On the other end, DB, the same problem exists: I can't SearchLight.findone(MyCustomType, field_name = field_value).

Right now, I need to basically replicate my custom type for each tier and add the missing fields for the two tiers (e.g. id for the DB). Or alternatively, have a business type that includes fields that "pollute" it's actual function (e.g. the id field has nothing to do with a user being a User on its own right).

This feels like a magnificent opportunity for Genie to be the first framework to solve this duplication of code. Imagine a framework where full-stack developers only need to define the business structs and their methods, while the front and back end automagically follow suit! This might be a high order, but if there is a way to make this happen, then the appeal to use the Genie framework should be huge.

[essenciary]

@yakir12 Maybe you need to explain your use case as I can't understand the pain point. Basically you want to invent a whole new architecture because you don't want to put one id field into your business model?

Also, why would you have types in the view layer?

Are you familiar with MVC? Do you use it?

[yakir12]

Basically you want to invent a whole new architecture because you don't want to put one id field into your business model?

That is more or less correct 😆

Maybe you need to explain your use case as I can't understand the pain point.

I'm trying to build a fairly complicated app, thankfully with someone much smarter and more experienced than me (a webdev that is new to Julia). Instead of building the front-end and have it directly affect the back-end, we're trying to build a middle tier that is free of the constraints and needs of the back- and front-ends.

Also, why would you have models in the view layer?

I must have misunderstood you: I mean, everything in a Stipple board is based on a "model" struct. So of course I'd need to have a struct for the front-end.

[yakir12]

Here is an example of what I mean.

Business side of things:

# a type of users
struct User
  name
  email
  password
end

# methods with users
hashpassword(u::User) = ...
sendemail(u::User) = ...
...

back-end:

mutable struct UserDB
  name
  email
  password
  id::DbId
end

front-end:

Base.@kwdef mutable struct UserUI <: ReactiveModel
  name::R{String} = ""
  email::R{String} = ""
  password::R{String} = ""
  register::R{Bool} = false
  reset::R{Bool} = false
end

It is true that I could solve most of the back-end duplication by just adding a id field to the business type and removing UserDB, but how would I avoid the duplication in the front-end?

[yakir12]

My much wiser friend has explained to me that the UserUI looks a lot like the User struct -- a duplication of code -- but that that is just a coincidence: we could have a had a form that allowed for changing the password, or updating the email, and that the form built by UserUI just happens to include all of the fields in User. User could have a ton of other fields that wouldn't be needed for a registration form such as the one built by UserUI.

So I think I'm seeing the light...

[yakir12]

Also, to avoid adding the id field to the business struct, we can do this:

abstract type HasID end

struct User <: HasID
  name::String
  email::String
end

Base.getproperty(x::HasID, field::Symbol) = field === :id ? hash(x) : getfield(x, field)

and then this works:

julia> a = User("a", "b")
User("a", "b")

julia> a.name
"a"

julia> a.email
"b"

julia> a.id
0xc537b8713bce82db

[essenciary]

OK, I understand now: you want to use Genie with Stipple combined with SearchLight.

There are lots of things and it's hard to comment as I don't know the details of what you're building. But here are some things.

1. There are various architectures: Genie are SearchLight are designed for MVC (multi page, server rendered web pages) while Stipple is MVVM (single page applications with data exchange). Genie+SearchLight are focused on CRUD type of apps while Stipple is for interactive data visualisation.

Mixing them might not work so you might want to chose one and roll your own code of the missing features. For ex, do you really need a reactive UI for a user to edit the password? Are you going to save the password to the DB on each key press? (I hope not). The user can submit a form, you apply the validations on the server side and redirect to a success or error page. So MVC/CRUD. This can be enhanced so the form is submitted via AJAX, using the same MVC/CRUD architecture, so there is no full page reload - but still, no reactivity (no Stipple).

This is a good read: https://hackernoon.com/mvc-vs-mvvm-how-a-website-communicates-with-its-data-models-18553877bf7d

2. It's not just about properties. SearchLight does not use duck-typing - that is, it's not just looking for an id field that you can add to any struct. The SearchLight API is designed around the AbstractModel type and the objects must inherit from that.

So there are language limitations. SearchLight works with children of AbstractModel and Stipple with children of ReactiveModel and because Julia does not support multiple inheritance, your objects can't be both AbstractModels and ReactiveModels at the same time.

3. The way I see it, very high level, is like this:
   a) I would have my "pure" core data structure/type. Say the User which has username, name, password.
   b) I would set up an API for converting/marshaling between "pure" and "persistable" (SearchLight models) and "pure" and "renderable" (Stipple models). So that you pass your pure User type, and it's converted to the corresponding SearchLight model which can be store/retrieved to DB (and the same, rendered to UI).
   c) add convert methods between pure and persistable and pure and renderable - and reverse.
   d) then you can work with your pure models and Julia will automatically convert.
   Eg: SearchLight.findone(PureUser, name = "John") |> pure
   (this will invoke convert to turn PureUser into a SearchLight user type, would run the query and retrieve the SearchLight model instance and pass it to pure which strips away the SearchLight properties, returning just the pure user type. Or maybe you don't even need to call pure if your API is designed to work with children of PureModel and set up conversions between/to SearchLight, Stipple, and Pure - then you can pass all these 3 types around, and Julia will use the convert methods to turn one type into another).

I hope it makes sense. I'm super busy the next 7 days or so, but if you want we can have a call towards the end of next week so you can tell me more and dig into the possible API/architecture.

Good luck!

[yakir12]

No, this is gold, thank you so much!!!

[essenciary]

Awesome, you're welcome!

Basically an API like:

abstract type PureModel
mutable struct User <: PureModel
convert(::PureModel, ::AbstractModel)
convert(::AbstractModel, ::PureModel)
convert(::PureModel, ::ReactiveModel)
convert(::ReactiveModel, ::PureModel)

I think that's the core. Let me know how it works if you try it 😁