The road to min 1.0.0

[h3rald]

Right, I started thinking about releasing version 1.0.0 of min. The idea is to do so once the API stabilizes a bit, and overall it has been mostly stable (except a few symbols being removed or changed) for quite some time now, so we could be close, but there are still quite a few things that don't feel quite right yet, and possibly some (major?) incompatibilities to introduce before the big milestone. I just wonder what other people think...

What to include in the prelude?

At present, all standard library (native) modules are automatically imported at startup. This happens in the prelude file which is executed every time min starts up. While this is convenient and (until recently) it was the easiest way to use the symbols exposed in those modules, it also meant that there could be clashes in symbol names... which in turn lead to symbols being called fwrite or dswrite etc., just to avoid collisions.

Now min provides a more robust way of requiring modules and invoking methods, so maybe instead of using dswrite it would be possible to just call *dstore/write, without importing the whole dstore module by default.

What do you think? Should the prelude only load and import a few of the stdlib modules like Lang , str, seq, but maybe don't import modules like crypto or dstore?

Searchable min-lang.org

This is an obvious one. min-lang.org should be searchable, to make it easier for developers to go through the min documentation.

I'll probably use LiteStore to add a full-text search API. The only problem is that at present there would be no direct link to a specific min operator, as they are grouped by module, so if you search for invoke you'd end up on the lang module page.

"try min" online

It would be cool to be able to try min online. Other languages like Ruby and Nim itself provide a way to do so. For example, the Nim playground lets you compile your own Nim program on the fly.

The code is actually open source, and they are basically executing the compilation in a docker container. Min could provide something similar, but it would be better to offer a REPL-like experience. The catch? Security obviously, because I don't fancy exposing the full power of min to the internet unless it's through a completely sandboxes environment. Any idea on how to realize this? Yes, probably a docker container...

On the other hand, there are so many programming languages that don't offer anything like this and are surviving just fine, so meh...

min flavors vs. additional compilation flags

min currently comes in three flavors: standard, lite, and mini. Additionally, you can compile it with or without SSL support. Is this really needed? Would it make more sense to provide different compilation flags to specifically enable/disable each module (besides lang)?

Not entirely sure. Maybe provide many different switches and release just the standard (full) min could be better. Any thoughts on this matter?

(See also #76 for more details)

What else is missing?

Then... well, there are a few other things on the radar, some are already planned as a matter of fact, others not so much. Let's see...

Multiline comments (planned)

Originally I was not gonna add them but they are kinda useful after all. They are probably going to be lispy flavored (like single-line comments), so something like this:

#|
  This is a comment
|#

custom type definitions (planned)

This could be a very useful extension to make the operator symbol even more powerful. Essentially, create a new "type definition" representing a check to execute on a value. More info on #73.

proc module (planned)

While there's already exec and run, it would be useful to provide access to running processes, suspend them, stop them, start them, etc. Essentially a min wrapper of some of the functionalities exposed by Nim's osproc module.

Object system (probably dropped)

I don't think I am gonna bother with this. After all this language is supposed to be minimalist! You can already use dictionaries (and invoke works on plain dictionaries as well), and typed dictionaries like modules, errors, etc. I wouldn't like to add things like inheritance, not gonna go down that rabbit hole -- just use functional or concatenative programming instead!

Dynamic libraries (probably dropped)

This used to be an implemented feature (see #8), but it wasn't working reliably, so it has been dropped for some time. Don't think we can resurrect this unless someone else comes up with a nice and tidy (with docs and tests) PR.

xml module (not sure)

Not sure if this would be beneficial or not. The idea would be exposing a module to easily write XML (and HTML) code in min. There's something like this for some lisps, e.g. PicoLisp's html function, but in a concatenative language with postfix notation it could look somewhat weird.

Imagine something like this:

(("This is a " ("Test" em) h1) html)

Interesting, but not very intuitive. Perhaps it could operate in prefix notation instead, but then you'd expect some sort of interop with strings containing XML code I guess, and so an XML parser should be included in the language... not sure, but it could be cool.

yaml module (not sure)

Same as above... a YAML parser is quite big, and not sure if it would be that useful considering min is fully-interoperable with JSON already. There are already from-yaml and to-yaml symbols, but they only work for simple objects containing strings (basically useful for extremely simple semicolon-separated frontmatters).

sqlite module (not sure)

This would be cool, but it would increase the size of the executable. That's the only drawback really, otherwise I would have implemented it already. Instead, currently there's support for persisted in-memory (JSON) data stores in the dstore module. Kinda does the job for persisting (small amounts of) data, but it's far from being called a proper datastore, nevermind a database.

ui module (not sure/stuck)

Creating minimal UIs in min would be awesome, the problem is how. There's a lot of good UI libraries out there, some even fairly minimal but I didn't find anything suitable right now, especially nothing that has already been powered to Nim. Some of the possibilities are:

• NiGui — but the fact that it doesn't yet work on OSX is somewhat off-putting.
• nimx — Probably the most mature GUI library for Nim, but not really minimal.
• webview — A WebView wrapper in Nim. I like the idea of writing UIs in HTML/JS/CSS as it simplifies a lot of things (for me, as a web developer). Not sure how big it gets.
• Neel — Run your HTML/JS/CSS user interface in a locally-deployed web server. Simple idea, just not sure how secure it is.

Benchmarks (not sure/stuck)

Last time I tried implementing some benchmarks it didn't quite work out as planned and I got depressed (see #31). The fact is that min is fast enough. How much is enough you say? Well, enough.

Improved multi-line REPL (stuck)

Cool idea but difficult to implement. See #52 and h3rald/minline#12.

compile to JavaScript or WASM (stuck)

That was the idea behind minimin. Tried it, didn't go too far. If anyone else wants to give it a try, please do! It could be an easy way to implement an online min playground!

[h3rald]

@baykus871 have a look at this list... any thoughts?

[ghost]

My thoughts are branched to Min-side .Otherly, there are some topics that i cannot say anything about themselves.

[ghost]

I am leaving Nim-side developing to Nim developers. I'm sure that you can handle with that.

[drkameleon]

As a fellow language-designer (in Nim), first of all I must admit I admire your patience and commitment and craziness (and mine too lol). So, well done!

Not that I'm an expert, but a few thoughts since I noticed the post and I have - at one point or another - played with some of the ideas...

"try min" online

I totally like the idea - even though my own - similar - site has been "under construction" for the past weeks. Most users want something quick and easy to get their hands dirty with. And that's probably the easiest way to make a new language accessible.

Object system (probably dropped)

I'm not a fan of too... object-oriented languages and leaning towards minimalism is most definitely the opposite. I consider them too bloated. Plus, they are going slowly out of trend (thank god!).

xml/yaml/sql module

Done that. Because, why not? I would prefer a minimalist, albeit as-batteries-included-as-possible approach. Plus, with Nim in mind, it's not difficult at all.

ui module (not sure/stuck)

I have focused on the Webview idea. Think of Flutter but in a more ... alternative language. Why not? Also, libui is a great - quite OS-agnostic - option.

---

In any case, great job! Keep up the passion. You have my respect! ;-)

[drkameleon]

@h3rald Hi!

Well, bytecode is just an idea. I mean... initially I thought of it simply as the way to do it, if you want your VM to be fast. (basically, "since everybody's doing it, it must be good" kind of logic).

Now, after having dealt with bytecode for the past few years, a few "inside" ideas:

• yes, obviously if you have the bytecode, and your VM can directly execute it (= without going through the parsing stage), you can obviously save the time it takes to parse the script (and the larger the script, the more you save)
• Arturo in the past year at least has also introduced an AST representation, despite my initial reluctance. In a few words, the way compilation works goes like this: script ---(parser)---> values ---(ast processor)---> ast ---(evaluator)---> bytecode ---(exec)---> program execution.
• This addition (along with the very nature of bytecode itself) has lead to a very substantial speed-up; I would estimate the overall gains (compared to the version before) could be up to 400%.
• Now, the reason is quite simple: bytecode is not just a serialized pre-encoded translation of the initial script; it also allows for many optimizations at the bytecode level (and in that case, you can see the performance boost, even when you still keep the parsing stage in the whole compilation queue).
• Bytecode operations (what each bytecode actually does) - although not generally true - are thought/meant to represent a single, simple and basic operation, which should be roughly equivalent in time needed. So, let's say you have 4 bytecodes, that take 4X of time. Are you able to reduce these 4 operations down to 3? Then you'll have a 25% performance boost! (it's very simplistic and not totally accurate at all times, but it's still an interesting way to look at it)
• Also, the AST introduction alongside the bytecode works wonders: I mean... even things that I hadn't even imagined work fine. Expression reduction? (a complicated arithmetic expression that can be pre-calculated, or different expressions that can be transformed to simpler ones) Done. Bytecode simplification? (e.g. a + 1 - 3 operations - can be transformed to inc a - 2 operations) Done. The same goes for loops and if-else, conditionals in general and everything. I mean... technically an if-statement, if you look at it as if it was assembly, it's just a conditional jump. Bytecode actually makes it work like a breeze... 😉

---

I'm saying too much. But if I had to resume the whole thing into a short "TL;DR" type of thing: I would be a totally vocal proponent of bytecode when implementing a VM. (And paired with a stack and an AST, the mix is... explosively performant! :) )

---

Feel free to ask anything else! I've always been watching Min and I would totally like to help in any way I can!

Cheers from 40-degrees boiling-point Granada!

[drkameleon]

For further reference (not that any of this is like THE implementation, but oh well...):

• The main bytecode implementation (all of the opcodes, paradoxically quite well documented in a table; and, yes, the initial inspiration is the JVM, despite my total and absolute dislike of Java): https://github.com/arturo-lang/arturo/blob/master/src/vm/opcodes.nim

• The AST implementation: https://github.com/arturo-lang/arturo/blob/master/src/vm/ast.nim, with various optimizations: https://github.com/arturo-lang/arturo/blob/master/src/vm/ast.nim#L279-L406

• Bytecode optimizations (at the bytecode level, mainly conditionals/loops/etc): https://github.com/arturo-lang/arturo/blob/master/src/vm/eval.nim#L418-L464

[inivekin]

Hey sorry to butt-in on the convo, but I noticed what the nuance nim module can do the other day and was wondering if you thought that might be useful for the compile mode of min (by producing actual nim ast for some parts of min that could use performance improvement?)

I have little clue about making programming languages and also am not greatly familiar with min or the nuance package or even nim itself - so it won't surprise me if you decide to just ignore this whole comment without reading further - but was thinking maybe I could lazily play with these things if you thought it might be able to fit? Basically creating specific nim ast that gets called on for, perhaps, type annotated dictionaries and simple arithmetic or something, at least to start? Not sure about details like fitting it into scopes, error handling, etc - but wanted your opinion on if it was unreasonably difficult in ways I didn't think of before trying too hard to iron it all out.

Just linking this comment into this chain because I thought why make a bytecode VM to improve performance when you could make something that compiles the expensive parts to machine code if needed (though maybe it's not as fun of an idea).

[h3rald]

Thank you both for your interesting replies.

Concerning the bytecode interpreter, I think I may just try it out because with a stack-based language like min it just seems stupid not to implement a stack-based VM. I had a look at the optimizations and yes, maybe it would be possible to introduce some arithmetic optimizations and short-circuits some conditionals (although min provides operators for doing that already) but I am not sure what other optimizations I could introduce. I'll give it a try, run some benchmarks, and see. I am actually looking forward to implement my own VM with my own opcodes for min, and I will just do it for fun.

Concerning the nuance project... I am struggling to understand how that could be beneficial. The thing is, I can already generate Nim code from min and it is pretty basic code as well (again, stack-based language so at the end of the day I am always pushing something on the stack)... so I am not sure. I would even say that there is no need of an AST in a stack-based programming language, because in a way it is so low-level that the only thing you do is pushing stuff on the stack, and at most evaluate it. That's it. Conditionals, loops, expressions... they are all ultimately symbols that get pushed on the stack and consume some arguments that have been previously pushed on the stack.

Let's see... for the minute I am kinda busy trying to implement multiline support for my line editing library, and that is proving to be quite challenging. I really gotta do it though, I feel I am getting close.

After that I'll implement an xml module for parsing and generating xml/html and then tackle the VM implementation.

[drkameleon]

I am actually looking forward to implement my own VM with my own opcodes for min, and I will just do it for fun.

You'll love it; trust me! 😄

[ghost]

Object system (probably dropped)
I don't think I am gonna bother with this. After all this language is supposed to be minimalist! You can already use dictionaries (and > invoke works on plain dictionaries as well), and typed dictionaries like modules, errors, etc. I wouldn't like to add things like inheritance, > not gonna go down that rabbit hole -- just use functional or concatenative programming instead!

This can be done via custom type definitions and dictionaries. Special types offers special dictionaries too. And that makes: dict:Car dict:Vector dict:Deque etc.

After that, there can be some methods to handle with these like:

( symbol len
  (dict:vector :vector ==> num :result)
  (...)
) ::

[ghost]

At this time, this makes symbol overloading problem. That problem can be solved with symbol inferring system. The system will infer symbols where are they came from. Maybe.

[ghost]

Or just need to define same symbols in from different files that uses same symbol.

I say:

... *vectorfile/len
... *soundfile/len

[ghost]

You don't like but generics should be usable in Min. For example of usage:

( symbol mray
  (string|num :gnc) ; --> This will be new!
  (gnc :begin    gnc :end    natural :step)
  (...)
) ::

; Desc: Makes a ray which contains consecutive elements.

and then:

1    10  2  mray puts    ; (1 3 5 7 9)
"a" "c" 1   mray puts    ; ("a" "b")
1   "z"  3   mray puts   ; --> <ERROR will be raised!>

[h3rald]

@baykus871 Another proposal for a possible syntax... I think it's a good compromise between what you proposed and what I was thinking about:

(
  symbol ++
  ((string|num|quot :t) :a t :b ==> t :result)
  (
   (a type 'string ==)
     (a b suffix @result return)
   when
   (a type 'num ==)
     (a b + @result return)
   when
   (a type quot ==)
     (a b concat #result return)
   when
  )
) ::

Whay do you think? Basically generics could be defined on-the-fly in a quotation when used for the first time in the signature. Also coupled with custom type definitions this could become very powerful.

[ghost]

Okay! It is more great, more perfect solution! I just thought a typeface as a box where we can define generics and special types that belongs to operator definition. But this is not so important. I am on your side! Your definition, if you can handle within Nim, is good. If any missing of that definition happens then we'll think about later. This is no problem. However please don't implement string^num . There are a lot of problems that i am thinking for solving them in my head. If you use string^num implementation, somethings may be won't work properly and probably more problems will occur and i don't want to make you bored with problems and me too.

string^num syntax is not fully equals to - i think - generics

[ghost]

@baykus871 Another proposal for a possible syntax... I think it's a good compromise between what you proposed and what I was thinking about:

(
  symbol ++
  ((string|num|quot :t) :a t :b ==> t :result)
  (
   (a type 'string ==)
     (a b suffix @result return)
   when
   (a type 'num ==)
     (a b + @result return)
   when
   (a type quot ==)
     (a b concat #result return)
   when
  )
) ::

Whay do you think? Basically generics could be defined on-the-fly in a quotation when used for the first time in the signature. Also coupled with custom type definitions this could become very powerful.

One more thing:

Flying type definitions in symbol parameter quotion should be parallel working with lexical scoping.
For example:

(:n	 n (int) expect ->	0 >=) 'natural typedef

( symbol mray
	(
		(string | int :t)
		t :begin  t :end  natural :step
	)
	(
		; ...
	)
) ::

A custom type natural used in mray. mray may wants to define its own natural type for constrainting:

(:n n	 int?  n 0 >=  and) 'natural typedef

( symbol mray
	(
                ( (:n    n int?  n 0 <   and) 'natural typedef) ;mray will use this 'natural definition
		(string | int :t)
		t :begin  t :end  natural :step
	)
	(
		; ...
	)
) ::

This helps with blocking some future conflicts that can be happen with next-time custom type definitions.

[ghost]

Is it possible?

[ghost]

Yeah mmm not sure either. Another possibility could be introducing the concept of "exclusive unions" for types:

(string^num :n string^num :m ==> string^num :result)

In this signature parameters can be either string or num, but once you choose one the next one can't be different. A bit too verbose maybe...?

The problem here is that we are not just trying to introduce a placeholder for a type, we are also trying to add constraints at the same time, and those constraints are valid for one specific signature.

I understand your original proposal of adding an extra quotation to define generics but... well, the operator definition is already quite complex, I wanted to avoid adding one more quotation to it (which furthermore must always be present, and in that specific position, whether you are using genetics or not).

For now I can see that I can implement easily the two things separately:

• custom type definitions (or type classes), valid everywhere
• type placeholders which identify one specific type within a signature (but without constraints) -- i.e. t, t1, t2

So far the only way to do both that I can think of (but may need some more time for a better idea) would be the "exclusive union" syntax string^int, but that would have to be repeated several times within the signature, so that's not great either. 😞

You are saying it's hard to add a quotation for generics. I understand you. We should a try with your proposal.

[ghost]

Multiline comments (planned)
Originally I was not gonna add them but they are kinda useful after all. They are probably going to be lispy flavored (like single-line >comments), so something like this:

#|
This is a comment
|#

That is useful!

[ghost]

@h3rald
Will we be able to do (1 2 3 4 5 6 7) #| HEY! I AM A COMMENT! |# puts ?
The idea is commenting in code

[h3rald]

Yeah that's the plan multiline comments could go anywhere

[ghost]

Searchable min-lang.org
This is an obvious one. min-lang.org should be searchable, to make it easier for developers to go through the min documentation.
I'll probably use LiteStore to add a full-text search API. The only problem is that at present there would be no direct link to a specific min operator, as they are grouped by module, so if you search for invoke you'd end up on the lang module page.

CTRL + F ⌨️

[h3rald]

Yeah... exactly my point, as long as you know the module, you can just do CTRL+F and find most of the stuff. If you don't know the module you may try a couple of times (was hit io or lang? math or num?) and eventually get there. Maybe not worth the hassle...

[ghost]

Yes CTRL + F is not a solution for the problem. But you can use if you need to search something that you know what module is it declared in.

[ajusa]

Overall, I do like a lot of the ideas in this post. I'd like to refer to the issue I made almost three years ago. Since it seems like dynamic libraries aren't going to be a thing (which I honestly think is for the better, as they are a hassle to get working cross platform in other programming languages as well), would it be possible to add a simple wrapper around nimquery for web scraping? Many scripting languages have web scraping functionality. min can technically web scrape - it just isn't easy and requires parsing the HTML/XML by hand. I noticed that you were considering adding DSL for HTML/XML. Before that, I think it might make sense to add a way to quickly match existing HTML/XML strings.

min flavors vs. additional compilation flags

I think that only a standard min flavor (the one with everything) should be offered unless someone can make a good case for the other smaller versions. Fragmentation is never good for languages, especially since (as far as I know) to run a certain min script you might need to change the flavor that you have installed on your machine. This could be replaced by a section that describes how to compile a more minimal version of min, similar to the one currently on the website. That way power users can still have the customization that they need without confusing novices.

"try min" online

I haven't read any other related information to this feature, so correct me if I am wrong. I think that a try online website would be pretty nice, just because min is pretty unconventional compared to other languages. I think that this is a nice feature, but I wouldn't consider it essential. The WASM route is probably the best one, but you have already tried it so it isn't simple/straightforward.

GUI

I would shy away from frameworks that describe GUIs using only web languages. Specifically, I don't know how I feel about using JS instead of min for a GUI application. Very generally, I think that a non-native gui (fidget, mui, dear-imgui, nuklear, etc...) would probably be best. A native GUI isn't consistently displayed between different machines. Additionally, non-native GUIs are easier to extend and work off of. While this might be abusing min (a lot), if someone wanted to implement something like google maps using min, the framework itself shouldn't limit them.

sqlite module

I think that this might be kind of nice to have, but also opens up a can of worms with supporting other DBs. This might be the exact kind of problem dynlibs would solve lol. sqlite is a good fit though if you decide to only include a single database, as it is fairly minimal as very fast.

Quite a bit of text here, but remember that you don't need to implement anything I say 😄. You've done a great job with min and I do enjoy using it for smaller tasks with nim would be overkill, especially the REPL. Keep up the good work!

[h3rald]

Ohhh yes! Well, since that issue a lot more things has been sneaking in erhm... to be honest HTTPS support did it: I realized that most of the web is HTTPS-friendly now so it didn't make sense not to provide a way to support it while offering an HTTP module. And that's was the end of the 1MB limit for the min executable 🤣

And now I'd like to contradict myself even further and say that a minquery wrapper would be a nice addition to min at this stage... The language is becoming more and more batteries included, and the goal should be to be being practical enough for doing small tasks, and get something done easily.

As for the different flavors, well, I'd like to keep at least one simple way of building a more lightweight executable and especially exclude extra libraries like OpenSSL, sgregex, then maybe SQLite when you want to embed the language like I do with HastyScribe and HastySite. But on the other hand Nim'a dead code elimination would work anyway in these cases? Not sure if it's smart enough to remove a statically linked library if not used, probably not.

The one flavor that doesn't make much sense is the new mini flavor, and the only reason why it's still there is because I'd still like to use that as a base to compile to JS or WASM (and provide a try min feature). But no, for now I didn't have much luck with it.

So yes, SQLite is probably going in as well. I have been using the new dstore module as a way to persist data for a simple web app and while it works fine for small apps and prototypes, SQLite is awesome and would be a nice fit. I used it a lot with LiteStore and I love it. On the other hand I could also expose a LiteStore module for low level access but... meh, it offers an HTTP API for a reason after all.

As for GUI... I understand. I would like something really lightweight though, and sadly I have no experience with any of those and support in Nim is not exactly great right now.

Thanks!