Merlin impressions

[worksofliam]

I got to see IBM's new Merlin product last week and I am quite excited about some parts of it. I wanted to write a blog post to write about some of my findings at a technical level.

There are three main parts to Merlin that I am covering right now (May 2022):

• The vault and Inventory
• Merlin CI/CD
• Merlin IDE

To put it briefly, I am excited for Merlin and I am looking forward to seeing how IBM will make it more extendable for users and vendors alike. Merlin really pushes developers into the right direction and that is so exciting to see.

The vault and Inventory

The reason these two are grouped into one on this post is because they are somewhat connected to each other. Inventory is where Merlin configures connections to different IBM i LPARs. Some shops have many development environments, and you can consider this the inventory of IBM i LPARs in your private or public cloud.

The vault is where usernames, passwords, keys, and much more, are managed. Inventory and vault largely play a part when setting up development environments and CI/CD processes.

You will also be able to manage Merlin users here. This will allow you to create accounts for your developers to sign onto the instance of Merlin.

CI/CD

CI/CD is one part of Merlin. To keep it brief, the CI/CD part of Merlin is a layer above other CI/CD orchestrators. The main one that Merlin currently supports is Jenkins but may eventually support more (like Bamboo, etc). This means that Merlin CI/CD does not implement its own pipeline processes, or containers to manage deployment, but instead creates the steps for Jenkins to run them. Moral of the story: it's a UI above Jenkins and will automate creating pipelines in Jenkins from the Merlin UI.

The Merlin CI/CD UI is actually quite nice. It allows you to set up an instance of Jenkins automatically, or point it to an existing instance that you may have set up elsewhere.

You can create what is called a 'profile', which is effectively a Jenkins pipeline made up of many Steps. Merlin CI/CD provides some very basic steps, such as:

• deploying files to the IBM i
• running a CL command
• others I cannot remember

Your 'profile' setup (the set of steps) happens in the Merlin UI, where you can add and arrange steps dynamically. When you create or change your profile, it will make the necessary changes in the connected Jenkins instance. A profile can have a 'working directory' on the remote system, as well as being connected to a git repo (to deploy sources from there) and eventually a specific branch.

The idea is that the users will never really have to touch the Jenkins instance and can manage everything from Merlin. But, for really complex pipelines, then the user can access their Jenkins instances whenever they want.

Note that Inventory plays a large part here, as you can select which IBM i you want to run your 'profile' against, as well as select what user (from the vault) to use when connecting to it.

IDE

Merlin IDE is my interest. Merlin IDE is an instance of Theia managed inside of Merlin. A user can manage many instances of Theia from the Merlin UI (if they're working on many projects for example). Theia and Visual Studio Code are different. Theia is an open-source re-implementation of VS Code by the Eclipse Foundation. It can run VS Code extensions, assuming the extension doesn't use the latest and greatest VS Code APIs, until Theia can catch up.

Merlin IDE provides the RPGLE language server, among other languages, running commands on the remote IBM i, uploading the working directory to the remote server (rsync), and many other things. Neat!

The issue is, which isn't directly related to Merlin, is that it requires a certain workflow of development. It totally depends on using git to get the sources into Merlin. IBM are trying to push users to use source control and develop remotely using neat tools like Merlin IDE. Most IBM i businesses are still using source members. I have been pushing users to at least develop in the IFS with the use of git (which people are doing today!), but Merlin is expecting users to make two big changes:

1. Move the source code into the git (which means getting it into the IFS)
2. Get developers working with repositories and build tools

So: Merlin IDE is really neat, but there is some work to be done in terms of users and businesses to get their source code into git.

There is more, exciting, work to be done and I can't wait to see it grow. I can see the RPGLE language tools continuing to grow, as well as improvements for SQL, CL, and COBOL. Also, now that IBM is only encouraging this methodology of development (git-flow), hopefully, more businesses will move to it.

What about ARCAD?

The good news is that it seems like you don't need their tools. ARCAD is providing additional 'features' to Merlin CI/CD and IDE (at an additional cost I presume) that you can use. This is likely good if you're an existing customer, but if you're not then it's nothing to worry about.

ARCAD adds hooks to Merlin IDE to make calls to ARCAD Builder and add hooks to Merlin CI/CD to make use of ARCAD Drops. Using ARCAD Drops could be an additional step in your CI pipeline.

In the future, I imagine the intent is to allow other vendors to be able to extend Merlin to make use of their products also.

Update from ARCAD Employee: Twitter

ARCAD comes with Merlin at no additional cost though, since...it's a part of Merlin. [...] it would be a shame if nobody uses it, at least to get some feedback. I'll be happy to share more info soon, for sure!

I will update this again when more detail is shared!

More info

Check out the IBM Support page for more information. Feel free to leave any questions here or on Twitter.

[DavidERansdell]

Hi Liam,
Are you sure you want IBM to make it more 'expendable'? Maybe you mean 'extendable'?

Whatever you mean, it is good to have more tools available to more users and making them more accessible.

Thanks for the work you have done and are doing on VSCode.

[sebjulliand]

Hi! This is Seb from ARCAD from Twitter!
I'm the lead dev on ARCAD/Merlin integration (aka. if this part crashes, it's on me) and I'll try to provide insightful details about ARCAD in Merlin.

Disclaimer

I work for ARCAD - but on the tech side of things, so I'll remain objective (as much as possible).

ARCAD in Merlin

💰 Cost

Let's deal with this right away: if you buy Merlin, you get access to the full ARCAD toolchain too, and the support that goes with it. It's included in the license for at no additional cost. It's up to you if you want to use it or not, depending on your needs and if you feel like you can take advantage of the ARCAD suite.

Tools

For now, this is what is available to use from Merlin. More will be added in the future.

Observer 🔍

Cross-references analyzis and browsing. It helps finding what is used where and by who. From Merlin IDE, you can either ask to see the callers, callees and file usages of a component of any kind (RPG, CL, SQL, you name it) or you can perform a global search in the entire application repository (ie. the knowledge database of the application, maintained by ARCAD behind the scene). This is useful when refactoring or modernizing code, to get an idea of the impact a change could have accross the entire application.

Builder 👷

This is BOB's counterpart. It takes care of unitary compilations and builds. And unless you have specific needs tom compile your application, it will do the heavy lifting for you (ie. it will compile everything in the correct order and recompile what needs to be based on the application's cross references and what ARCAD knows of the application).
Builder also includes a REST API server that can automate builds (and a few other things), thanks to GitHub/GitLab/BitBucket/Azure/GitBucket webhooks support.
In Merlin IDE, you can see if a version build (ie. the build of a remote branch) is running or ask for a compilation or a developer build (ie. the build of the current workspace's content).
In Merlin CI/CD, there's an ARCAD Builder's build step that can be used.

Transformer RPG 💾 ▶️ 💿

It converts RPG to Fully Free RPG (all of it, no exception...yes, even gotos). Pretty straightforward.
The conversion can be done directly from Merlin IDE.

Jenkins plugin for Macro command execution 🏃

It's a step found in the CI/CD that allows the execution of ARCAD Macro commands. Macro commands allows CL scripting with extra stuff that makes it a powerful and customizable tool (think of it as what is Groovy for Java). A lot of stuff are based on macro commands in ARCAD which allows for a lot of cutomization. So this CI/CD allows to execute a macro from Jenkins, which allows to do more than a simple CL call.
For now you cannot modify or execute the macros from the IDE but...that will be arranged at some point 😉

🚀 What if I want to start an ARCAD project

It can be done by configuring a project to be used with ARCAD. This is done from the IDE, during the project's creation or afterwards. It adds some extra metadatas in the project's, so it is "linked to ARCAD".

💥 What if I wan't to get away from ARCAD?

Just fall back to using BOB for the build and that's it. There is nothing that will impact the project or the code while using ARCAD, so you're free to do whatever you want.

🌒 Behind the scene

We tried to keep things simple and seamless...as much as possible, there's plenty of room for improvement. The main point is to hide ARCAD complexity behind a lot of automation so developers don't have to worry about configuring a lot of stuff. In its current state, you can start a new IBM i project in Merlin using ARCAD without any specific ARCAD knowledge (keep the documentation close...just in case 😏).

There is still a lot to do on our side to make things even smoother and useful for developers, but I'm confident we'll get there.

💬 That's all folks

Reach out to me here or on Twitter @SebJulliand about anything related to Merlin and/or ARCAD, i'll be happy to oblige.