All languages (COBOL first) - Search for COPYBOOKs / INCLUDEs in Git Repositories

[FALLAI-Denis]

Description of the enhancement requested

IBM Z Open Editor can search for COPYBOOKs or INCLUDEs in the local Workspace, or on a remote z/OS system.

In a context of source management under Git, Z Open Editor should also be able to search for COPYBOOKs and INCLUDEs in Git repositories, in particular to access COPYBOOKs and INCLUDEs describing interfaces published by Applications external to the current Workspace.

This would also conform to the implementation provided by IBM Dependency Based Build, and more particularly by the zAppBuild, which may not be required is still recommended.
Cf. Managing the build scope in IBM DBB builds with IBM zAppBuild

Without this possibility, it forces to duplicate the COPYBOOKs and INCLUDEs on the z/OS remote system, so a double management between Git and the storage on the z/OS remote system.

[FALLAI-Denis]

An alternative to git cloning to get external COPYBOOKs / INCLUDEs should be to use http request to Git Repositories Server.

In our case we use Bitbucket.
Bitbucket expose an API to obtain an artefact in a Git Repository. So Properties group should use a generic url with placeholders.
This should apply to any http server, not necessary a Git Server.

[phaumer]

Pointing to generic http urls is an interesting idea that we can explore, but our first approach would be to support multi-root workspaces in VS Code in which each sub-workspace could be from a different git repo and and branch.

[FALLAI-Denis]

Hi @phaumer

The operation of Z Open Editor in a multi-root workspace context is a need that we have identified ourselves.
For example, for the same application, we manage several Git repositories and it is all of these Git repositories that constitute the context of the Application.

However, with regard to access to COPYBOOKs / INCLUDEs managed in Git Repositories, forcing to clone the Repositories of external applications will have implications:

• potentially large volume of data to be brought back to the workstation, (with a security risk in the event of theft of the workstation).
• obligation to refresh the Repositories of external Applications regularly when they are not part of the Application under development.
• control of the use of the branches of these Repositories (how to control that it is the COPYBOOKs / INCLUDEs of the master or develop branch according to the construction context which are referenced when editing the source code).
• control of use limited to "public" COPYBOOKs / INCLUDEs.
• access rights to Git repositories.

Accessing a Git repository directly via http requests will also not solve some of the problems discussed above.

For these reasons, we are considering developing a "proxy" server for accessing the COPYBOOKs / INCLUDEs of external applications, rather than direct access to the Git repositories.

This "proxy" server could apply management rules, such as:

• determination of the Git Repository according to a naming standard for COPYBOOKs / INCLUDEs
• determination of the branch to use in the external Git Repository according to the working branch on the current application in Z Open Editor
• limitation of access to "public" COPYBOOKs / INCLUDEs according to an organization of files to be respected in the external Git Repository
• and many other rules that can be linked to the organization specific to each company.

This "proxy" server might not be limited to accessing Git repositories. It could also access other repositories and therefore expose a unique solution to Z Open Editor to search for COPYBOOKs / INCLUDEs.
This could also meet some of our needs expressed in other issues (management of the search context of COPYBOOKs / INCLUDEs according to the Git branch used in the workspace).

As input to the "proxy" server, the search context would be needed:

• which name of COPYBOOK/INCLUDE is searched
• what is the current development context: workspace name, current program name, current program language, current Git branch (feature, hotfix...), etc.
• set of external references to query (in a symbolic form, such as application references)
• free attributes

In fact, this "proxy" serveur need a mix between the information carried by the properties-group, the information carried by the IBM User Build interface, and the current development context.

This same "proxy" server could also be used by IBM Dependency Based Build, thus simplifying the writing of Groovy scripts and probably even allowing Groovy scripts not linked to the context of the application being built, (in the examples provided, the Groovy scripts are linked to the Application because access to the Git Repositories of external applications is hard-coded in the script).

[FALLAI-Denis]

Hi @phaumer

What do you think about Git submodules ?

My idea is to create Git repositories dedicated to "resources" as COPYBOOKs and to reference them in main Git repositories as submodules.

When cloning the main Git repository, all "resources" Git repositories are also cloned, so all COPYBOOKs are local, no more access to mvs are required.

Probably same process can apply to IBM DBB.
Groovy scripts are simplified.

[FALLAI-Denis]

Hi,

In my post #233 (comment), I suggest using a proxy system to access COPYBOOKs.

This would be the equivalent of the IBM Enterprise COBOL compiler exit LIBEXIT.