Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Domino On Disk Project Examples #669

Closed
JoelProminic opened this issue Feb 26, 2020 · 44 comments
Closed

Domino On Disk Project Examples #669

JoelProminic opened this issue Feb 26, 2020 · 44 comments
Assignees
Labels
enhancement test ready Feature/bug ready for testing

Comments

@JoelProminic
Copy link
Contributor

JoelProminic commented Feb 26, 2020

We would like to have a Domino On Disk Project template that we can use to create a new local Domino database from Moonshine. We have done some work with this for #646 already, but we want to separate this out from the Visual Editor code for now.

The example template should have a simple "Hello, World!" example of each design element type, including:

  • Form
  • View
  • Subform
  • Page
  • XPage and SSJS - Added by @dpastov 2020/03/06
  • Shared action
  • Java agent
  • Java script library
  • Lotusscript agent
  • Lotusscript script library
  • Outline
  • etc.

In addition, we should provide some useful views. These should have web access disabled by default.

  • All By UNID
  • All By UNID/By Creation Date
  • All By UNID/By Form Used
  • All By UNID/By Last Modified
  • etc.

And we also want some useful shared actions for forms and views. This can be used for the forms and views generated by #670.

We should try to minimize Prominic references in the database code.

In general, we should use good practices for all of the example elements. We should include good security practices as well, but some of this will require the user to configure the ACL appropriately for their application and environment.

We may want to consider providing a template for an empty database as well, but we can revisit this after we have the main template done.

Note that the DXL Import project (#668) should not be based on this template - it is designed to run in a Java project instead.

@JoelProminic
Copy link
Contributor Author

JoelProminic commented Feb 28, 2020

I have a basic database created for this in Domino Designer. I will review this with @JustinProminic and send an official copy when this is read.

The database is here (requires Prominic database access):
Server: domino-1@dmytro.cloud (formerly domino-49.prominic.net)
Database: MoonshineExampleODP.nsf

@JustinProminic
Copy link

@JoelProminic please update this location above to reflect that the master database is now on @dpastov server.

@JustinProminic
Copy link

@dpastov will add an XPages example for us, and I will add other things I think are useful to the traditional forms and views. FYI @JoelProminic

@JoelProminic
Copy link
Contributor Author

JoelProminic commented Mar 5, 2020

I was able to test On Disk Projects on macOS Mojave with HCL Notes 11. Here is what I needed to do to get set up:

Allow passwordless access to the Notes API using these instructions.

Create (or update) ~/.m2/settings.xml. Update <notes-program to your HCL Notes installation directory. Update <notes-platform> to the directory where you intend to create an update site.

<?xml version="1.0"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <profiles>
        <profile>
            <id>nsfodp</id>
            <properties>
                <!-- for local operations, macOS example -->
                <notes-program>/Applications/HCL Notes.app/Contents/MacOS</notes-program>
                <notes-platform>file:///Users/-snip-/workspace/OnDiskProject/UpdateSite</notes-platform>

            </properties>
        </profile>
    </profiles>
    <activeProfiles>
        <activeProfile>nsfodp</activeProfile>
    </activeProfiles>
</settings>

Next, you need to build an update site. A user should only need to do this once per workstation, and we'll look to see if there is a better way to do this for Moonshine.

  1. Clone this project: https://github.com/OpenNTF/generate-domino-update-site
  2. Open the project in Terminal
  3. Run mvn install
  4. Run mvn org.openntf.p2:generate-domino-update-site:generateUpdateSite -Ddest="/Users/-snip-/workspace/OnDiskProject/UpdateSite, updating the path to match <notes-platform> above.

This command automatically detects the Notes installation. We'll probably want to manually compute the path from the Notes path configured for #667 using the -Dsrc parameter, but I need to find where the update site is stored in the installation

UPDATE 2020/04/01: At this point, you should be ready to create a project from the Moonshine branch build (see the instructions here). Otherwise, you can use the instructions below for the external On Disk Project demo.

Then, you should be able to run an example project like this:

  1. Clone this repository and open the "example" subdirectory
  2. Note that the database files are defined in nsfs/nsf-example/odp
  3. Open the project in Terminal
  4. Update the <versionRange for eclipse-plugin in pom.xml. See the example below
  5. Run "mvn install"
  6. Double-click the .nsf database in nsfs/nsf-example/target to open it in Notes

The <versionRange> to update is here:

      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <configuration>

          <pomDependencies>consider</pomDependencies>
          <dependency-resolution>
            <extraRequirements>
              <requirement>
                <type>eclipse-plugin</type>
                <id>com.ibm.notes.java.api.win32.linux</id>
                <versionRange>[11.0,11.1)</versionRange>

However, I found that I have to make a couple modifications for now to let it work. These instructions are for the example project from the repository above:

  • in nsfs/nsf-example/pom.xml, comment out the <updateSites> section for the nsfodp-maven-plugin
  • in pom.xml, comment out the modules besides nsfs/nsf-example
  <modules>
    <!--
    <module>bundles/org.openntf.nsfodp.example</module>
    <module>features/org.openntf.nsfodp.example.feature</module>
    <module>releng/org.openntf.nsfodp.example.updatesite</module>
    -->
    <module>nsfs/nsf-example</module>
    <!--
    <module>releng/example-dist</module>
    -->
  </modules>

@JoelProminic
Copy link
Contributor Author

Some things I want to change from the above instructions for use in Moonshine:

  • Remove the dependency on the global settings.xml. Find another way to provide notes-program and notes-platform based on configuration stored in Notes
  • Automate generation of the local site, and configure the path in the Domino settings page (see Detect IBM/HCL Notes Client for Domino Support #667)
  • Automatically detect the Notes version to automatically update versionRange
  • On Build & Run, automatically open the compiled database in Notes.
  • Support deploying the database to a remote server. The server and path should be configured in the project settings. This is a more advanced feature, so we may not include it in the initial implementation.

@JoelProminic
Copy link
Contributor Author

I experimented with exporting the test database from Domino.

To do this, I selected the database in Domino Designer, and opened File > Team Development > Set Up Source Control For This Application. When selecting a project path, keep in mind that the database subfolders will be created directly under the selected directory.

The resulting files look like files in nsfs/nsf-moonshine/odp/. I think we can use the rest of the files as a base template for the Maven wrapper.

The Database details are stored in AppProperties/database.properties:

<database xmlns='http://www.lotus.com/dxl' version='9.0' maintenanceversion='1.10'
 replicaid='-replicaid-' path='C:\Flex\workspace\odp\DominoExport\JoelTest1_MoonshineExampleODP.nsf'
 title='JoelTest1 - Moonshine Example On Disk Project' increasemaxfields='true'>
<databaseinfo dbid='-replicaid-' odsversion='52' diskspace='847872' percentused='90.8816425120773'
 numberofdocuments='7'>

I see most of these properties were stripped out in the official example:

<database xmlns="http://www.lotus.com/dxl" title="NSF ODP Tooling Example" increasemaxfields="true">
  <databaseinfo/>

Once we have a proper exported example, we'll need to go through and add insertion parameters so that the user can customize the name.

Some other changes:

  • make sure that the ACL is generic and has sensible defaults for a user trying to create a brand new database.

@JoelProminic
Copy link
Contributor Author

JoelProminic commented Mar 11, 2020

Here is a breakdown of the required files for the On Disk Project template. You'll see some extra files in @feather812002's template, but I think we can narrow it down to just these files (plus any additional files we need to add for #670 or #646)

File Notes
nsfs directory for the on disk projects (only one for now)
nsfs/<db-project-name>/ maven wrapper for the On Disk Project. This design can support multiple on disk projects, but I plan to only support one for now
nsfs/<db-project-name>/build.properties TODO: does this require any updates for the template?
nsfs/<db-project-name>/META-INF/MANIFEST.MF TODO: references the project name - needs more investigation
nsfs/<db-project-name>/odp DXL files representing the forms, views, etc. These could be edited directly if desired, but it would only be practical for very simple changes.
nsfs/<db-project-name>/pom.xml database-specific configuration
pom.xml general configuration.

I expect these actions for On Disk Projects:

@piotrzarzycki21
Copy link
Collaborator

piotrzarzycki21 commented Mar 12, 2020

I experimented with exporting the test database from Domino.

To do this, I selected the database in Domino Designer, and opened File > Team Development > Set Up Source Control For This Application. When selecting a project path, keep in mind that the database subfolders will be created directly under the selected directory.

The resulting files look like files in nsfs/nsf-moonshine/odp/. I think we can use the rest of the files as a base template for the Maven wrapper.

The Database details are stored in AppProperties/database.properties:

<database xmlns='http://www.lotus.com/dxl' version='9.0' maintenanceversion='1.10'
 replicaid='-replicaid-' path='C:\Flex\workspace\odp\DominoExport\JoelTest1_MoonshineExampleODP.nsf'
 title='JoelTest1 - Moonshine Example On Disk Project' increasemaxfields='true'>
<databaseinfo dbid='-replicaid-' odsversion='52' diskspace='847872' percentused='90.8816425120773'
 numberofdocuments='7'>

I see most of these properties were stripped out in the official example:

<database xmlns="http://www.lotus.com/dxl" title="NSF ODP Tooling Example" increasemaxfields="true">
  <databaseinfo/>

Once we have a proper exported example, we'll need to go through and add insertion parameters so that the user can customize the name.

Some other changes:

  • make sure that the ACL is generic and has sensible defaults for a user trying to create a brand new database.

@rat-moonshine Very first point is something which I envision to do by you. We need new type of project. That point in theory doesn't need code from Bing's work, it needs only his template files of Visual Editor - which is going to be copied and changed into On Disk Project. To be safe let's do that on branch as part of this issue unless you think to raise separate one.

@piotrzarzycki21
Copy link
Collaborator

@JoelProminic This two points are crucial and my next question may have been asked already on a call. I may have repeat them here just to make sure that my direction when I start coding is the right one.
Does On Disk Project gather two type of form creation ? I thought that Visual Editor (#646) will eventually have second option - Create Tabular interface or something like that later once On Disk Project will be ready to go.

@rat-moonshine
Copy link
Collaborator

rat-moonshine commented Mar 12, 2020

I spent quite some time today understanding/reading-through how Domino is in Moonshine IDE. I see I have been away for quite some time from related discussion on this (#646 , #668 , #669, #670 ), which became inevitable for me to understand what we already have and what we will need, before jumping into any implementation.

Reading through the discussion hopefully helped me to clear my understanding to an extent, which were basically filled with questions.

I also tried to give a test to Bing's branch-build (#646 ). Unfortunately, I couldn't able to complete to .nsf generation due to many an errors. We tried to address the errors with @feather812002 however no success so far.

I still have some questions related to Domino on Disk. Hopefully I shall able to discuss them with @JoelProminic tonight.

@rat-moonshine rat-moonshine self-assigned this Mar 12, 2020
@piotrzarzycki21
Copy link
Collaborator

I still have some questions related to Domino on Disk. Hopefully I shall able to discuss them with @JoelProminic tonight.

Maybe ask them here, since I will be working on Tabular interface so I would not be completely blind with any of your issue.

@rat-moonshine
Copy link
Collaborator

rat-moonshine commented Mar 12, 2020

I talked to @JoelProminic in length on different queries those I have, mostly to confirm my understanding from the issue readings, and some other queries on how-to things.

As I understood correctly, we'll probably going to need an exclusive project-type for Domino On Disk having ability of visual-editor (for #646 ) and tabular-forms (#670 ). Both the form types will generate DXLs.

Being said 'exclusive', we'll probably create this as a new project-type (separated from creating as a "Visual Editor Project" type option) and show as a new type-option in home screen of Moonshine-IDE. Since the said new project type will also require visual-editor abilities, it will be a visual-editor type of project but with different project extension preferably (i.e. ondiskproj.as3proj). The project should generate taking Bing's nsfs file (those @JoelProminic mentioned at here).

@piotrzarzycki21 do you see the above assumption fits your idea of having a separate type of project? Or, you have other suggestion which I might failed to understand.

@JoelProminic
Copy link
Contributor Author

I discussed the On Disk Project with @rat-moonshine today, and I wanted to document some of my thoughts. Here is a mockup of how the project structure might look. Note that this is not finallized - it is more to present the current ideas I have for feedback. Soem clarifications

image

With this design, the user will update the Form Builder and Visual Editor forms from the src directory only. I split this into multiple subdirectories for now, but we could also distinguish the files with different extensions (i.e. .dfb and .dve, respectively). Double-clicking FormBuilderExample.form will open the Form Builder interface (#670), while VisualEditorExample.form will open the Visual Editor interface.

The intermediate files for Form Builder and Visual Editor will be stored in formbuilder-src and visualeditor-src, respectively. These directories should be hidden so that they are not edited directly.

The actual form DXL needs to be placed in nsfs/nsf-moonnshine/odp/Forms so that it can be imported to a Notes Database. Also note that the user can then edit the form in Domino Designer and export it back to this file, so these files will not necessarily match up with the intermediate XML files. The .form files can be edited directly, but this will only be practical for simple changes or advanced users.

TemplateExample.form is a form included in the template, so it has neither form type. It can only be edited as DXL or with Domino Designer.

Alternatively, we could allow the user to double-click on the forms in nsfs/nsf-moonnshine/odp/Forms and open the associated editor. @feather812002's implemenation in #646 works like this.

  • FormBuilderExample.form - the FormBuilder form
  • TemplateExample.form - Text editor
  • VisualEditorExample.form - Visual Editor

However, this has some problems

  • It ignores the problem that the .form file may have local changes that will be lost
  • I think it may be confusing for the user if the .form files open in different ways in the same directory.

@rat-moonshine
Copy link
Collaborator

The actual form DXL needs to be placed in nsfs/nsf-moonnshine/odp/Forms so that it can be imported to a Notes Database. Also note that the user can then edit the form in Domino Designer and export it back to this file, so these files will not necessarily match up with the intermediate XML files. The .form files can be edited directly, but this will only be practical for simple changes or advanced users.

However, this has some problems

  • It ignores the problem that the .form file may have local changes that will be lost
  • I think it may be confusing for the user if the .form files open in different ways in the same directory.

Yes, I see the advantage and related problems to this design.

Generally, double-clicking an exported file (i.e. nsfs/nsf-moonnshine/odp/Forms/VisualEditorExample.form) locates its corresponding code/XML file inside 'visualeditor-src' and opens as mockup-editor. Having the .form file edited/updated externally means the file inside 'visualeditor-src' will have sync issue.

Perhaps, in that case a file timestamp checks between both files may help (as we did for #633 ). Now we need to think what action(s) we want to offer to the user if we found a sync problem.

Overall, the design looks pretty clean and fantastic.

@piotrzarzycki21
Copy link
Collaborator

The actual form DXL needs to be placed in nsfs/nsf-moonnshine/odp/Forms so that it can be imported to a Notes Database.

I don't understand this one - if in the end whatever we create in src folder has to be in /odp/Forms - what is the point of having src folder at all? Shouldn't Forms be an src folder actually ?

@piotrzarzycki21
Copy link
Collaborator

@rat-moonshine Your work would be reach the point where user is able to open stuff by double clicking - I will handle part where user see real features for "Form Builder" (Tabular interface).

@rat-moonshine
Copy link
Collaborator

The actual form DXL needs to be placed in nsfs/nsf-moonnshine/odp/Forms so that it can be imported to a Notes Database.

I don't understand this one - if in the end whatever we create in src folder has to be in /odp/Forms - what is the point of having src folder at all? Shouldn't Forms be an src folder actually ?

/odp/Forms can be updated by external sources (or say, by a database). If we manage same file also for Moonshine visual-editor changes, those changes can lost. Thus, having a separate intermediate file managing the Moonshine visual-editor changes (src/main/formbuilder/..) may save this situation.

However, there are certain update-sync problem can arise which Joel already pointed in his comment.

rat-moonshine added a commit that referenced this issue Mar 13, 2020
- Enable project open from recent list
- Source icon visibility to sidebar (working only in open/import case for now)
(reference #669)
@rat-moonshine
Copy link
Collaborator

rat-moonshine commented Mar 13, 2020

(I forgot to reference some commits which we can find here 2509c6f)

I see @joshtynjala has introduced some flexible and modular system to introduce new project types to Moonshine (which he used again Haxe, Groovy etc.). I followed the path to introduce the new On Disk Project type, as well.

One thing we need to decide, how we going to introduce @feather812002 's implementation (#646 ) to this new project source. I'm not yet came to the steps where we shall be needing to look into #646 functional implementation, but mostly dealing with creating new project container to have it avail in Moonshine along with its regular tasks like create/import, menu, configuration-updates etc.

So far here are the updates until now and upcoming tasks:

@JoelProminic
Copy link
Contributor Author

Sorry @rat-moonshine, I meant to give you the updated template files before you started working on the templates.

Here are the files for the Maven wrapper:
2020_03_13__ODP_Wrapper.zip

My updates are:

  • Removed releng - this is an unnecessary update site module from the original demo (I actually never got this working properly on my side)
  • Updates the two pom.xml files to remove references to releng
  • removed nsfs/nsf-moonshine/odp - this will be populated with the second zip below

Be careful not to include ./target or ./nsfs/nsf-moonshine/target when you create the template, since these will be populated by Moven.

Here is an initial version of the example project. These files should be placed in the above framework nsfs/nsf-moonshine/odp, so that nsfs/nsf-moonshine/odp/Forms has the example form.
2020_03_13__Example_ODP.zip

Here are some additional notes from my side, so that I can remember how I generated the Example ODP later:

To generate the example On Disk Project from an NSF, I ran File > Team Development > Set Up Source Control For This Application from the Domino Designer menu.

The only file I saw that required cleanup was odp/AppProperties/database.properties. I replaced the database, databaseinfo, and acl sections like this. Some notes:

  • "adminserver" is a Prominic server currently (domino-49.prominic.net). I'm not sure how to handle this in a generic way, so I'll revisit it up later (the server will not be exposed to general users). Otherwise, I stripped all Prominic references out of the ACL:

  • The title section in <database> and the $TITLE item below should eventually be updated to an insertion parameter to use the template name. We'll also need to update the .nsf name, but this is currently built with a version number using the Maven properties

    deletedocs='true'/> deletedocs='true'/> deletedocs='true'/>

I also updated the title at the bottom of odp/AppProperties/database.properties:

<item name='$TITLE'><text>Moonshine Example On Disk Project</text></item></note>

I got this error when testing the template:

Compiling Java source
- Compiling no classes
Uninstalling bundles

!ENTRY org.eclipse.osgi 4 0 2020-03-13 17:06:40.064
!MESSAGE Application error
!STACK 1
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: no source files

It seemed that a Maven task was failing because there were no Java files to compile. I need to review this more, but to get the demo working I added a placeholder Java file in odp/Code/Java/demo/Placeholder.java

package demo;

public class Placeholder {
    /*
     * This is an example class compiled with the On Disk Project.
     * Without at least one class in odp/Code/Java, Maven throws an error.
     * This may be removed later if it is no longer necessary
     */

}

@piotrzarzycki21
Copy link
Collaborator

I'm not sure what do you mean here - what are those files ?

rat-moonshine added a commit that referenced this issue Mar 16, 2020
- OnDiskProjectVO, importer/exporter, build-options cleanup
(reference #669)
@JoelProminic
Copy link
Contributor Author

I was discussing this issue with @dpastov, and I noticed that it was not really clear from the comments how to setup an On Disk Project using the Moonshine branch build.

So far, we have implemented a very basic On Disk Project template for Moonshine, but it is currenlty only in a test branch.
Source: https://github.com/prominic/Moonshine-IDE/tree/features/issue_669_domino_on_disk
Automated Build: https://moonshine-ide.com/downloads/releases/dev/branches/features/issue_669_domino_on_disk/macos (Use Moonshine_Signed_NonSandbox.pkg)

@rat-moonshine or @piotrzarzycki21 should be able to handle the Moonshine Flex updates, but if you need to build the source yourself, there are instructions here: https://github.com/prominic/Moonshine-IDE/wiki/Running-Moonshine-instance-through-Moonshine-IDE

To setup your environment, follow the steps from this comment up through generating the update site. The example project will be generated from Moonshine instead.

To create a new On Disk Project:

  1. Install and run the test Moonshine build from the link above
  2. File > New > Domino On Disk Project
  3. Choose a project name and directory
  4. Right-click on the project and open the Settings
  5. In the Maven Build tab, change the Build Action to "install". Save and close the settings
  6. Project > Build with Apache Maven
  7. Right-click on the project and choose "Open in Finder"
  8. The generated database will be in nsfs/nsf-moonshine/target/. You can open and view the target in Moonshine

@JoelProminic
Copy link
Contributor Author

Revisiting the project layout here, I see that the intermediate XML directory for Form Builder (formbuilder-src) is probably not necessary. We do not have any specific content that needs to be placed in the src folder, so we can store the intermediate XML in the .dfb file.

Similarly, we could probably remove visualeditor-src, but we may want to keep this for consistency with the other Visual Editor projects.

@rat-moonshine
Copy link
Collaborator

Revisiting the project layout here, I see that the intermediate XML directory for Form Builder (formbuilder-src) is probably not necessary. We do not have any specific content that needs to be placed in the src folder, so we can store the intermediate XML in the .dfb file.

Similarly, we could probably remove visualeditor-src, but we may want to keep this for consistency with the other Visual Editor projects.

On a second thought, I'm thinking on some other points in term of consistency. I think I want to discuss with you broadly @JoelProminic before writing every pieces now.

@JoelProminic
Copy link
Contributor Author

Currently, the default Maven action for the On Disk Project template is "war:exploded". I would like to change this to "clean install" instead, since this is what is normally needed to generate the NSF. I have been using "install" primarily in my testing, but I have seen some cases where the DXL file updates are not detected and the NSF doesn't update.

@rat-moonshine
Copy link
Collaborator

On addition to above, we noticed that files-tree last-opened-state not properly repopulate when Moonshine restarts, in case of OnDisk project. Also, any last opened files from the project do not auto-opens, this needs to be fixed. I shall work on this shortly.

@rat-moonshine
Copy link
Collaborator

Currently, the default Maven action for the On Disk Project template is "war:exploded". I would like to change this to "clean install" instead, since this is what is normally needed to generate the NSF. I have been using "install" primarily in my testing, but I have seen some cases where the DXL file updates are not detected and the NSF doesn't update.

clean install now default for new OnDisk projects.

rat-moonshine added a commit that referenced this issue May 12, 2020
- Fixed closing all opened editors during project close
(reference #669)
rat-moonshine added a commit that referenced this issue May 12, 2020
- Updated wrapper component property to return project path
(reference #669)
@rat-moonshine
Copy link
Collaborator

rat-moonshine commented May 12, 2020

  • Fixed tree-state-history population issue
  • Fixed re-opening of previously opened editor during Moonshine restart
  • Fixed not closing of project's editors when project being closed

@piotrzarzycki21
Copy link
Collaborator

Moving this for next release. No announcement in CHANGELOG

@rat-moonshine
Copy link
Collaborator

Moving this for next release. No announcement in CHANGELOG

We already have the project type included in the 'master' so it doesn't make sense to push back in another release. We also have this updated into CHANGELOG.

@JoelProminic do you see any reason for holding and we may need to comment-out the type population in v3.0 (?)

@piotrzarzycki21
Copy link
Collaborator

piotrzarzycki21 commented Oct 5, 2020

@rat-moonshine hide it. We are waiting for tests over 2 months - I don't think that something is going to change for now, so I don't see any reason to wait even a single day.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement test ready Feature/bug ready for testing
Projects
Moonshine-IDE - Bug Fixing
  
Awaiting triage
Development

No branches or pull requests

4 participants