Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project sync support for reference to files outside of project folder #1399

Closed
makandre opened this issue Dec 5, 2019 · 13 comments
Closed

Project sync support for reference to files outside of project folder #1399

makandre opened this issue Dec 5, 2019 · 13 comments

Comments

@makandre
Copy link
Contributor

@makandre makandre commented Dec 5, 2019

Codewind version:
OS:

Che version:
IDE extension version:
IDE version:
Kubernetes cluster:

Description of the enhancement:

I would like to be able to create/define in my project, references to files that exist outside of the project folder (think of it like a symlink, but cross-platform). Conceptually, something like this:

myProj/
    fileA --> /someAbsolutePath/fileX
    fileB
    fileC

When the contents of fileX changes, it should be synced into PFE as fileA

Use Case:

fileX could contain some (potentially sensitive) configuration information required for PFE to build and run the application, but I do not want to keep this file inside the project folder because it may get accidentally checked into a repo. Also the configuration would be retained if I were to delete/re-clone my project etc.

Proposed solution:

See comments

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Dec 5, 2019

I discussed with @jgwest on a possible solution.

The first part would involve adding a new optional property to the payload filewatcherd gets from the /api/v1/projects/watchlist API:

{
   "projects":[
      {
         "projectId": "real-project-id", /* project id of an existing project */
         "pathToMonitor":"/codewind-workspace/myProj", /* the actual real project path to monitor, since this is an existing project it will already have one */

         "refPaths" : [ <-- new optional property
            {
               "from" : "/someAbsolutePath/fileX",  /* absolute path to file on local machine */
               "to" : "fileA",  /* relative to root of project, inside container */
            }
         ],

         "ignoredPaths":[
            "/target/*"
         ],
         "ignoredFilenames":[
            "*.class"
         ], 
     "projectWatchStateId" : "[...]",
      },
   ]
}

The refPaths defines the additional referenced files to watch for the project; this could possibly be defined by user in .cw-settings

Then, whenever a file change occurs, filewatcherd would call:

cwctl project sync -p /codewind-workspace/myProj -i real-project-id -t timestamp -r "representation of the refPaths JSON"

The new optional -r flag would instruct cwctl to sync the additional files based on the from and to of each entry.

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Dec 16, 2019

I have the first part, /api/v1/projects/watchlist API with the extra refPaths attribute prototyped in #1535 (where refPaths is defined in .cw-settings.)

It turns out it was more involved than just the API however, because the watchlist also gets sent over to filewatcherd over WebSocket. The PR also takes care of this.

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Dec 19, 2019

The syncing of referenced files is implemented in eclipse/codewind-installer#304

The sync command was reading .cw-settings already, so actually there's no need for filewatcherd to pass the json to the command as in the original design; this should simplify the logic in filewatcherd. Now it only has to pass --refPaths true if a change to a referenced file occurs.

cc @jgwest

@jgwest jgwest removed the tech-topic label Jan 13, 2020
@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Jan 14, 2020

Summary of design meeting with @elsony @jgwest @sghung on Jan 13:

  • refPaths config should not be in .cw-settings, will put in separate .cw-refpaths.json file, contents will be:
{
    "refPaths": [
        "from": "absolute *or* path relative to project root locally",
        "to": "relative to root of project inside container"
    ]
}
  • refPaths takes precedence over physical file in the project, this means:

    • cwctl sync should always process refPaths to ensure the precedence is maintained (therefore no need for a separate --refPaths flag on the command)
    • the regular sync portion of the command should ignore files in the project that is included in a refPath
  • if a referenced could not be read (does not exist etc), cwctl sync should log error and additionally return error code

@micgibso

This comment has been minimized.

Copy link

@micgibso micgibso commented Feb 4, 2020

@jgwest Hi, what docs might be required for 0.9.0?

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Feb 4, 2020

Hi @micgibso,

We need doc about how users can define references in the new .cw-refpaths.json file and what the behaviour will be.

Additionally, we should also add a mention to this section (the caution part) to say user can use references to avoid putting env.properties into their project

@micgibso

This comment has been minimized.

Copy link

@micgibso micgibso commented Feb 5, 2020

Seems like a Developing projects topic but I'm not sure where this is going to go in our new format of documentation. Some guidance and input appreciated.

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Feb 5, 2020

This is the new Developing project topic right?

image

I agree it can go under there.

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Feb 6, 2020

@micgibso Here's my attempt at a first draft:


It is possible to define references to files that reside outside of a project:

  1. Create a file called .cw-refpaths.json in your project
  2. Define the references as follows:
{
    "refPaths": [
        {
            "from": "",
            "to": ""
        }
    ]
}

refPaths is an array of references. A reference's from property is an absolute or relative path to the file you are referencing (note: you cannot reference directories). The reference's to property is a path relative to the root of the project.

Codewind will copy the contents of the file that is being referenced, and place it in the file specified by the to path when building and running the project.

@micgibso micgibso self-assigned this Feb 10, 2020
@micgibso

This comment has been minimized.

Copy link

@micgibso micgibso commented Feb 10, 2020

@makandre Thanks, have drafted new topic and added it to the toc, please review PR https://github.com/eclipse/codewind-docs/pull/350/files, thx!

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Feb 11, 2020

Docs was merged.

/pipeline verify

@malincoln

This comment has been minimized.

Copy link

@malincoln malincoln commented Feb 12, 2020

@makandre pls verify and close. Thanks

@makandre

This comment has been minimized.

Copy link
Contributor Author

@makandre makandre commented Feb 12, 2020

Verified on 0.9

@makandre makandre closed this Feb 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.