PG Simplification: Merging Legacy & Simple

[Daandelange]

Hello :)

After some discussions with @ofZach and others about PG simplification, I'm making a plan of attack to merge both the PG Simple and the PG Legacy.

This issue can be seen as a continuation of openframeworks/openFrameworks#3941 where @ofTheo states that we have to be able to generate exemples and update projects in order to kill the PG Legacy. Another great feature is to be able to generate project files for any platform.
So this thread could be used to clarify & discuss directions and new features for a unified PG.

I saw @arturoc 's PG refactoring which (temporarily) puts all common code in a cross-platform ofxAddon. This is meant to be put somewhere else later.
The ProjectGenerator, the ProjectGenerator CommandLine and other users will then be able to use this common code. (ex: @bakercp plans to use this common code in his ofSketch app).

Zach notes that there's no need to use OF or that we need to simplify the UI (which he has already begun).

For the record, @kritzikratzi coded a command line tool to add, remove and update addons to an of project which can be useful too.

Btw, there's currently a PG submodule while having the PG core in the OF addons folder (ofxProjectGenerator)... having the PG code in 2 different repos makes no sense to me...

---

So here's what I suggest doing:

• Add pages to the PG:
  • Create page: not much to do. (copy of PG Simple)
  • Update Page (Adapt PGLegacy code)
    • Usage: Load project, Settings are imported/detected into UI and can then be altered / updated.
    • Add Generate Examples page (adapt code from PGLegacy)

Maybe all those pages can share a “target” bar where you can tick the desired targets.

And I'm wondering, what's the best way to achieve this? (regarding versioning)

• Create a new folder called something like "ProjectGenerator Unified" or
• Alter the PG Simple

[kylemcdonald]

if you can do this by altering pg simple, that'd be great -- but you'd have to make sure to keep pg simple functional whenever you push to master (since people are using it). if you want to stay up to date with master but not worry about breaking the pg, create a "unified" folder. in theory, you should be able to edit pg simple in a branch, but in practice not as many people will test your code unless it's in master.

not sure if zach already mentioned this, but there were some ideas around what the PG could look/act like here:

https://docs.google.com/document/d/1GjgjQZCHTcOD3cT9OFSl0AZ1nhfYwwvxTl5L4OJE6qA/edit#heading=h.w46dgd3qcugf

[bilderbuchi]

yeah, it might be easier to create a third PG temporarily (and make it so it can easily be renamed, so that we can nuke the other two as soon as it works).
I am only concerned that this may become another, third, semi-permanent PG if the project does not progress as planned.

[arturoc]

please, don't push directly to master, keep everything on a branch and send a PR whenever there's changes you want merged.

[bilderbuchi]

aside of that, +1 to the OP, I think it's an accurate depiction of the current state. one minor thing: pay attention that kritzikratzi's tool is for xcode only.
Also one thing I found missing: the unified PG should be able to be driven from the command line, without GUI, too. This way we can use it on servers, during CI, etc, to generate the example projects, etc.

[ofZach]

to add to what @bilderbuchi said, the original PG has command line (linux only) for making the packages. This is also a more full featured command line approach that I think is interesting:

https://github.com/openframeworks/projectGenerator/tree/master/commandLine/src

on OSX it's a little wonky since it's a .app but the exe is kind of deeper in the package.

personally I feel like the best way to start is with the PG simple but restructure it abit so it's easier to add new pages. I sent you a start of that work I did earlier.

[ofTheo]

Yeah actually I would vote with simply adding a 'develop' mode to PG simple. Within that it just lets you generate the examples for the different platforms.

[Daandelange]

Ok, that sounds good. The Roadmap document is certainly useful.
For the moment I feel more confortable working on a "unified PG" UI using the ofxProjectGenerator common code (as most as possible). The command line & batch part is still a little bit vague to me.

[ofZach]

just a quick note that I've gotten the command line for osx up and running and it seems to work well / could be tested:

https://github.com/openframeworks/projectGenerator/tree/master/commandLine

the nice thing if we did make the command line really slick is that we could have any number of front ends to it and it will also work for making packages, etc.

the printouts from the PG need some love, but it's still pretty cool to be able to do recursive updating of projects from the command line....

[ofZach]

here's a screenshot of a gui I cooked up in a day with electron:

definitely command line is the way to go ! this was fast, I think adding options for upgrading a project (which exist in the command line tool) will be easy to add to this....

[ofZach]

if you want to try this, here's a link:

https://www.dropbox.com/s/f8k6kz7rsppo3hg/pg.zip?dl=0

note that the OF path will need to be set as well as the project path and it doesn't have smart defaults yet. there is a settings.json file deep in the app if you explore package contents (there you can also find the node / electron code for this front end)

[arturoc]

what's electron? :)

[bakercp]

http://electron.atom.io/

[bilderbuchi]

Engine behind atom, the text editor. Node/WebKit afaik

[ofZach]

(and what i posted is osx only at this point... but electron is cross platform as well as the pg command line...)

electron is similar to node webkit, it's a tool for writing native apps that use node / javascript as the front end. I think they are all using chrome under the hood.

[ofZach]

update now + update recursive :)

just tested and I was able to generate all the examples :)

feedback (like seeing what the command line tool does) would be useful and if something takes a while it's useful to know...

zip is here:

https://www.dropbox.com/s/pea7tis33grpp5a/pg_w_update.zip?dl=0

note OF path is moved to settings now. you have to set that path before you do anything :) it's kind of hardcoded for me now.

[ofZach]

drag and drop coming shortly... I infinitely <3 guis made via web

[ofZach]

here is a version that should just flat out work in the root of your OF directory.

Internally it stores a relative path but it displays a absolute path to make things easier to read (it looked odd to see . as the path.. also internally it's adding ../../../ since the "app" for the pg is embedded in the resources folder of the package):

https://www.dropbox.com/s/e8dl06oua5w2qde/pg_V3.zip?dl=0

[ofZach]

version that has copy and paste working (needed to add a menu)

https://www.dropbox.com/s/xmnxesggodqmv23/pg_v4.zip?dl=0

[kylemcdonald]

cool! copy/paste is very nice to have.

one thing that's unclear to me is whether the goal of this new GUI is to follow the spec from the roadmap, or if it's headed in a new direction?

and if it's following the spec, should we get someone to wrangle the CSS to make it look decent? or is that something you wanted to handle @ofZach and @Daandelange?

[ofZach]

To me this is "proof of concept" ie does an html front end make sense. I think it could be made to look more like the roadmap / styles etc. just want to get a sense (without putting to much design work in) if this seemed right.

[ofZach]

Also theos comment makes a strong argument for why just adding update functionality would be good enough for an update to the pg. the command line tool has had that, this sort of exposes that.....

(the most important goal I think being merging all the PG work into one place, something that simple enough to make a new project but also strong enough to generate tons of projects on different platforms and do releases, etc)

I think the design on the roadmap sort of has one panel that morphs between generating and updating based on what people do, I am not sure if having a more separate distinction (ie, seperate update and generate pages) makes sense. The backend (command line tool) is generally there save for addons in different locations which would require some smarts on the command line side.

[ofZach]

@arturoc I'm wondering if the command line tool can also work for packaging releases? (Right now, I think you are compiling the original pg for the command line for this but I'm not sure if there are much differences now)

[arturoc]

it should, both legacy and simple are exactly the same now after i merged everything in ofxProjectGenerator. the only difference is the gui and the main in the legacy version which has some options to run from the command line in linux. if you keep the command line syntax or something similar i can adapt the packaging scripts to use this one.

it would be nice to optionally use the old method to find the OF install by having a config file or perhaps to find the place where the exe lives by using ofFilePath::getExePath() so we can run the command line tool from anywhere. i usually put it in /usr/local/bin so from the console in any folder i can do:

projectGenerator project

and it'll create a new project wherever i am. if we use getExePath one could create a softlink to the tool to /usr/local/bin and will still work the same way

[ofZach]

makes sense, right now the command line has the following options:

-r, --recursive                                 update recursively (applies
                                                only to update)
-h, --help
-c, --create                                    create a project file if it
                                                doesn't exist
-u, --update                                    update a project file if it
                                                does exist
-x"platform list", --platforms="platform list"  platform list
-a"addons list", --addons="addons list"         addons list
-o"OF path", --ofPath="OF path"                 openframeworks path
-p"project path", --projectPath="project path"  project path
-v, --verbose                                   run verbose
-d, --dryrun                                    don't change files

I wonder if it makes sense to add a flag to use / look for a config file instead of command line options? Or, if there's some way of using environment variables or other tricks that would allow you to skip some of these options? ie, PG_OF_PATH... I've always been a little worried about the dotfile being hidden and all, but that's just because I don't have a ton of experience configuring apps with those.

(alternatively no flag for the config file but that command line parameters overwrite the config options?)

also, I'm not sure what we need "ofFilePath::getExePath()" for, is that so it knows where OF is located? In my command line app we specify where OF is located and where the folder to be created / updated is by path.

if you can give the command line app a try and give me feedback about it, I'd love to push for this to be the main interface and that we drop both the older and simpler PGs and move to the electron.io / html based front end for user facing functionality as well as guide about how to install this for actual command line usage, which seems great.

[arturoc]

i was thinking about detecting the exe path so if you use the PG from the command line you don't have to constantly specify the OF path, this is not for the package generator but for users who use it from the command line. i never use the PG simple graphics interface and instead i have the projectGenerator binary in /usr/local/bin and usually do from the command line:

projectGenerator apps/myProject/newApp

or

cd apps/myProject
projectGenerator newApp

which is really handy. without a config file or detecting the exe you would need to do:

projectGenerator -o /home/arturo/Code/openFrameworks apps/myProject/newApp

which is a pain

[arturoc]

an env variable which people can put on their .profile or whatever it is in windows could work too

[arturoc]

btw, this are the options that i have right now:

OF Project Generator Usage:
projectGenerator [options] [pathToExample]
Options:
--osx: generate osx project files
--win_cb: generate windows codeblocks project files
--vs: generate windows visual studio project files
--linux: generate linux project files
--linux64: generate linux 64bits project files
--linuxarmv6l: generate linux armv6l project files
--linuxarmv7l: generate linux armv7l project files
--ios: generate iOS project files
--allplatforms: generate project files for all platforms
--allexamples: generate project files for all examples. [pathToExample] is not needed in this case.

default: create project files for current platform for selected path
running over existing example updates project files for selected platforms
without parameters, shows gui

i guess your --recursive could work for --allexamples right?

[ofZach]

yes you could use update + recursive but currently there's no "all platforms" or "all examples" options -- you have to specify the platforms by name, they can be comma seperated:

/commandLinePG -u -r -x"osx,win_cb,vs,linux,linux64,linuxarmv6l,linuxarmv7l,ios" -o"/Users/zachlieberman/Documents/openframeworks/git/openFrameworks" -p"/Users/zachlieberman/Documents/openframeworks/git/openFrameworks/examples"

the paths can be absolute or relative. I know it's a mouthfull but maybe we can come up with convenience options like allplatforms and allexamples.

I think I understand you about the exePath now, it seems really useful to be able to create projects like that. To clarify:

a) you would use some sort of .dotfile or environment variable to specify the location of OF
b) when you run the command line app, you'd look at where the app is being run (ie the current folder) to get the project path

does the target platform come from the .dotfile as well?

[arturoc]

specifying the platforms like that is not that bad i guess, it's not used that much, not even by the packaging script, but having an option to create all the example projects for a specific platform would be useful for people using OF from git for example.

with the dotfile i only specify the path, the target platform if not specified is by default the one in which the PG is being run or actually in which it was built, you can query it in runtime with ofGetTargetPlatform()

a) is correct
b) yes, if we know where the exe is we can look for it's resources in exe_path/data and there check the OF root in the xml config file as it's doing right now. but i think that solution is too hacky better to just use a config file which could have other options or just an env variable which specifies the path to OF, like OF_ROOT

[ofZach]

@arturoc see #8 -- this seems like a useful tool and it's fun to make projects this way now :)

[arturoc]

super nice, i think this already has the same features as the legacy one in the command line. should we merge it?

[kylemcdonald]

i've used this for a few projects now and haven't had any issues, but it would be nice to do a bit of "cleanup" before shipping it.

one essential thing:

• display confirmation when successfully finished.

some other things that are more aesthetic suggestions:

• the "pg" headline at the top of the window is redundant and can be removed completely
• we should switch the icon from the default electron app to an OF one
• make the window height smaller to fit the contents better
• capitalize the name of the app to "Project Generator", for the file and for the title of the window
• capitalize first letter of a lot of thing ("Create", "Update", "Settings", "Project name:", "Project path:", "Platforms:", "Addons:", "Generate")
• "set project location", "set update location", "set OF root" could all just be "Open", and the dialogs should just say "Select parent of project folder", "Select project folder", "Select openFrameworks folder"
• in "settings" the field description should be "openFrameworks root"
• "project path" should have a default value
• some corrections & suggested text below for the "warning" box in "update" below. even better if the "recursive" warning only pops up when you select it.
• "create" panel should have description text. suggestion below.
• it shouldn't be possible to drag&drop onto the window. right now, if you do it just makes the window go white.
• i wish the buttons were styled differently but this is the most arbitrary bit of personal preference in the whole list :)

Update text:

This will use the "src" folder and "addons.make" file in your project folder to generate a new project file. It will overwrite old project files. Use "recursive" carefully, or it will create a mess of your all data.

Create text:

This will create a new project file for your target platform(s), and a skeleton openFrameworks app in the "src" folder, including any addon(s) you select.

[ofZach]

@kylemcdonald thanks for the good suggestions. the nice thing is that's it's all css / html so should be easy to change (for the most part). it's just using jquery + boostrap internally (the trickiest thing is the multiselect component...). If you poke around the app package you can even mess with the css yourself.

will post up a cleaner version shortly.

[kylemcdonald]

@ofZach cool! i'm also happy to poke around once the code is merged

[Daandelange]

Wow, lots of new stuff :)
Nice to see the new Electron-based interface with HTML functionalities. I can't wait to test it out.
I'm very familiar with web languages so I can help with that even though it already looks good! 👍

Also, I think the update could be more comprehensible by making an Import settings or Import project button which would simply load the settings from a project into the UI, then you can just hit "Generate" and your project gets updated. What do you think of this ?

Edit: this would match 2. in the PG Philosophy of the Roadmap.

[ofZach]

I sat with @kylemcdonald yesterday where we looked at this for a while (I talked through some of the general UI issues I had with the road map design) and ended up with seperate pages for generate and update but now I'm thinking that the best solution might be:

generate / update multiple / options

where generate has a new / update sub-panel that switches between what you want to do and if the you drag in an existing folder / etc but that addons and platform choice are there:

we also talked about an advanced toggle in options that would allow you to choose platform and / or use update multiple.

I guess in generate->update you'd have a path and a button for choosing the path. this could say "import"

[Daandelange]

I think that's still confusing. What about removing the new / generate sub-panel with just an import button which then switches it to an "update" mode which is like generate->new but with preloaded (adjustable) settings ?
Something around the "generate" button (or the path) could indicate if it's gonna be an update instead of a new project.

[ofZach]

honestly, I feel like we are bike-shedding here. Maybe keeping it simple for now (generate / update / options as in the mockup of the UI I made) might be the easiest, then if you want to try import on a later release, go for it.

[ofZach]

(also, I feel like with this UI stuff, maybe building it is better then talking about...)

[ofZach]

on second hand, you might be right about the import button :) I just can't see it so easily as I have update and generate really seperate in my mind, and after a long talk with @kylemcdonald we even decided to separate them. I'm happy if you want to give a shot at the gui, I feel like I've gotten the command line tool to a pretty good place and did a sort of P.O.C. on the gui so it should be pretty easy to make something cool. I think doing import should be as simple as parsing the addons.make file and the path.

[Daandelange]

Yes, that's what I'm thinking about (parsing config.make & addons.make)
and I'd love to give it a try.
And yes it's time to start coding :)
When do you plan to push the new PG with the Electron-based UI ?

On 17 August 2015 at 15:28, ofZach notifications@github.com wrote:

on second hand, you might be right about the import button :) I just can't
see it so easily as I have update and generate really seperate in my mind,
and after a long talk with @kylemcdonald https://github.com/kylemcdonald
we even decided to separate them. I'm happy if you want to give a shot at
the gui, I feel like I've gotten the command line tool to a pretty good
place and did a sort of P.O.C. on the gui so it should be pretty easy to
make something cool. I think doing import should be as simple as parsing
the addons.make file and the path.

—
Reply to this email directly or view it on GitHub
#5 (comment)
.

[ofZach]

0.9.... time is ticking :)

[kylemcdonald]

i can see generate / update as separate, but the mockup you posted @ofZach looks like it's generate>new / generate>update / update multiple (three options instead of two).

but yes, just building something that does what we want first is best -- then we can simplify and combine things that are not ambiguous (if we have time). if @ofZach is still aiming for PG by this friday, ofBook by next friday, there might not be time for making this more elegant before the release, but at least we can get it working!

[ofZach]

@kylemcdonald actually I'm suggesting 2 options, but where there's a sort of sub-choice. @Daandelange is working on merging all these things now, I've gotten the command line tool to a pretty good place in terms of clean feedback and features that this would need so it should be pretty straight forward to get this together.

[ofTheo]

what about simply. Generate Project / Update Projects If you select Update Projects you can do one or multiple?

[ofZach]

@ofTheo that's kind of how it is now (in the example I posted), the idea behind some of these other suggestions is that a common operation is to change the addons in a current project (ie, update w/ addon selection) for this, it kind of differs from recursive updates, where you would use the addon.make file in each project like when we generate the example projects. I think the command line tool is pretty great (once you get the env variable set and an alias going, it's so nice to use) but @Daandelange is working on the gui now and trying to come up with a reasonable approach that's clear. We also want to avoid making it easy for people to do batch updating of projects since I think that you could really muck things up if you don't know what you are doing :)

[Daandelange]

In the UI I'm making there's a (re)Generate section which does both updating & generating (single) projects. (I really see these merged in a dynamic interface)
In the settings tab you define the target(s) and OF_path. (these are stored for next launch)
Another "Advanced" tab lets you do recursive (batch) updating projects with the needed warning messages & so on.

So it's a kind-of combination of the discussions here, with @ofZach and ideas from the roadmap.
( I'll post screenshots soon )

[Daandelange]

Here's 2 screenshots of the current setup; this might make it more comprehensive.

[kylemcdonald]

aha, this explains a lot! so now the "(re)Generate" is "create/update" and "Advanced" is "update multiple".

this makes more sense, disambiguates all drag&drop behavior, and puts the dangerous stuff in one tab.

i'd love to take a second pass at some of the title/copyediting if it's ok.

some more thoughts:

• the OF logo looks cool, but i don't think the words "PROJECT GENERATOR" should appear apart from the window title. maybe if the OF logo is smaller and at the top right corner? it shouldn't take up any vertical space. the "Project" title also feels unnecessary.
• the "you are missing the following addons" warning is great. but i'm undecided about the best behavior:
  • is this an error, and it will not generate a project file? (probably the best behavior)
  • is this a warning, and it automatically removes those addons? (dangerous)
• the warning about replacing the project file could be moved up to just beneath the "path" field, to match the addon warning. or the addon warning could be moved to the bottom to match the project replace warning.
• alternatively, the project file could just be renamed from projectFile.xcodeproj to backup.projectFile.xcodeproj or old.projectFile.xcodeproj or similar, and there could be a checkbox "delete old project file" that is unchecked by default.

[Daandelange]

Thanks for your feedback, I'll try to do something to save vertical space and put warnings next to the concerned items.

About the missing addon warning.... good question. For the moment it generates the project removing the missing addons.
Not a big issue for the moment; I created ofZach/pgElectron#12 for this.

I don't feel good about duplicating projectFile.xcodeproj but I love the idea to make "snapshots" of a project before doing dangerous operations. Some kind if backup system.

[ofZach]

@kylemcdonald @Daandelange it's kind of a lot of work for the PG tool to do "backups" of the projects that it's updating because there are multiple files per platform, not just xcodeproj files and it's not really built into the system and also I think it will make folders really messy, I don't want to get in the business of unwinding backups, etc. For example, if you change a file to "backup.projectFile.xcodeproj" do you also change all the files that may reference it, like in the case of VS? I think a general "be careful" warning is enough, and maybe some output from the command line tool whenever it deletes something / overwrites something. Let's keep it simple !

[ofZach]

it's cool to see this progress, I'd be curious if some of the text on the screen can be alerts and or tool tips, so we can get it really more minimal. I also think the multi-select looks weird (the 3d style, etc) so maybe it needs to be integrated more. It's definitely coming together...

[bilderbuchi]

For backups you always have git, which I recommend using for this! Also, way back I had created https://github.com/bilderbuchi/ofStateManager to manage versions of OF projects, but nobody ended up using it.