Package Manager Proposal Discussion

[gtalarico]

Sketch of Package Manager Concept for discussion:



### What does it do

Allows pyrevit users to install/remove/update pyrevit extensions from within revit.



### How it works

1. A Package Index in json format is Stored on PyRevit Repo

2. Package stores (name, version, url, and description).

3. Pulls json from Repo using requests or .net webclient

4. Find pyrevit local dir by package manager script location

5. Check local folders and compare to name of packages in packages.json

6. Determine if remote package is installed, and compare version to local packages

(use version variable insite package's init.py or version.py file instead of opening files and re searching for version for better performance and reliability)



7. Render a Datagrid with action buttons for every row:

Package name, remote version, local version, description, and actions:

Not installed: Install

Installed but not latest: Upgrate + Remove

Installed and latest: Remove



Actions:

* Remove: Delete folder

* On install/Upgrade: Clone remote repo from json: url path into folder 'packagename'





### Implementation Details:

PACKAGES.JSON (stored on pyrevit repo): item # 1/2/3 abovelist:

example:

[

{'name':'pyrevitplus','version':'1.0.0', 'url':'github.com/gtalarico/pyrevitplus, 'description': 'Tools for XXX'},

{'name':'selectiontools', version':'1.0.0', 'url':'github.com/gtalarico/pyrevitplus, 'description': 'etc'},

]



Once packages is read and compared to local folders, a local_packages dictionary is created in memory, adding the following keys:

'installed': True/False,

'local_version': '0.2.0''



local_packages is then used to render datagrid, and add appropriate actions.

Note: If program cannot reach packages.json, TaskDialog warns user, and dialog is closed.

UI:

- Use WPF

- Simple white window, with full-size data-grid as described above

- Future Feature: Add button to manually install package by browsing to .zip file (version 2.0)

[eirannejad]

Looks pretty good to me. One suggestion: Use git rev-parse HEAD to get the HEAD id of the extension git repo (since all extensions are git clones) and use that for version comparison. The json file in pyRevit repo should only contain extension name,url,desc for flexibility.

Package manager will compare git HEADs and will determine to Install/Update/Remove the extension.

[eirannejad]

git has a library for C# but I wasn't able to get it to work in IronPython libgit2sharp
This might be useful

[eirannejad]

Also the json file is part of the pyRevit git repo so it gets cloned on the local folder. You might be able to use that instead of pulling from web.

[gtalarico]

Agree on head Vs manual version control.

What about using something like gitpython in conjunction with git portable?
I guess using subprocess and popen can achieve the same results, but it
would help make the code cleaner and more manageable.
I can give it a shot while testing the package manager if you think it's
worth pursuing.
http://gitpython.readthedocs.io/en/stable/tutorial.html#tutorial-label

Gui Talarico
[ mobile ]

On Aug 16, 2016 5:11 PM, "Ehsan Iran-Nejad" notifications@github.com
wrote:

git has a library for C# but I wasn't able to get it to work in IronPython
libgit2sharp https://github.com/libgit2/libgit2sharp

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#119 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJEr8HyyFn-rYugoE_lnz9e6T7ZiY5_Yks5qgidigaJpZM4JlqBV
.

[eirannejad]

Never used gitpython. is it properly maintained?

[gtalarico]

Gitpython: Looks like it's active:
Over 1k stars, 90+ contributors, 1.5k commits, latest commit is hours old.
https://github.com/gitpython-developers/GitPython

Either way, I want to wrap the clone/delete into functions, so later we can change method more easily.
I will probably start just copying the code from your settings_download updates.

[gtalarico]

UI test preview. build dynamically from packages JSON + compared data (ie: Installed)
Thoughts from a UI perspective?
(I will remove/clean up final visible columns)
I think eventually it could be more sophisticated, with search (dynamo package manager, or sketchup extension warehouse) but I thought this could work for version 0.1

[eirannejad]

I like it! I think it's simple and easy to use! Thanks pal!

[gtalarico]

Just a quick update
Found out the hard way git does not like having other git directories within a repo.
The solution is to use the git submodule interface instead of traditional git clone.

It is was also hard to delete package/files when package is removed because git locks some files, but I found a work around. (remove read only first if fails to delete).

I have a beta working on my fork under a the manager branch.

Missing functionalities before beta release:

• add version check / head compare
• I am still intrigued by the idea reaching for a pyrevit repo web hosted packages.json and a local fall back that ships with the pyrevit repo. That way user doesn't have to update pyrevit repo just to get list of available packages. Thoughts?

[eirannejad]

_Question:_ How did u end up having a repo inside another repo? Isn't the package manager cloning the extension tabs in the root folder? (where git, init, and pyRevit clone exist)
Agree with you on the web hosted packages.json. It could be a newer version and the user might not update the local repo before running package manager. You knew better 👍

[gtalarico]

I am actually just just working out of the latest commit on your master : )
I was more focused on getting refactoring the prototype code base, getting the UI working and the clone/remove/cleanup functions.

Do you have a branch of the "new structure" you emailed me ?
Do we have a working loader for the new structure?

https://github.com/gtalarico/pyRevit/tree/manager

[eirannejad]

Yeah I'll email you a dropbox link. I have it in the private pyRevit master repo but I also have other company tools in there that has made the repo a bit big. I'll be cleaning that up later as the main development repo for pyRevit and will share with you. Till then I'll send you the folders in compressed archives.

[gtalarico]

Just Played around with new dev version and modified package manager to work with it.
It's working well.
Meanwhile, more questions and issues to discuss @eirannejad 😄

Remote + Local Packages Repo

Implemented as following:
Attempts to get packages.json from pyrevitrepo/master/packages.json
If it failes for any reason, look for pyrevit/pkgManager/packages.json.

Remove Tab after removing pacakge

1. Do you know if it's possible to remove a tab without restarting revit?
   Would be a nice touch to add to package manger>Remove Package

Potential Solution: Hide Tab by Name?
http://forums.autodesk.com/t5/revit-api/remove-or-hide-a-ribbon-tab/td-p/6043321
http://thebuildingcoder.typepad.com/blog/2013/07/dynamically-hide-and-display-a-ribbon-panel.html

Adding Scripts after installing.

1. Can you think of way we can call the " Reload Scripts by code (import?) so after install/clone, we can reload automatically.

Relationship of Pyrevit Core, Pyrevit scripts, and Packages Repos.

1. I think I am starting to understand why I was confused about the embeded .git folder:
   You typically keep only the pyrevit scripts in the pyrevit repo.
   I guess I am a bit confused as I have the pyrevit core (loader, git, etc) and pyrevit scripts all in one repo, and the official pyrevit repo includes only the scripts
   Anyhow, I think we will have some work to do to fully integrate my fork and understand how the core+scripts+package repos will all work together.
   In the meantime, I am using git submodules which are made exactly for this purpose (have git repos inside another repo, in this case, the pyrevitplus repo as a module of pyrevit repo (core+scripts).

Video of latest tests

https://www.dropbox.com/s/p8igh0orprxeo0y/2016-08-23%2019-55-09.flv?dl=0

[eirannejad]

Nah. Revit doesn't return any objects when creating Ribbon Tabs

[gtalarico]

@eirannejad
I have a semi stable version running here if you want to take a peak
This fork includes the full thing (core+scripts+git) but should work as a fresh clone.
https://github.com/gtalarico/pyRevit/tree/v3

Users this packages.json: https://github.com/gtalarico/pyRevit/blob/v3/pyRevit/pyRevit.tab/pkgManager/packages.json
Demo Package: https://github.com/gtalarico/pyrevitplus

[eirannejad]

Looks really good to me! Thanks so much!

On 24 August 2016 at 20:40, Gui Talarico notifications@github.com wrote:

@eirannejad https://github.com/eirannejad
I have a semi stable version running here if you want to take a peak
This fork includes the full thing (core+scripts+git) but should work as a
fresh clone.
https://github.com/gtalarico/pyRevit/tree/v3

Users this packages.json: https://github.com/gtalarico/
pyRevit/blob/v3/pyRevit/pyRevit.tab/pkgManager/packages.json
Demo Package: https://github.com/gtalarico/pyrevitplus

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#119 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH0XHBfMZPPYfZTbIgWlVtbsQSgEqQBIks5qjQ7IgaJpZM4JlqBV
.

[gtalarico]

@eirannejad I made a few revisions to the packages manager and I think it's ready for beta, most importantly, I updated the git commands to the use separate git directories (non-nested) and the correct folder structure.

The latest packages is in this branch:
https://github.com/gtalarico/pyrevit/tree/pyrevitv3

I can't do a pull request because the directory structure doesn't match the your master (ie. pyrevit.tab directory),

[eirannejad]

I get an error. The portable git is installed in __git__ folder.

[image: Inline images 2]
[image: Inline images 1]

On 27 August 2016 at 15:04, Gui Talarico notifications@github.com wrote:

Reopened #119 #119.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#119 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH0XHCs47L2f8UtS6gKQW2Hp7aXmtCLMks5qkLSKgaJpZM4JlqBV
.

[gtalarico]

I belive this was fixed on commit bb53c65
And
a785a8a

Gui Talarico
[ mobile ]

On Aug 29, 2016 12:41 PM, "Ehsan Iran-Nejad" notifications@github.com
wrote:

I get an error. The portable git is installed in __git__ folder.

[image: Inline images 2]
[image: Inline images 1]

On 27 August 2016 at 15:04, Gui Talarico notifications@github.com wrote:

Reopened #119 #119.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#119 (comment), or
mute
the thread
<https://github.com/notifications/unsubscribe-auth/
AH0XHCs47L2f8UtS6gKQW2Hp7aXmtCLMks5qkLSKgaJpZM4JlqBV>
.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#119 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJEr8G3VcsYuAICsV8Ir4QMYsGT72JlYks5qkwuxgaJpZM4JlqBV
.

[gtalarico]

Just tested on my work computer, worked ok. Latest config file should look like this.

[gtalarico]

@eirannejad I think I figure out why yours was not working
Looks like when I rebuilt the pyrevit dir from the .zip you sent me
I nested the pyrevit.tab one level deeper than I should have.
either way should an easy fix by just adjusting the config dir constants.
sorry about the confusion!

[eirannejad]

Gui,
Well done master!!! I edited the config file like this to get the correct folder address:
PYREVIT_DIR = os.path.dirname(SCRIPTS_DIR)

And it's working like a charm! Love the way it removes the extension tab being uninstalled. I'll cleanup everything else this weekend and publish v3 for the labour day.

Thanks again!!