New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leverage VAM-kr to determine names and dependencies #117

Closed
ipkiss42 opened this Issue Jun 23, 2013 · 15 comments

Comments

Projects
None yet
3 participants
@ipkiss42
Copy link

ipkiss42 commented Jun 23, 2013

Hi,

The VAM plugin (MarcWeber/vim-addon-manager) works uses VAM-kr (MarcWeber/vim-addon-manager-known-repositories) as a database of plugins, providing nice names, sources and dependencies among other things. I think it would be very useful to use VAM-kr in NeoBundle as well. Here is how it could work.

Using FuzzyFinder as an example, the current way to install it is:

NeoBundle 'bb:ns9tks/vim-fuzzyfinder', {'depends' : 'bb:ns9tks/vim-l9'}

This is tedious, because it implies to find the right plugin source (and not one of the forks), to identify the dependencies, and to find again the right plugin source for the dependencies. And every user of FuzzyFinder has to do the same work.

Relying on VAM-kr, it could be as easy as:

NeoBundleVamKr 'FuzzyFinder'

or maybe:

NeoBundle 'FuzzyFinder', {'vam-kr' : 1}

To do that, NeoBundle would simply lookup the provided name in VAM-kr, and automatically deduce the source and dependencies (doing the same recursively for each dependencies).
Note that it is still compatible with the 'name' option: you could give another name to use with NeoBundle, for example if you don't like the one in VAM-kr. Same for 'depends' and 'build', if the VAM-kr database is incomplete.

This would be particularly interesting for vim.org plugins, which are quite annoying to install with NeoBundle at the moment.

VAM-kr would be an optional dependency of NeoBundle (just like the dependency on vimproc). If not present, NeoBundle would work as normal, and the NeoBundleVamKr command would return an error.

Additional bonus: it would make it much easier for users currently using VAM to try NeoBundle, because the migration could be (almost) a simple search&replace.

So to summarize the advantages:

  1. Users deal with friendly names
  2. Users do not need to care about plugin sources
  3. Dealing with vim.org scripts is not painful anymore
  4. Users do not need to care about (non-optional) dependencies in most cases
  5. Users can still use NeoBundle options like 'name', 'depends', 'build', etc.
  6. Any improvement in VAM-kr automatically benefits to NeoBundle as well
  7. The dependency on VAM-kr is optional
  8. VAM users can migrate to NeoBundle more easily
@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Jun 24, 2013

Yes. I know VAM and your plans is already in my todo list.
https://github.com/MarcWeber/vim-addon-manager-known-repositories

But it is big feature. So I don't have much time to implement it.

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Jun 24, 2013

I think vim.org should have repository database like VAM-known-repository.
I want to standard repository detabase.
The database is important than plugin manager.

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Jun 24, 2013

If neobundle support VAM-kr, neobundle must parse it. I think it is too slow.

@ipkiss42

This comment has been minimized.

Copy link

ipkiss42 commented Jun 24, 2013

About speed: it cannot be so slow, because VAM itself is reasonably fast. And in the worst case, it's always possible to warn the users in the documentation that it may be too slow (like camel case omnicompletion in NeoComplCache).

About standard repository: I agree it would be good to have, but I don't think this is going to happen anytime soon unfortunately... You could provide support for VAM-kr in a first step, and change the implementation when/if another database appears. The change will be transparent for users, and more importantly, users can already take advantage of automatic dependencies without waiting.

About big feature: I understand that it may take time to implement this. Maybe it would take a reasonable time if NeoBundleVamKr is implemented as a wrapper around NeoBundle, and reuses the VAM-kr API? I don't know.
I just hope that you will find some time for that :)

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Jun 24, 2013

I already implemented receipe feature.

https://github.com/Shougo/neobundle-vim-scripts

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Dec 29, 2013

@ZyX-I

This comment has been minimized.

Copy link

ZyX-I commented Dec 29, 2013

VAM is not slow with VAM-kr because it needs VAM-kr exactly for three jobs: updating, installation and printing some information about plugins. It is though known to be fast enough to load pool in less then one second (0.5 seconds (0.49 with FS cache, 0.55 with it dropped)), but I have rather good PC. I.e. the main idea is that you do not need VAM-kr for regular operation and when you need it one second delay is standable.

I already see that you duplicate parts of autoload/vamkr.vim. Is this really required? Unlike functions it is harder to keep files stable. You may for sure expect either patchinfo_generated.json or dependencies_generated.json file in the future: I have plans for making autoget.py recognize dependencies written in GetLatestVimScripts format. For VAM users this change will have immediate effect because appropriate functions will get updated. With your parser you will have to waste time updating it.

Currently the only dependency on VAM in autoload/vamkr.vim is vam#Log (except for dependency in strings added by vamkr#AddCopyHook) which is used to give ignorable notice messages about name conflict in VAM-kr database (they will be fixed once I try to commit something manually and not by cron job: commit with this problems will be prevented by hook) and it can be easily removed.

ZyX-I added a commit to MarcWeber/vim-addon-manager-known-repositories that referenced this issue Dec 29, 2013

@ZyX-I

This comment has been minimized.

Copy link

ZyX-I commented Dec 29, 2013

LOL 40% of time wasted on generating snr_to_name dictionary for the second time using inefficient cycle. 0.29 seconds with FS caches, still 0.5 with them dropped (first result is highly stable (0.29-0.30), second result is highly unstable (0.45-0.53)). Bad I have not checked the stability of results in previous test run.

@ZyX-I

This comment has been minimized.

Copy link

ZyX-I commented Dec 29, 2013

* Missed the fact that in the current state if you want to automatically update changes you either need to download full archive, use git like we do or use something more geeky like FUSE to load files on demand. If you describe the preferred format I can make my cron job post pregenerated contents as one file to specific location (e.g. zyx_i.bitbucket.org/VAM-kr_pool_for_NeoBundle.json).

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Dec 29, 2013

VAM is not slow with VAM-kr because it needs VAM-kr exactly for three jobs: updating, installation and printing some information about plugins. It is though known to be fast enough to load pool in less then one second (0.5 seconds (0.49 with FS cache, 0.55 with it dropped)), but I have rather good PC. I.e. the main idea is that you do not need VAM-kr for regular operation and when you need it one second delay is standable.

OK. Unfortunatelly, I think 0.5s is slow.
Because my Vim starts with less than 0.5s.

You may for sure expect either patchinfo_generated.json or dependencies_generated.json file in the future:

Yes. But I think the fileformat and specification must be more stable.

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Dec 30, 2013

Currently the only dependency on VAM in autoload/vamkr.vim is vam#Log (except for dependency in strings added by vamkr#AddCopyHook) which is used to give ignorable notice messages about name conflict in VAM-kr database (they will be fixed once I try to commit something manually and not by cron job: commit with this problems will be prevented by hook) and it can be easily removed.

I think VAM-kr must be more simpler code. It is for VAM only.
I don't want to contain the code.
If VAM-kr is simpler, I may use VAM-kr functions directly.

@ZyX-I

This comment has been minimized.

Copy link

ZyX-I commented Dec 30, 2013

OK. Unfortunatelly, I think 0.5s is slow.
Because my Vim starts with less than 0.5s.

Why do you need to load VAM-kr on startup? patchinfo.vim contains some data essential for it, but slow loading is exactly why data from there gets written to addon-info.json.

Given the database in format that does not require any transformations and suitable for eval() (i.e. string(vam_known_repositories#Pool())) it is 0.05s (which is standable I guess), but it is also increases memory consumption by more then 4 MiB (resulting file size is 884 KiB, but great amount of memory is spend on things like hash tables for reasons like making hash table size be always a power of two, not to mention malloc always returning aligned memory (dunno whether the latter contributes to size/vsize column reported by ps)).

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Dec 30, 2013

Why do you need to load VAM-kr on startup? patchinfo.vim contains some data essential for it, but slow loading is exactly why data from there gets written to addon-info.json.

In the world, there are many plugins has not addon-info.json.

Given the database in format that does not require any transformations and suitable for eval() (i.e. string(vam_known_repositories#Pool())) it is 0.05s (which is standable I guess), but it is also increases memory consumption by more then 4 MiB (resulting file size is 884 KiB, but great amount of memory is spend on things like hash tables for reasons like making hash table size be always a power of two, not to mention malloc always returning aligned memory (dunno whether the latter contributes to size/vsize column reported by ps)).

OK. This overhead can be ignored in recent machine.

@ZyX-I

This comment has been minimized.

Copy link

ZyX-I commented Dec 30, 2013

In the world, there are many plugins has not addon-info.json.

"Gets written" means "data from VAM-kr gets written". It happens during the installation process assuming addon-info.json was not found (i.e. was not provided by plugin author) and VAM-kr contains something that has to be written.

OK. This overhead can be ignored in recent machine.

I would not welcome it on my router.


Reply to this email directly or view it on GitHub.

@Shougo

This comment has been minimized.

Copy link
Owner

Shougo commented Dec 30, 2013

"Gets written" means "data from VAM-kr gets written". It happens during the installation process assuming addon-info.json was not found (i.e. was not provided by plugin author) and VAM-kr contains something that has to be written.

OK. I get it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment