Revive the vim-scraper/scripts repo #55

jdevera opened this Issue Oct 21, 2011 · 4 comments


None yet
2 participants

jdevera commented Oct 21, 2011

The information in it is invaluable! Perhaps a single JSON file with all info would suffice.

I'm asking because I was writing something that relied on it and only today I read that it would not be updated any more :(

bronson referenced this issue in vim-scraper/ Oct 23, 2011


Document the scaper API #13


bronson commented Oct 23, 2011

The scripts repo is just a side effect of tracking scraper bugs. Now that the scraper has been burned in, it's not really needed anymore. Not only that, it exposes internals that have already changed.

Yes, I can create a ~20 MB JSON file that includes all scraped data. I'll try to get to this in the next week or two.

(mental note: @bramswenson, author of vimpack, also asked privately about this in March. The existing api files were enough for him)

jdevera commented Nov 14, 2011

I think Ideally we could have a full version and a light version, with the header part for each script only. Something like that could also be leveraged by gmarik/vundle to get script summaries in the search option. This way, each run would not need to get the full 20 MB of JSON. However, for the tool I was writing I'd still need the full version. I think there is value in having both, What do you think?

Edit: I just took a look at the format, what I mean is the light version would exclude the fields description, versions, and install_details.

Edit 2: This filtering would take it down to 537K, which might still be a little too much for each run a BundleSearch

Edit 3: After some further filtering and key compression I get to a total of 300Kb, keeping only the name, type and summary. I know this is very specific, but I guess if these files are going to be generated anyway, something so widely used as vundle is worth having some massaging done for. Maybe I'm wrong.


bronson commented Nov 15, 2011

There are a few json files in /api right now. Check out #14 for some quick descriptions. AFAIK Vundle is already using them for some things (cc @gmarik).

I'm happy to produce whatever files people need. If what's there doesn't fit your needs (and I hear the need for all info, it's coming), tell me and I'll put it there.

jdevera commented Nov 15, 2011

Oh wow, I think scripts_recent covers exactly this last thing I was asking for, thanks for pointing it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment