greasemonkey / greasemonkey

This is a platform thing we might not be able to overcome.

Couldn't we get dirty and go straight for the XUL in this case? We could overlay about:addons.

https://bugzilla.mozilla.org/show_bug.cgi?id=565610
Perhaps. I hadn't actually realized that bug was fixed, until now. I'd hate to re-implement from scratch, when we're provided with an API to extend the add-ons manager. Hopefully it can be made to work the way we need it to.

I'd hate to re-implement from scratch, when we're provided with an API to extend the add-ons manager.

I hope we can avoid re-implementing this but I'm pretty sure we will have to overlay the manager to complete #1182.

This is a platform thing we might not be able to overcome. I've reached out to someone at Mozilla, but not yet heard back.

Maybe we should open a bug for this? (they can always mark it notfixing)

I, at different points in time, might use Eclipse, TextWrangler, or EditPlus.

Notepad++, for me.

UltraEdit or Vim

Eclipse (as for plugins I've used Aptana, WTP, amateras, JS Eclpise all b4 but I'm currently using WTP or amateras).

Emacs

Notepad++

UltraEdit 12.1, the main useful thing here is the f8 function list to jump around the file. I think this mirrors eclipse issues where:

components/greasemonkey.js it's basically useless but
content/browser.js it works pretty well. Prototype seems to work well in
content/miscapis.js but creates extremely long function names.

Other than that I'll use Vim for a small change usually to a configuration type file.

Other than that I'll use Vim for a small change

Me too.

vim is great for small changes, I tend to use it when I'm in command line land and want to make a quick change.

I use Notepad++ but I find it a bit awkward to jump around large files (more than a few k lines), making heavy use of search in lieu of a function list (perhaps this exists - I just haven't looked properly) ..

Disclaimer: I don't explore the source of GM much, but Notepad++ is a pretty awesome text editor for creating single file user scripts (note: imho not an ide + perhaps not so good for GM addon dev?)

Emacs, and occasionally but rarely TextMate.

All the above but usually kwrite/kate depending on how I have my O.S. set up.

Okay, so far I'm seeing:

Eclipse:2
EditPlus: 1
Emacs: 2
Notepad++: 3
TextMate: 1
TextWrangler: 1
UltraEdit: 2
Vim: 4

But Vim was mostly mentioned in one-off cases, for "small changes" -- and I'm not even sure how it provides such a feature, probably only through one of many possible plugins. So I'll probably not concentrate on that.

At least for myself, when editing JS with Eclipse, that means the WTP plugin. So we'll begin from there.

There's actually quite a lot of possible ways to arrange functions in JS. Especially considering that some of them might cause really long lines that we'd want to wrap -- but some programs will fail to handle function definitions split across lines. So here's the arrangements I've managed to think up so far: http://gist.github.com/498392

There's a lot of edge cases to consider. I'll post some editor-specific observations momentarily.

Edit: not mentioned previously, but something I thought-but-didn't-remember-to-say is that checking how Venkman / Firebug debuggers see all these styles is also super valuable!

For EditPlus (which doesn't matter much, but is most accessible to me ATM):

• GlobalConstructor.prototype.anonymousAssignedToPrototype works
• prototypeObjectLiteralAnonymousFunction and objectLiteralAnonymousFunction work "okay" (but neither visibly namespaced into their object)
• 'objectLiteralNamedFunctionOL': function ObjectAsNamespace_namedFunctionOL() { } really confuses the editor, but both function ObjectAsNamespace_namedFunctionNL() { } work (the "assignment" and the function on the same line confuse this simple line-based regex system, but what "looks like" just a regular function-on-a-line it can handle)

Results here probably only matter to me (as only I listed EditPlus), and I only use it semi-frequently. And it's customizable a bit, so these results are probably not super important but:

• Full assign-to-prototype is best IMO
• Assign (long-)named function to object literal second best
• Assign anonymous function to ojbject literal third

Eclipse:

• Cannot "see" any anonymous functions, except ..
• Can see GlobalConstructor.prototype.anonymousAssignedToPrototype -- and in this full prototype-assigned format, it can also see the full structure of the object.
• Can see that GlobalConstructor2 is an object (it's got a prototype) but knows nothing about its structure.
• Can see that GlobalConstructor2_prototypeObjectLiteralNamedFunction (and it's newline cousin, and its object as namespace relatives) exist, but knows nothing about their object structure.
• Assigning a named function to a prototype is counter-productive, because it causes the function to be listed twice (once for the prototype name, once for the function name).

So for eclipse, fully x.prototype.y specified code is best, even if the functions assigned to these names are themselves anonymous. Object literals assigned to prototypes or namespaces need to have their functions named and then all these names show up in the flat global space.

I had to google to find the Function List plugin (is this what you guys use?)
http://randronov.blogspot.com/2008/11/function-list-for-notepad-5x-unicode.html

With it installed, it seems to act a lot like Eclipse:

• It can see GlobalConstructor.prototype.anonymousAssignedToPrototype by that name.
• It can see all named functions assigned to something (be it prototype or literal) by that name i.e. GlobalConstructor_namedAssignedToPrototypeOL but never the full prototype name (like anonymous) or the name in the literal.
• It cannot see anonymous functions assigned into object literals.
• It doesn't care about splitting long lines.

So it seems the best, like eclipse, is to do anonymous functions assigned to full prototypes. Otherwise any function assigned to a literal must be named.

I got the UltraEdit demo:
http://www.ultraedit.com/downloads/ultraedit_download.html

• It can see the GlobalConstructor.prototype.anonymousAssignedToPrototype by that name.
• It can see GlobalConstructor_namedAssignedToPrototypeNL by that name, and not its prototype assigned one.
• It can see GlobalConstructor2_prototypeObjectLiteralNamedFunctionNL by that name, but not the version on one line, same for namespace object literal.
• It can see prototypeObjectLiteralAnonymousFunction by that name, and same for namespace object literal.

I'm seeing a pattern. Assigning one-by-one to the prototype seems best, and anonymous functions to literals when literals are necessary.

For the rest of the editors, I'd appreciate if people take a look at what they can see with the linked example file.

TextWrangler sees everything just about perfectly; named functions as their name, anonymous functions as whatever they're assigned to, all flavors recognized as part of the object they really are, and visually organized as such. So whatever we adopt it will adequately deal with.

At least for myself, when editing JS with Eclipse, that means the WTP plugin. So we'll begin from there.

For Eclipse there are many different plugin editors, Aptana, WTP, amateras, JS Eclpise, reg text editors, etc.. I don't think we can assume WTP will be used (not saying that you are Anthony, afaict you seem to agree w/ what I just said). There are so many different possibilities here that we cannot make general claims that Eclipse can do x,y,z but not a,b,c

We have to judge each plugin individually.

We have to judge each plugin individually.

Good point. I did say "begin from there" in regards to WTP, even if I later oversimplified to just "Eclipse does ...". In any case though, I'm still seeking feedback from anyone familiar with any editor that I haven't already tested.

However, across only those I've tested, I already see a common denominator: named global functions, and assigning anonymous functions to foo.prototype.bar seems to always work.

JS Eclipse recognizes everything in your gist, and ignores the names of the named functions provided to prototypes (ie: foo.prototype.x = function bar() {}, 'bar' is ignored in the outline).

However, across only those I've tested, I already see a common denominator: named global functions, and assigning anonymous functions to foo.prototype.bar seems to always work.

I think it makes sense to use this method.

I alternate between vim, Smultron, and gEdit, although some of the editors mentioned here sound useful and I might have to try them out.

I don't remember where I ran my original eclipse tests, but right now with eclipse:

Version: 3.5.2
Build id: M20100211-1343

And plugin:

JavaScript Developer Tools

Version: 1.1.2.v200908101420-77-FGDCcNBDjBXMoBbFb
Build id: 20100211202452

I'm seeing much crappier behavior: it can't figure out the structure of any of the objects. It sees exclusively named functions, and for some reason none of the ObjectAsNamespace ones.

I've just stumbled across JetBrains WebStorm [1] and I'm pretty damn impressed with everything that it does. Admittedly I've only been using the (45day) free trial for an hour or so and haven't had a chance to probe it fully yet, but it looks like it is going to replace Notepad++ on my Ubuntu 10.04 machine.

Not sure which bits of the test file [2] are important, nor how would be best to describe it but..

• it lists all defined functions in the 'structure' panel as a "method" --> with the sole exception of GlobalConstructor_namedAssignedToPrototypeOL() but I think that it is behaving correctly in this instance and would not want it to behave any differently
• It lists the remaining constants as 'variables', placed at the correct points within the hierarchy

As I said, I'm impressed but I've not had much sleep so I strongly suggest you take a look at the feature-list [3] yourselves! Supposedly Git is supported (yet to test) and a whole bunch of refactoring tools that I was actually looking for seem to work pretty well. I'm toying with the awesome error spotting functionality atm.

Well, hopefully I don't sound like a rambling advert but I am genuinely surprised by how much I like it... I'm not entirely sure about plugin support etc yet though but tbh I'm not sure it needs it...

[1] - http://www.jetbrains.com/webstorm/ [2] - http://gist.github.com/498392

[3] - http://www.jetbrains.com/webstorm/features/

Ps, I changed the theme pretty quickly though.. (I prefer the one called 'Nimbus')

I've got an implementation in this branch, here are a couple of tests cases:

• gist 402611
• gist 402767

I agree.

As Greasemonkey user, now I use wescript.net for update scripts. It works, but it's another add-on. Not so many people know about it.

Look at Stylish add-on. It has "Find Updates" button Greasemonkey should learn from this.

I see this issue labeled "0.9.x". Great!

GM needs UI for updating scripts.

What do you think about it?

This is also being discussed on Userscripts.org: http://userscripts.org/topics/48267

Well here's a start to solving this issue. This commit involves mostly changes to the GM core to support update checking and doesn't do anything to notify users of an available update. Specifics are in the commit message.

Update: I'm currently working on this feature in this branch. Any help would be greatly appreciated.

I finally got around to integrating recent upstream changes into my branch for this issue. So far the actual update checking works fairly well given the few tests that I've run. I've only been able to get some basic integration with the addons dialog working. On Firefox start up, if there are any updates that have been accumulated the addons dialog will open and display them for the user. You can install available updates via the context menu. I've made the "Find Updates" button filter the scripts list to show only scripts with available updates. Scripts that have an update have a little green exclamation mark next to them in the addons dialog.

Great work sizzle. I might flick though a few patches if I get some time off my other projects :)

The function you describe, by its name, is the one that is called from the "DOMContentLoaded" event handler. If you're intending to run something which is not fired from that event, you should not be calling this function. If you want to do essentially the same thing (inject scripts) from two different places/in two different ways, there probably needs to be some refactoring, so the common stuff exists once, simply called twice.

There should be no "inject these types of scripts" parameters, but each case is handled by calling the appropriate function, which by its nature knows what scripts are concerned.

Even if there is a flag, it should be a series of named constants, not just "0", "1" and so on.

That said, yes. Components must adhere to an interface. There's a trick to get around that, and I think we're even using it, but it has caveats. So it's still best to document it properly, through the standard interface definitions.
https://developer.mozilla.org/en/XPIDL

See the "idl" files in the source directory of that name.

An alternative would be allow the wrappedContentWin object to be extended

Again, absolutely not. This object has a very distinct meaning from its name, and it should NOT be used just to "pass parameters" not related to its named function. (And anyway, being a XPCNW, it also must adhere to its interface, you can't just tack anything you like onto it.)

Thank You! I've found the correct file (in my haste/lack of bash I was modifying the public xpi), I will make a separate function later we could remove the existing one. If only there were an easy to use idl->xpt compiler on my system it would be ready, I still need to satisfy some prerequisites or build one. The format of the xpt file doesn't look that difficult so I should be able to get my testing working either way. I will probably call the function handleInjectionState or something similar injectScriptsForStage and it will accept three parameters. As far as naming goes I'm open to suggestion.

I agree about named constants however defining constants that are exposed to both greasemonkey.js and browser.js in a single place seems like another challenge.

INJECT_STATE_ALL, INJECT_STATE_DOMLOAD, INJECT_STATE_DOCSTART

If we should plan on supporting additional states such as document-end or document-idle is another question although the most useful one at the moment is document-start there are a lot more possibilities.

If only there were an easy to use idl->xpt compiler on my system it would be ready,

One of the less fun bits of XPI development. Googling "xpidl.exe" or "xpidl binary" can be helpful, if you trust what you find. I've saved a copy for my own use: http://arantius.info/downloads/xpidl.exe

I will probably call the function handleInjectionState or something similar

The existing function, domContentLoaded is called at domContentLoaded time. The new function, to run at document start time, might be called documentStart (the logic is tricky, eh?). They should both gather the list of appropriate scripts for the event they handle and, in turn, call helper functions to do most of what is today done in domContentLoaded.

I agree about named constants however defining constants that are exposed to both greasemonkey.js and browser.js in a single place seems like another challenge.

Add an object to the GM service object. Fill it with a few read only keys.

If we should plan on supporting additional states such as document-end or document-idle is another question

No. Our general policy is that we don't implement for the authors anything that they can implement themselves. That's the realm for @require -- which can be maintained much more nimbly, among other benefits.

This particular issue regarding compiling gmIGreasemonkeyService.xpt from gmIGreasemonkeyService.idl should be resolved now. Thanks for your help!
reverted commit I have yet to merge in the rest of the implementation. I was planning on merging the config.js portion next. There are also additions in browser.js plus greasemonkey.js.

Is one huge commit preferred? I was thinking it would be easier to split into pieces as to not collide with other work that's being done, since there will possibly be a lot more debate about naming conventions the particular sections (especially config), but it's also easy enough to include these changes grouped in another commit instead.

I verified the 3 line breaks in the complied file posted correctly, also compiled and ran this in the latest version.

http://greasemonkey.github.com/contrib.html

What's best is one branch per contribution. With any number of commits. I, personally, find committing early and often helps move things forward. Small commits, with descriptive log messages on each, are easy to understand.

Given a branch, we can discuss it, changes can be added in the branch, and then the whole thing can be merged upstream. Start with a "master" branch that directly mirrors upstream's master (with no changes). Start a branch containing changes, isolated from said master.

Sounds great! I was wondering why everyones graph looked so nice. I've started posting some of the changes to a branch called document-start, and i got going to the point where it should be basically all done now. I posted a userscript to test with http://gist.github.com/418707 however I imagine there will possibly be some refactoring since we could eliminate the somewhat redundant initEarlyScripts versus initScripts function (could possibly be handled with a parameter if preferred) plus I had to change initScripts that existed before and split out an operation into another function, which is something that could also be done a couple places in browser.js as far as populating the menu with scripts coded block appearing twice.

I suppose I'll probably sleep on it for now. I didn't include the value I had originally planned for about:config to dissableEarlyInjection however I'm not sure it's necessary. There are a couple things we could still probably optimize but it works fine in my testing so far.

Here is a link to the commit history: commits

Here is the complete diff to date: document-start

I will periodically upload a diff here: diff file upload colorize

First off, I wanna say I'd really love this feature(as I'm sure many others would) and I applaud your efforts qufighter. Now getting down to business.

Having a global preference for injecting all script early sounds very evil to me. Most installed scripts would likely be broken since they rely on the DOMContentLoaded to ensure the document is ready to mess with.

Therefore in my branch I've made the enableDocumentStart pref basically allow scripts to specify their early injection via the metadata value. If this pref is false then no script will be injected early. Because this early injection feature is only available in Firefox 3.5 or greater, the enableDocumentStart pref will be set to false. There still needs to be some code to turn this feature on when a user migrate from a version of Firefox where this wasn't possible to a version where it is.

In addition I've made an effort to reduce duplicate logic in greasemonkey.js.

I will be keeping the diff here updated: http://vidzbigger.com/diff/colorize/index.php?file=document-start.txt

Or you could let GitHub do it for you: http://github.com/qufighter/greasemonkey/compare/master...document-start

Select the "Files Changed" tab.

I'm about to run out to dinner so this is going to be kind of quick I'll have to look more closely later. To clarify a couple things however:

scripts only run at document start if the
@run-at document-start metablock imperative is present, this should not effect any existing scripts UNLESS those scripts have the @run-at defined

Secondly there is code that falls back, I've tested this in 2.0, and the preference value is defined as false for anything less than 3.5, so this feature is completely disabled on first run for anyone who has less than ff 3.5.

For anyone with 3.5 or greater the @run-at should take care of this being disabled by default. For ff 3.5 or greater about:config isn't even troubled with the value to enableDocumentStart unless the user explicitly adds it.

The only trouble I was thinking was that what if the try catch that attaches the documentstart listener fails for some other reason, then the ff3.5 user will be stuck with a disabled document start and not necessarily realize it, but that shouldn't ever happen anyway.

I agree if you would like to keep the value either defined for everyone or have it configured to false by default especially.

Thanks for the link to the actual compare feature! I definitely discovered it just a little too late :D

scripts only run at document start if the @run-at document-start metablock imperative is present, this should not effect any existing scripts UNLESS those scripts have the @run-at defined

I've looked over your code again and I believe you are correct. In which case, the changes in my branch are just re-factoring of the code.

I took a quick look at the changes I think it looks pretty solid and simpler. I'm sure it's just about ready. The whole file looks a lot better now.

One case that still needs to be addressed is migration. If a user starts using this with a version of Firefox less than 3.5, the feature will be disabled and enableDocumentStart will be set to false. Then later they upgrade and enableDocumentStart will still be set to false even though this feature is now possible.

Update: I've added code to turn this feature back on if a user migrates to Firefox 3.5 or greater.

I've yet to read in detail, but based on the comments:

A) Don't worry about Firefox below 3.0, we're adjusting to that minimum version.
B) There shouldn't be any sort of "preference" for this. Either we support it or we do not. Any script meant to run at document start time will have to run then, and vice versa. Any situation that makes it work only some times (users, settings, firefox versions, etc) is not good enough.

A) Don't worry about Firefox below 3.0, we're adjusting to that minimum version.

Yes but according to qufighter this feature isn't available until Firefox 3.5.

B) There shouldn't be any sort of "preference" for this. Either we support it or we do not.

The preference is for Firefox below 3.5 where this will fail. And rather than keep failing we've set a preference to not use this feature in that case.

Yes but according to qufighter this feature isn't available until Firefox 3.5.

What feature is "this" feature?

The preference is for Firefox below 3.5 where this will fail. And rather than keep failing we've set a preference to not use this feature in that case.

That doesn't invalidate my point. If it doesn't work 100% of the time for 100% of the users, I don't want it in GM. Can you imagine the documentation otherwise, not to mention the users (both end users and script authors) trying to deal with it?

What feature is "this" feature?

Running a script before the document has loaded.

That doesn't invalidate my point. If it doesn't work 100% of the time for 100% of the users, I don't want it in GM.

The feature in specific is called addTabsProgressListener it is well documented that it is only supported in firefox 3.5 or newer.... addTabsProgressListener

I'm sure we can still keep it simpler than the documentation for the @run-at parameter here especially since we are only supporting one of the states (and probably the only one necessary)

PS the original specification the @run-at notion is based on is mentioned design-documents/user-scripts

I don't have the time to read all of that, but I did find an old method of using addProgressListener on each tab to achieve the same result as addTabsProgressListener. I've implemented it as a fallback in this branch. It is confirmed to work in Firefox 3.0 and I've also removed the now useless preference.

Well it looks great to me, ditching support for 1.5, 2.0 is going to make it a lot easier, I still think that simply running the script at the normal time if doc-start isn't available is also sufficient as far as this feature goes considering if you attach a DOMContentLoaded listener of your own in a doc-start script (basically a must) versus a regular script it still fires at the correct/same time. Doc start seems just for above and beyond optional type functionality of doing things more seamlessly, it's great if we can support the entire user-base with this feature!

I still think that simply running the script at the normal time if doc-start isn't available is also sufficient

No, arantius is right. We can't add half-assed supported features. However, a work around for calling scripts at doc-start is available for all versions of Firefox we support in this branch. Test script.

I still think that simply running the script at the normal time if doc-start isn't available is also sufficient

If "simply running the script at the normal time" is sufficient, then we can just close this issue as a wontfix -- that's what GM does today every time. This feature does have significant value, but only if it works reliably. For:

• Injecting content-scope functions/data that inline page scripts rely on, but don't work / are broken (e.g. which rely on Internet Explorer only features).
• Injecting CSS for display, which will not "flicker" into view after the page (DOM) has loaded.

And so on. As for making this work reliably, I'd expect this can be done with way way less code. Look at some older versions of Firebug. Firebug injects a content-scope object (console) at document start time, via XBL (IIRC). Essentially, bind to the that holds the content page, use XBL to hook into the constructor event for that element, and you naturally have a reference to the window at exactly the time that the content window is being created.

Firebug is BSD licensed, which is MIT compatible, so we can reuse their code as long as we include the proper attribution and licensing nodes as per the BSD license's terms. If you investigate this route, I'd highly suggest looking at as old a version of FIrebug as you can -- their code got a lot more complex very quickly, and we don't need most of those complex features. I remember doing something like this this back in the 2007 or so time frame, so that's something like Firebug 0.4, maybe 1.0.

According to their wiki, they only started adding the console at document start in version 1.3, but they are pretty vague about what they used before 1.2 so perhaps they used the same method as you described. Anyway, I still have a lot more digging to do.

I've been looking around in firebug 1.5.3 since the start of this project but it is kind of complicated. Some thing to look at might be injector.attachIfNeeded, I'll have to locate the older source code soon

However in our own pre-existing listener that captures the user clicking tabs event also fires when the user changes pages (at document-start as close as I can tell) including page refresh (confirmed doc-start timing is correct in 3.5)

LOC2 [xpconnect wrapped (nsISupports, nsIDocShell, nsIWebNavigation, nsIDocShellHistory, nsIInterfaceRequestor, nsIWebProgress, nsIDocumentLoader, nsIRequestObserver)] || [xpconnect wrapped (nsISupports, nsIHttpChannel, nsIRequest, nsIChannel, nsIUploadChannel, nsITraceableChannel)] || [xpconnect wrapped nsIURI]

GM_BrowserUI.onLocationChange = function(a,b,c) {
  console.log('LOC2 '+a+ ' || '+b+ ' || ' +c,true)
  if (b) GM_BrowserUI.tabLocationChange(a.DOMWindow)

CONCLUSION: b is null during tab switching and has a value at ~document-start (fires before current implementation only slightly). I'm not sure how well or reliably...

Modified: b is null during tab switching EXCEPT during session restore or loading of inactive tabs

Same functionality seems to work, it has its own limitations though:

(added) Namely that it does not always fire for each tab on browser startup/restore, which our current implementation accounts for.

UPDATE: unfortunately B does not have a value during session restore (in inactive tabs), so it won't be enough on it's own I don't think, since during restore this needs to fire whether B is null or not, then immediately after that, fire only if B has a value.

UPDATE2: I managed to get our existing GM_BrowserUI.onStateChange listener for this firing.... turns out it fires too early for document start (the old document still exists) and then fires again (too late) after everything is loaded. (edited)

I'm going to re-state update1 that our pre-existing NOTIFY_DOCUMENT for the menu commander function GM_BrowserUI.onLocationChange DOES fire at the correct time, however, it only fires as expected in the active tab, so it does not work well during session restore (at least not yet, or not based on testing for B being null).

I undid all of the experiments and am going to test the current implementation FF3.0 next comment covers what I found there

FINAL UPDATE: I seem to have found this condition.... a.loadGroup.activeCount

if (b||(!b&&a.loadGroup&&a.loadGroup.activeCount>0))

a.loadGroup is either undefined or it's a.loadGroup.activeCount is zero when switching tabs, however the active count is 1 (or greater I assume). Any time b is defined it is safe to inject scripts, in the absence of b an a.loadGroup.activeCount>0 is sufficient to indicate scripts should occur. It is probably sufficient to:

if (a.loadGroup&&a.loadGroup.activeCount>0)
  GM_BrowserUI.tabLocationChange(a.DOMWindow)

the PROBLEM still exists however, that sometimes tabs are grouped into the same loadGroup, in which case this doesn't seem to fire for all tabs during restore :/

SO I think I'm just going to test for the body tag, if there is no body tag, we can almost certainly assume a load is occurring and it's at document-start ....

the problem that this listener is still not firing for all tabs on startup still exists, although it does sometimes, and was firing fine what I was restoring 6 tabs (with progress listeners attached but not registered with the function where I was detecting them) but then only firing for one of them when I had 5 tabs (and set up to only use the "main" listener), which makes very little sense, but it seems like we DO need to attach listeners to each tab or at least listen for new tabs being added and attach the listeners ourselves then.

Unless we can latch onto the constructor that creates documents somehow, since it would have to petty much create a new instance of document no matter what, at least I'm pretty certain. In firefox source code there is a nifty line that throws the DOMcontentLoaded, however there is no corresponding line near the constructor (or a few functions in) to throw the DOMdocumentStart or all these would be more easily solved.

There is an odd bug with firefox3.0 portable where a certain tab (usually the first one that is opened when the browser is launched) does not fire the TabOpen event required for ff3.0 and the document-start listener never gets attached. During restore all tabs fire tab-open except the first one, so one tab fails to apply document start no matter how many times it's refreshed.

Notification_when_a_tab_is_added_or_removed but not when the tab is always there.

Somehow at startup we need to iterate existing tabs, I expect we will find one tab, and need to attach the listener to that. I am guessing the browser xul contains one tab by default so it never fires the tab creation event.

So it might just be a matter of doing this once

EDIT: this is fixed as far as handling the one tab that exists during startup (3.0) by this: commit since all the other tabs in the restore are added they fire the normal tabOpen event.

As I started to document this feature on the talk page and noticed there is a discrepancy between the design documents

http://wiki.greasespot.net/Talk:Metadata_Block

It appears @run-at document-start becomes @run_at document_start in the more recent document.

I have posted a commit containing this change unless we should support both. reverted

I would definitely like to make this mirror the same syntax Chrome uses, so scripts will work on both platforms:

http://www.chromium.org/developers/design-documents/user-scripts

// @run-at document-start

The document you linked from that talk page which uses underscores is about chrome extensions, not user scripts. They're very similar, but different.

Firebug injects a content-scope object (console) at document start time, via XBL (IIRC).

So I finally got the time to look into this more and I haven't found any evidence of this. All versions that I've looked at use a similar method of using progress listeners to tell when a new url has been navigated to that I use in my branch.

As I explained earlier the wiki details when they have modified the implementation of the console so I examined versions within those time frames of release.

Version 0.4 uses progress listeners and that code is located in firebug/content/firebug.js. Shortly after that they moved the progress listener code to it's own file: firebug/content/tabWatcher.js. I saw this in version 1.01, 1.3.0, and even the latest version 1.5.4.

I think we have a solid implementation, and attaching the listener is really only one line of code in firefox 3.5, it is added in a try block and the catch block is kind of large but handles firefox 3.0 legacy support and functions identically to the firefox 3.5 code.

The changes to date have been simplified and can be reviewed here

The only thing that was left is what happens during restore, since we are listening for tabOpen and adding our listener to the tab then, tabs that are already open before greasemonkey runs did not necessarily receive listeners in firefox 3.0 (since one tab always exists), I handled that issue in this commit However I think that's all it takes to solve it, since it's only the initial tab that doesn't seem to fire the tabOpen event during restore.

There may be an alternate implementation but it requires a lot of looping through tabs and only works for active tabs (not those that change location in the background) and I'm not really confident it's the best approach since it would be a lot easier to run things twice accidentally during restore since the script only works in the active tab under that approach.

I handled that issue in this commit

I don't know, this seems kind of a half-ass approach. I wrote this commit that adds progress listeners to all tabs already open.

Sounds great, I think that it's just in the Firefox xul that has one tab defined by default, I think they removed this in later releases, but your method is definitely more robust since there could be some variety of modification between different releases and no guarantee about when browser.js will execute that I'm aware of.

I'd love to see an example of what you're talking about with XHR like with log.

here is a patch to fix the issue for GM_log..

I'm not sure this should be changed for GM_xhr, the string received from the userscript is passed along to a standard component, why should GM mess with the string and not the userscript?

Hehe was just about to post something like this, except slightly different...

You should be able to pass flags in. I usually pass in -p --no-remote to GVim, so new files open in a new tab within the same editor instance. Firebug implements this quite well I believe.

That's a good idea too. That belongs in #1012 I'd think, it's part of making the editor configurable.

I don't know about a bar, but on the generic level of being able to overview just scripts shown in the monkey menu for a page (without being a power user), I want something. That far, it sounds more like a default-checked checkbox "only show scripts that could run on this page", when opening the manager from its menu entry in the monkey menu to me.

As I think we'll toss around lots of ideas in this ticket, let's call that one 1152A, in the interest of addressability in discussion. Label other propositions similarly when you introduce them. We can short them to A, B, C, ... in this ticket, but on GM-dev and in other tickets, should references show up there, kindly use the whole id.

Is the <= 3.6.* angle about "> 3.6 has a search field in the addons dialog", that we don't yet support the > 3.6 dialog, and/or about something else (like "moz 4 already comes with a search field we'd adopt and reporpose")?

A filter bar sounds like starting out from an over-engineering angle, but ui mocks, ideas and opinions are invited (in about that order :-).

This seems a little related to issue #1122 which is about adding a search box. Stylish has both a search and a filter dropdown box. I'd like to have the ability to filter scripts in the addons dialog.

So my proposal is 1152B, which is adding a dropdown box that can filter the list of scripts for certain criteria. For instance: scripts running, enabled scripts, disabled scripts, alphabetically sorted by name(which I think is way better than the permanent sort in the context menu we currently have), installed scripts(obviously the default), recently installed scripts, scripts with available updates(when we get that implemented), etc.

patch, on this branch (compare to master) for 1152A.

I don't know about a bar, but on the generic level of being able to overview just scripts shown in the monkey menu for a page (without being a power user), I want something. That far, it sounds more like a default-checked checkbox "only show scripts that could run on this page", when opening the manager from its menu entry in the monkey menu to me.

With my 1152A patch it's be easy to add a 'Manage Page's User Scripts...' menu item to the GM statusbar and/or toolbar menus.

Is the <= 3.6.* angle about "> 3.6 has a search field in the addons dialog", that we don't yet support the > 3.6 dialog, and/or about something else (like "moz 4 already comes with a search field we'd adopt and reporpose")?

Yes and yes; I'd like to leave that discussion for another ticket if possible or just have it be part of issue 1123.

A filter bar sounds like starting out from an over-engineering angle

It's a necessity imo.

This seems a little related to issue #1122 which is about adding a search box. Stylish has both a search and a filter dropdown box. I'd like to have the ability to filter scripts in the addons dialog.

Sorry sizzle I totally forgot about that issue, although I must've seen it at some point. The two tickets do appear to be addressing the same issue, perhaps we can keep this one since a discussion has started?

So my proposal is 1152B, which is adding a dropdown box that can filter the list of scripts for certain criteria. For instance: scripts running, enabled scripts, disabled scripts, alphabetically sorted by name(which I think is way better than the permanent sort in the context menu we currently have), installed scripts(obviously the default), recently installed scripts, scripts with available updates(when we get that implemented), etc.

Filtering by enabled/disabled, recently added, and updates available are all simple filters to add to my 1152A patch. The 'currently running' filter would take some work in order to figure which scripts are currently running, but with that then this would also be easy to add.

alphabetically sorted by name(which I think is way better than the permanent sort in the context menu we currently have),

We could add another drop down for 'Sort By' with which change the display mode such that menu items for moving scripts are not displayed, with values "name" and "order" for example. This could be another ticket imo though.

We could add another drop down

I'd much rather have just one dropdown to keep the interface simple. I'm thinking it would look like this:

• Show scripts:
• installed
• sorted
• enabled
• disabled
• running
• with updates
• recently installed

I'd much rather have just one dropdown to keep the interface simple. I'm thinking it would look like this:

I don't like this because users can't see the user scripts with updates sorted or not sorted. Also, when they are sorted I don't think the user should be able to move them, and I think it's less clear why these options are not available with one drop down.

I don't like this because then you can't see the user scripts with updates sorted or not sorted.

Perhaps just making the list of scripts sorted should be the separate checkbox.

Perhaps just making the list of scripts sorted should be the separate checkbox.

Hmm, do we want to sort by anything else? date installed maybe?

Actually I'm +1 for using a checkbox to sort.

With my 1152A patch it's be easy to add a 'Manage Page's User Scripts...' menu item to the GM statusbar and/or toolbar menus.

Here is a commit which adds a "Manage Page's User Scripts..." menu item.

I'm very -1 on anything that involves large complex UIs, setting states that cannot clearly be undone (externally applying filters?) or sorting in any order that is not the order scripts are running in.

If there's any way of applying any kind of search/sort/filter/etc, I want to have an obvious way to remove it as well. I can't right now imagine a clear UI for that, in the case of "filter for this current page".

Enough that, given the sort of discussion happening above, I might be inclined to propose a WONTFIX on this, and simply implement #1122, as a simple live filter. For any more complex ideas, I'd really like to see a proposal for how the UI works, before code.

"leading (and trailing) slash means pure regular expression"

Any word on if, or which version this will be implemented in?

"leading (and trailing) slash means pure regular expression"

This is something I would like as well, it would mean that the pattern would not need to be converted into a reg exp for 1, and 2nd it will mean that a far fewer number of regular expressions will have to be executed against the location.

The set of 36 thousand scripts I have from the survey I ran back in September has exactly one script with an include or exclude that has a leading and trailing slashes: one that has the entire script all in many @include lines, including some / ... / lines.

I call that zero backwards compatibility issues, and a feature win that I have wanted for a plenty long time, so I'm planning on putting it in 0.9!

sweeeet =]

I forgot reason 3: the regular expressions executed will be faster, because they are smarter.

And reason 4: the include patterns will be more accurate

Hey,

I took a swing http://github.com/erikvold/greasemonkey/commit/3bfdae96809290f2f9fc2d6f19014216995e39f8

I did my testing by modifying http://gist.github.com/303782

I'd like to vote against implementing @match, the rationale seems to be that Google wanted to be able to tell users which domains the userscript being installed runs on, so they created @match which is more restrictive than @include such that they will be able to deduce the domains the userscript runs on.

I agree w/ the rationale (it's good to show the user the domains the userscript is requesting access to), and I think that userscripts should only be able to opt-in to domain access, and not say they want access to everything accept domains x, y, and z; that will add to a user's confusion, because saying it needs access to everything is enough. This is why I think that they didn't @exclude (or did they?)

So I'd like to suggest a new metadata attribute @domain (or whatever else you think is better) to be a list of domain host names, one @domain for each domain, and the optional @domain *.

This could hit 3 birds with one stone I think. Firstly no need for @match. Secondly w/ some work we could reduce the number of user scripts that have @include/@exclude patterns compared to a page's location (on page loads) per request by indexing user scripts (or simply putting them into domain 'buckets'/folders) based on the domains they request access to. Thirdly it could be used for GM_xhr in issue #1052.

Thoughts?

If you are wondering what I have against a having @match implemented, well if you have a userscript with say 5 @include patterns, and a couple of @excludes, or one ~complicated reg exp :) and you decide to implement @match patterns for better compatibility w/ chrome, only that ended up being about 13 @match patterns.. well when you run that userscript in GM pages that the userscript does not run on will have executed 13+(1 or 7) regular expressions against page locations per request, because GM will execute both @match and @include (like chrome does atm...) which is slower.

I'd like to vote against implementing @match

I'm not aware of any GM developer's plans to implement @match today.

So I'd like to suggest a new metadata attribute @domain ... Thoughts

Complicated, and for little benefit. I don't like it.

As a big part of GM 0.9, I want to significantly simplify the UI for the average user, and encourage the sort of users that want to know things like what domain a script might run on to inspect the actual source. The vast majority of users won't read this list anyway, so it's potential value is small.

And remember:
http://www.greasespot.net/2009/11/greasemonkey-api-usage.html
At least of scripts that use GM APIs, the vast majority only affect 1 (~75%), 2, or all domains (~9% each). The list of includes will already communicate this about as plainly as it can be communicated.

@domain is complicated? I didn't think of that.. even tho authors can ignore it? meaning @domain * by default?

The benefit grows the more you use greasemonkey.

What domains would @include /.[jksgdfh][oasjdfh][jkaogkjb][jkagdnhfjk][ajsdhfksdfl][asghkfejk]./i run on?

btw I created a new branch in my repo, w/o any of my other changes, and just the @include reg exp update, which you can pull from http://github.com/erikvold/greasemonkey/commit/f2ecddecd632256264825ae2d81558574e5fd5f5 if you wish.

So I've added a check to every downloaded "script" to see if it begins with <!DOCTYPE. If it does GM will not try to install it, and will load the page in the browser instead.

Fixed with http://github.com/Tobbe/greasemonkey/commit/9fda98c09dd1a6eedad80e1be1ec477c80ceb80b followed by http://github.com/Tobbe/greasemonkey/commit/91b6a83e39a5243a4d9a36dccb75fdedd1708c85

I know this will probably not catch all web pages, but I think most modern pages begin with a doctype declaration. Could also add a check for <html> I guess, but no matter what we add we will never be able to detect all web pages.

Probably better to check the mime type in my opinion. Should only accept text/javascript and application/javascript files.

The window watcher, last time I checked and commented on devjavu before it went down, wasn't returning the correct content type... thus, last time I checked, this was definitely a Mozilla issue. I am still probing around the FF source to see what's up... but this time of year I get super busy. Even if the Moz engineers fixed this, it wouldn't be backward compatible in the browser. :\

http://example.com/myscript.user.js?

I realize the <!DOCTYPE check isn't ideal, but does it hurt to have it? It's backwards compatible in the browser.

I'm not writing a website to host user scripts. I'm using github to host my userscripts, and github has this problem. I could write a GM script to append the ? to the end of the url, but I think this is a more general problem that ought to have a general solution.

I generally agree with this. The reason it hasn't happened earlier is that it has potential backwards compatibility issues. We could take the route that Firefox does for XPI installs: site owners must set the proper MIME type. That would have been good long ago, but now could cause big problems, there might be lots of places out there that would break this way.

Thus, black-listing "known bad" values seems the best approach. Any text/html documents are really web pages, almost certainly. The doctype check is also quite interesting.

P.S. Tobbe: Thanks for contributing. If you want to increase the chances that your work is taken and used: always make a potential change in a branch. As far as I understand git/github so far, it's really easy to pull in all the changes made in any branch (like the two you mention above, perhaps) but much harder to pick and choose things not contained in a branch.

Yes, I've made my changes in a branch. The branch contains only those two changes. You should be able to just pull it in.

So what's the status of this now? Do you want my <!DOCTYPE "solution" but just couldn't pull it in cleanly, or do you rather just go with the mime type check? If you do want my patch I'll try to reapply it against the latest code as one commit in a new branch of my fork

I'm not sure.

A) I expect it could be pulled in as is, with manual conflict resolution.
B) I'm (personally) not excited about the way that it will repeatedly bounce the request back and forth, downloading the resource more than once (probably?) and worry about race conditions.

I've still got to think about it a bit, but I'm pretty sure I'll vote for filtering on MIME type,

I agree with this, and also with checking the mime-type and not the actual file.

I've put a patch together which checks the mime-type of the file 0b06c3f.

Github is disappointing for a variety of reasons. In this case:
Please paste the entire URL to a commit, when discussing proposed commits. In this case, said SHA hash does not exist in the greasemonkey repository, so the automatic link created from pasting the SHA:
http://github.com/greasemonkey/greasemonkey/commit/0b06c3f945421ba272cb883b007c62cfab2b0baf
is broken. Only the full link to your repository:
http://github.com/erikvold/greasemonkey/commit/0b06c3f945421ba272cb883b007c62cfab2b0baf
works.

That said, I'm very wary of this change. I really don't think removing the progress indicator is a good idea (otherwise things can look broken when, i.e., a big @resource, or just a .user.js from a slow server, is downloading but there is no indication).

I've made a branch for this http://github.com/erikvold/greasemonkey/tree/issue-1049 sizzlemctwizzle updated the last commit I mentioned so that the header was checked sooner, when readyState = 2, and after considering what you said Anthony:

I really don't think removing the progress indicator is a good idea

I revived the progress indicator, and updated it so that the "Fetching user script" msg was taken away after all of the dependencies are dealt with, instead of after the user script itself is done downloading.

This branch seems to work pretty well now.

I can confirm that erik's finished branch, now that the progress indicator is back, fixes this issue rather nicely. It's probably also important to note that issue #1099 relates to this issue.
Here is a good sample link to try it out on: http://code.google.com/p/gmconfig/source/browse/trunk/userscripts_comments_fix.user.js

Here is a test case for the progress indicator (a small user script with heavy dependencies):

Before changes in my branch:

• because the user script itself is small and the dependencies are large, the user will not see the "Fetching user script" msg unless their computer is slow or the connection to github (ie: the host of the user script) is slow.

• the install dialog window is displayed instantly after the user script has downloaded, but because the dependencies are downloaded in the background when the user clicks 'install' nothing happens until the cat2 resource fails to download, in which case the user is finally notified that the user script failed to download, because a resource returned a 404 error.

• (not tested, a guess so far) If there is an error downloading the user script (ie: it does not complete the download), then the user is not notified, and the "Fetching user script" msg must not get hidden, and I guess it is stuck..

After the changes:

• the install dialog is not displayed, and the "Fetching user script" msg is not hidden, either until all of the dependencies are found, or until there was an error getting one of the dependencies, at which point the window is displayed, then if there was an error and the user clicks 'install' they will be notified instantly of the error.

• (not tested, a guess so far) If there is an error downloading the user script, then the user is not notified, and if the readyState reached 2, then the "Fetching user script" msg must not get hidden, and I guess it is stuck..

Well I've gone ahead and created a patch for this issue, take a look and see what you think, it needs work, but works.. 8156c43

This patch accepts the @icon metadata key, which can be a data uri scheme, or a url to a file which is downloaded when the userscript is installed. The image should be 32x32 but anything seems to work so far..

I'd be -1 on this sort of change at least until scripts-in-addons is finalized, so that we have some sort of idea how it might work, if at all.

just added a default icon for when @icon is not defined: bd22087

erikvold wrote: until what is finalized?...

Most likely #1012 as well... and most likely it should work but it needs to follow the guidelines for em:iconUrl.

+1 for when the addons tab is completed for all supported platforms but 0 for now.

Just so everyone is aware, IEScripts supports // @defaulticon ie7pro://pluginpath/bg.png in case GM doesn't want to completely reinvent the wheel. (e.g. @defaulticon URI)

Just so everyone is aware, IEScripts supports // @defaulticon

cool, thanks for pointing that out, do they have any documentation on it that you know of?

In my patch the icon size can be anything, so assuming that ie7pro's @defaulticon is not meant to be 32x32 (although it may be idk) then it should still work as an icon for GM. So we can adopt @defaulticon (which I don't want to do), or we can accept @defaulticon as a secondary choice (which I prefer), or we can ignore it.

I'm all for "reuse" but this is a matter for Johan and Anthony to decide on the name... adding yet another key/value that uses the same standard that IE7Pro defined (and I think trixie might do this too) seems a bit redundant. I do know that the IEScripts website utilizes this key to display a script icon, but it can easily be turned around to show in the GM addons tab (when finished) as well. The rules for images should probably follow em:iconUrl as most likely the "reusable code" that will probably be used in the addons dialog will require this or at very least resize it to a 32x32 matrix. Check with MDC here.

Here is a clean branch for this update.

Here are two tests:
Test 1 Test 2

@sizzlemctwizzle (or anyone else interested) can you please review it to ensure that I integrated this branch with issue 1092 correctly, when you have a chance? I've tested it and it seems to work, but I'd like you to double check if possible.

Ran into another error from the live meta portion. Here's the patch. With that cherry-pick applied to your branch, the live meta seems to work properly and without any errors.

Ran into another error from the live meta portion. Here's the patch. With that cherry-pick applied to your branch, the live meta seems to work properly and without any errors.

This should be applied upstream right? (ie: applied to the master branch). If so, then request a pull from via the gm-dev mailing list, the GitHub pull request system has many faults.

Never mind, I see you mentioned this is in issue 1092 already.

Thanks for reviewing this sizzle! very much appreciated. I'll look into using ScriptResource as you suggested asap.

erikvold wrote:
... interested...

Not too shabby although needs to support QSPs (?yadda=yidda&that=this&this...) on urls such as the gravatar... currently if a query string is used it reverts to the standard script icon. Fragment (#) support would also be a plus. :) That's a big puppy too ;)

Not too shabby although needs to support QSPs on urls such as the gravatar... currently if a query string is used it reverts to the standard script icon.

Yeah, I didn't spend much time on the code used to detect if the url provided is actually an image, that can certainly be improved. I could update the reg exp to support query strings, but there is probably a much better way that I haven't thought of yet.

erikvold wrote:
probably a much better way

Have you tried a simpler method, maybe new Image(), instead of using the GM internal API's and throwing junk at it too to see how it responds? I don't recall how far back Image object goes but might be worth a try.

Have you tried a simpler method, maybe new Image(), instead of using the GM internal API's and throwing junk at it too to see how it responds? I don't recall how far back Image object goes but might be worth a try.

No I haven't.. why don't you give it a try? let me know how that goes, I'll be interested in what you discover.

erikvold wrote:
let me know

Just fooled it with a bogus QSP though... lots to check out including when Image object was implemented ... if it was just recently then there is no point to exploring that avenue.

Well I found a vague reference to it being supported in IE4+ and NN3+ and of course FF, but Image appears not to be a valid Object in chrome: for some reason.

Dove into the logistics of how this is saving the file and I'm not real thrilled with it keeping the original filename since there can possibly be an exploit via a fragment or a bogus QSP and essentially kill any file... say test.user.js is the script and the icon is @icon http://example.com/test.user.js#.jpg and all the sudden we have potential catastrophic failure. Even if the save routine renames it... it would have a false identification. This could also conflict with @require filenaming as well.

There is always the XPCOM interface library to load an image but I'm not sure anyone has dealt with that on GM before. Another alternative is to create an invisible XUL image tag and set it's src in JavaScript and if it fails then it's not valid and skip ahead to the default image... although this is sort of a cheat.

If the @icon file is downloaded like a @require/@resource(@resource seems like a better solution to me), it needs to be integrated with the live meta code. This means that it needs to be included in the dependhash and when it changes it should be redownloaded(along with the other dependencies).

Isn't it? I thought I had done this..

Could be. I didn't see it the first time I looked. I'll have to inspect closer when I get home.

Nvm, looks good. You should probably do a check of channel.contentType here like you do on the data:uri. I'm against checking the extension of the url for this. Content type seems like a more flexible solution.

Content type seems like a more flexible solution.

Yeah, I agree.

Content type seems like a more flexible solution.

patch

Can you link to (and create if necessary) a branch that is synced to a recent version of upstream's head, and contains just the changes for this issue?

here's my issue-1096 branch, I just did a pull of GM/master.

It would be a really good idea to get written permission from whoever owns the copyright on that icon, before we consider taking it.

Please check the style guide, I see at least a few if( with no space.

I don't like the way data: URIs are handled, checked for and mocked differently multiple times. Perhaps a new ScriptIcon object, extended from ScriptResource to handle this logic in one place, consistently?

I see private members being accessed outside their object, this should be avoided.

I don't like the way data: URIs are handled, checked for and mocked differently multiple times. Perhaps a new ScriptIcon object, extended from ScriptResource to handle this logic in one place, consistently?

Corrected now, see compare view of branch here.

I see private members being accessed outside their object, this should be avoided.

I corrected this to the extent that GM seems to follow this rule, script private members are accessed copiously w/in config.js and scriptdownloader.js atm.

+1, but as a long term feature, definitely after 0.9 is released and compatible.

Very -1 on the idea of ever automatically refreshing any tab(s) that the user has loaded.

discussion from gm-dev: http://groups.google.com/group/greasemonkey-dev/browse_thread/thread/9a2cbd3e552d4ba9

Very -1 on the idea of ever automatically refreshing any tab(s) that the user has loaded.

I'm going to remove this from the description, as everyone (including me) dislikes it, I was merely echoing the concern.

Simple test case: http://gist.github.com/164761
Complex test case: http://gist.github.com/164763

See also: http://greasemonkey.devjavu.com/ticket/106

GM_apiLeakCheck() is designed to stop JS calls from page land and that's precisely what it does, no matter the call has been piped through the event machinery or not. It probably makes sense at the Mozilla level to process it this way and i wouldn't expect it to change in the foreseeable future -- although it never hurts digging a bit deeper...
Whatever, there'd still be a huge user-base with older FF versions suffering from this issue.

We'd like to consider calls through the event machinery as "indirect calls" and allow them, but there's nothing in Components.stack or Error.stack about that. Perhaps there's another way to tell...
I believe Firebug is able to detect that a script is an event script, but i'm not really sure how, nor if it would be usable in GM.

There is a quick solution to this. It's far from being perfect, but it has to be mentioned nevertheless: we could add a GM API function that would do the wrapping, pretty much as demonstrated in the wiki.
For example:

function GM_listen(/**nsIDOMElement*/ element, /**string*/ event, /**Function*/ callback) {
    element.addEventListener(event, function(e) {
        window.setTimeout(callback, 0, e);
    });
}

The historical position (and the one I agree with) is to not introduce GM functions that don't have to be included to i.e. grant a privilege the script would not normally have on its own. Every extra GM function added makes scripts less inter-operable with other user script platforms.

This function could easily be developed and shared via an @require . Greasemonkey needs to come up with a good way to share useful @require-able code.

Frankly, i think this is a good rule of thumb but one should be careful not to push it too far...
For example, suppose there was some piece of code all scripts were using; then i believe it would belong to the GM API, not to some tiers library each and every script would @require, even though it's technically possible to implement it outside of GM core.
A real-life example would be GM_addStyle: if you remember, i've published uso statistics a few month ago showing it almost was the most frequently used API function (it probably is with other metrics).

Back to our issue, there may be a security concern here that would be worth a GM API function. Basically, piping through a sandbox setTimeout is enough to fool apiLeakCheck. In other words, a page-land object (possibly trapped) may slip through the net and end up in GM core.

Example of apiLeakCheck bypassing:

unsafeWindow.gmPrivilegeCall = function(/**string*/ method) {
    method = eval(method);
    var args = Array.prototype.splice.call(arguments, 1, arguments.length)
    setTimeout(
        function() {
            method.apply(null, args);
        }, 0);
}

UnsafeWindow is unsafe. It should never be used. Since, unfortunately, certain things are impossible without it, it exists. We can't make it safe. You can only be safe if you can't use it. In this example, you have done exactly that. Real scripts should not do that.

If you (a script author) intentionally try, you can use it to create an unsafe situation. Thus the name. You should never use unsafeWindow. The cases where it is legitimately needed are very rare.

Argh ! I bumped the 'comment and close' button by accident ... and there is no reopen button?

Heh. Reopened via the API. But fixed.

We can't make it (unsafeWindow) safe. You can only be safe if you can't use it. In this example, you have done exactly that.

I'm afraid you missed my point here.
This is not an unsafeWindow exploit: unsafeWindow exploits only allow to reach userscript content, not the GM API. Besides, it could have been triggered through an event listener just the same.
It is a setTimeout/apiLeakCheck exploit: API calls from page-land should be stopped by apiLeakCheck, they are not when piped through setTimeout.
What you have called a "standard workaround" in this ticket description actually is a security leak that may serve evil purposes just as well as legit ones.

My original point stands. If you accept arbitrary script contents from unsafe land, and execute them (eval), then you as a script author have created the hole.

While it's true that this is a potential security hole, it is not one that we can generally "fix". We can only protect the script author "so much". There are, however, discussions about how to move forward that would help prevent situations like these. It's nice to know all the potential problems, to check that a solution addresses all of them.

If you accept arbitrary script contents from unsafe land, and execute them (eval), then you as a script author have created the hole.

I'm afraid i have to disagree here:

• In the above code, eval only serves to map a GM API function name (string) to the corresponding Function. It could work just as well without eval -- although it would be less generic or would require another mechanism to select the target function.
• Normal page-land calls (i.e. without setTimeout) trigger apiLeakCheck, whether they use a sandbox-scoped eval or not.

While it's true that this is a potential security hole, it is not one that we can generally "fix"

The only "solution" i can see so far would be to remove all references to setTimeout from the sandbox (i understand it would break an unknown number of existing scripts).

If you accept arbitrary script contents from unsafe land, and execute them (eval), then you as a script author have created the hole.

Although i believe this comment does not apply to the featured code for the reasons explained above, you are of course absolutely right that using a sandbox eval on a page-land string would trick apiLeakCheck too -- whether said string has been acquired via a XPCNWrapper or not.

The wiki states:

it should not be possible for a user script, either intentionally malicious or accidentally problematic, to cause either security or privacy problems for the end user

From the above discussion, given the current state of GM and apiLeakCheck, it seems that trying to do our best in this direction would require removal of both eval and setTimeout from the sandbox. :-/

there may be a security concern here that would be worth a GM API function

In other words: if the current setTimeout was removed from the sandbox for security reasons, there would be room for a GM_listen API method (which would wrap the event callback in a setTimeout). It would so abid by GM policy regarding new API functions.

A) That page is very new, still under discussion/development, and bears a large and red disclaimer at the top.
B) It says "should". We live in an imperfect world, and must often make compromises. I'm not sure if we could even remove eval or setTimeout, if we wanted to. Both would severely limit real scripts (eval less, but still so). The best option (IMO) is to trust the script author to be responsible, rather than removing critical tools.

That page is very new, still under discussion/development, and bears a large and red disclaimer at the top

Sure, i'm trying to evaluate the situation. :-)

The best option (IMO) is to trust the script author to be responsible, rather than removing critical tools

I agree.

Webmonkey has some commit's that I could manually cherry pick, but I don't see a license mentioned, assuming it's the same as GM though (MIT) can I just cherry pick those commits? what else would need to be done if anything?

I just looked over the MIT license and based on:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

It seems to me like we'd have to include an additional copy of the license? since the Copyright holder is different, am I wrong?

So from what I've gathered from the the other conversation, this issue is about making the greasemonkey service a javascript code module?

I doubt we'll have licensing issues since the author of Webmonkey is also a Greasemonkey contributor. We can just ask him to use the code.

So from what I've gathered from the the other conversation, this issue is about making the greasemonkey service a javascript code module?

Well I don't think that we can get rid of the service altogether, but we can get rid of the interface definitions, use XPCOMUtils.jsm, and move some stuff out of /components/greasemonkey.js and into js modules.

For someone like me, who's total knowledge of JSM comes from reading MDC, could you clarify what exactly would be the benefit of using this in Greasemonkey?

could you clarify what exactly would be the benefit of using this in Greasemonkey?

"Modules can also be used to create global JavaScript singletons that previously required using JavaScript XPCOM objects".

In other words we save memory.

Also using XPCOMUtils.jsm greatly reduces the amount of XPCOM goop in the service script.

Alright this branch is at least a start. It takes care of #1028 as well.

I got a response from Olivier:

I did not make any license change to WM in order to make any back-porting to GM legally straightforward. Thanks for asking: you are very welcome to cherry pick anything you deem useful in the current code.

...

Afaik, the license only enforces that dev credits should be kept.

Thank you for asking him about it; it's the polite thing to do, even if license would have been in a less fuzzy state.

Also: really excellent work. Is this branch in or approaching a state of mergeable to master? (Not trying to rush it, or anything; I just really really like the direction of this, and it cleans up so much grit.)

Is this branch in or approaching a state of mergeable to master? (Not trying to rush it, or anything; I just really really like the direction of this, and it cleans up so much grit.)

Yep, it's done, should be reviewed though, but I'm pretty confident about it (still needs to try FF 3.0).

One thing that should be discussed:

The GM_apiLeakCheck function needs to be updated. Before these commits, new instances of all of the GM_API methods were created within the GM service's scope, so all the GM_apiLeakCheck method had to really do was to check if the stack.filename wasn't equal to the GM service's filename (and a couple other allowed cases). Now, after these commits, the GM_ api instances that are used are scattered is multiple JS modules, so GM_apiLeakCheck needs to allow more filenames.

There were a couple of options that I saw to deal with this:
1. Move all GM api methods (which call GM_apiLeakCheck) in to the same file as GM_apiLeakCheck.
2. Move all GM api methods in to the GM service file
3. Keep the current file separation, and modify GM_apiLeakCheck such that:
3a. Any file from the GM modules folder is allowed and the GM service file.
3b. Any file from an array of filenames which is dynamically updated on startup is allowed.

I went with 3b, because I thought that this could be made into a useful api for other extensions to use to extend GM with, so far it's not useful in that sense though, and could potentially allow files that shouldn't be allowed if a nasty script/extension is able to access this method (a bad jetpack/ubiquity cmd/user chrome/ff ext etc..). So I'm thinking 3a should be implemented now..

It's not difficult to make the changes required though.

For what it's worth Webmonkey does 1.

So what're your thoughts?

How about any file from the GM extension directory is allowed?

How about any file from the GM extension directory is allowed?

works for me

FYI: I used the technique described here to measure the effect of this branch on startup time (using my winXP machine).

Roughly speaking the startup time for GM alone was about 60ms before this update (for a profile with a dozen user scripts), after this update it is down to about 30ms.

Also: really excellent work.

Thanks =]

I've got this issue-1156-mods branch which seems to knock down the startup time another ~10ms or so (want the data?). It should also save some more memory, especially when GM is running but disabled (monkey is in statusbar but gray), because a config instance won't be created.

Alright, I think I solved the GM_apiLeakCheck issue nicely with this commit. I just created a facade for the GM_* api.

So after that last commit that I mentioned, I realized it'd be a simple change from there to making sure that instances of GM_xmlhttpRequester, GM_ScriptStorage, GM_Resources, and GM_ScriptLogger are not created until they are actually needed (and then cached).

Here's the patch, on a new branch issue-1156-api-mods.

I think I'm very +1 on this idea. I definitely need to do some more reading docs before I'm sure, but I don't expect to find anything terribly surprising. I am, on the other hand, wary of trying to do this all in one giant commit/merge. A lot of code is potentially going to change.

Is the wariness from wanting to see the codebase morph into post-1156 more slowly, one patch at a time (as Erik and Olivier did while bettering the old into the new, one step at a time), or something else, like thinking of merge commits like they work in svn (where that would appear as one changeset +LOTS/-LOTS point in history with a mostly worthless commit message)?

I think we could rebase this branch to master and then pull it in via rebase, if it's the merge commit (that arguably is represented somewhat confusingly in github, IMO) you don't like.

I hear you on the read-up, though -- I haven't initiated the process for the same reason. I'm quite confident that the code looks good, though I also think it'll need to marinate for some time on master with public scrutiny afterwards to shake out the last yet-unspotted gotchas bound to show up.

Any news if and when this branch will be merged? I'm quite eager to start using it. Although I realize this change is massive and needs reviewing so I'll be patient. I'm holding off on #1139 so I don't end up spawning a ton of conflicts.

I vote for a merge instead of a rebase since it seems to muck up the history(creating duplicate commits). But obviously if rebase makes it easier to pull in the changes, by all means use it.

I would expect to try to merge, and find it difficult or impossible. Rebase gives you the opportunity to merge conflicts at each commit, which is much easier for big changes like this.

As for timeline, I personally would expect a release order of:

• 0.8.7 for Firefox 4 compatibility, if timing proves it required (since it's still got the old manage dialog, the addons stuff doesn't need to be ported, probably the component registration goop is all that has to be fixed)
• 0.9.0 mostly as is, perhaps plus only what is required for Firefox 4 compatibility
• Something after that, including nice-but-not-necessary changes like this.

just let me know when you are ready to review and possibly pull the branch, and I'll rebase it, unless you'd like to.

Take a look at: http://mxr.mozilla.org/mozilla/source/browser/components/microsummaries/src/nsMicrosummaryService.js#1696

Using an iframe is a relatively 'hackish' way to tackle the problem, and some more investigation is probably due.

Implemented in this commit: http://github.com/Tim-Smart/greasemonkey/commit/5a9362dd9e23de853686d3630b3778ac9761411b

Still reasonably rough, needs some attention in area's of security most likely (I haven't developed extensions at all really), but succeeds in parsing HTML and XML.

Only limitations is that it requires a callback, due to the asynchronous nature of the iframe method used.

I have given this issue a shot a few month ago but did not push as far as to get a working solution.
Actually building the XML document is not a problem, however importing this document back into the sanbox throws an error because of same-origin security checks. The GM Sandbox is initialized with the userscript's target nsIDOMWindow, thus any document coming from another domain will not pass these tests.
I guess it would be interesting to dig deeper in this direction...

EDIT: had not seen your last post when i posted this, Tim.

Is this ticket about implementing the equivalent of XMLHttpRequest.responseXML or is it about implementing a method that would also parse successfully non-XML compliant HTML?

method that would also parse successfully non-XML compliant HTML

I was certain this is what we are trying to achieve when he mentioned new API for parsing a string of HTML , and that is what I have done.

XMLHttpRequest.responseXML is another kettle of fish in my opinion.

You're most probably right. I must have confused it with another DevjaVu ticket and the frequent user question of why GM_xhr does not provide responseXML. My bad. :-)

To parse non-XML compliant HTML document.implementation.createDocument can be used to create an empty document. I have an example of using this method to return a DOM from an xhr request here: http://userscripts.org/guides/9

While document.implementation.createDocument is an option, it has several limitations that I won't go into detail about. But it is possibly worth a look.

EDIT: Also stabbed at implementing responseXML: 1266792

Also stabbed at implementing responseXML: 1266792

I believe this issue is worth opening its own ticket (also made a comment on your commit page).

Idk... This isn't really a big issue. The workarounds we have from within the current context GM are fine. Besides, forcing scripters to find other ways of handling response xml / html is the way to go.

forcing scripters to find other ways of handling response xml / html is the way to go.

What do you mean?

I mean, don't make it easier for them to work with it. Creating whole objects out of XML is slower, and if they used regular expression or just regular string methods, it would be better.

I think of it in terms of this being nothing special: we were able to do it before, and it wasn't that great anyway.

I personally think this would be a useful feature. There's a few proposals, but I'm currently unsure which works/works best. So tagging for v.next; I intend to research this a bit, and if it proves infeasible or too difficult it may be postponed.

By the same logic with which you guys shot down my issue with GM_xmlhttpRequest + POST ("you can already do it, you just have to know what to do"), this should be abandoned as well.

I quote:

Making browser APIs comfier is the domain of @require libraries.

I vote -1. The benefit is minimal (saving one line of code?) and it's a perfectly normal JS/AJAX thing that a programmer needs to know.

Btw, it's not just saving one line of code. Maybe I'm an idiot, but it took me a long time to figure out why the hell my POST requests weren't working at all (I had about 3 months of experience in any programming at all at the time).

On the other hand, I figured out how to "parse a string of html" almost immediately.

I could easily be wrong. Show me the user-script-land code that can start with a string of HTML, and give you a DOM document, and I would agree. Unprivileged scripts can only parse valid XML though, to my knowledge.

Tim-Smart's implementation is the only one I've seen that seems to be able to (I haven't built and played with it yet to verify its capabilities and/or pitfalls) get a non-clobbered DOM (including things like script tags, head, body, html nodes and their attributes) that can be queried with the DOM tool chain (querySelector, XPath, getElement(s)By... et cetera).

The workarounds available work for trivial cases only, where a clobbered DOM is okay, and manual stuff using regexps will be orders of magnitude slower than browser native DOM generation code, if they are correctly implemented, and trivially broken when not.

I'm also +1 to this kind of feature, despite having pragmatically settled in my own user scripts for a @require http://ecmanaut.googlecode.com/svn/trunk/lib/gm/wget.js which creates an iframe after having massaged away as much dangerous javascript as it can from the HTML string.

give you a DOM document

I'm pretty sure it's possible, and even if not, I think this whole feature would be overkill. The most important thing to realize is that it won't enable us to do anything we couldn't ultimately do before.

How about doing something like giving us the ability to automatically download files (with custom file names of course)? I've implemented this with the little experience I've had in my extensions.

@Johan:

Maybe I'm missing something, but can't you just do something like:

var tempDiv = document.createElement("div");
tempDiv.innerHTML = someResponse.responseText;

// now you can:
document.evaluate(".//a", tempDiv, null, 6, null);
tempDiv.getElementsByTagName("a");

// etc... ?

What does building a fully fledged DOM really give you?

Quoting above and elaborating further: intact script tags (and their proceedings -- so <script>document.write('x');</script> comes out as such, without any trailing appended "x" -- which IIRC happens with my own wget.js hack), as well as intact head, body, html nodes and their attributes.

Example:

var div = document.createElement("div");
div.innerHTML = '<html lang="en"><head><title>ow!</title></head><body background="red">Ugh</body></html>';
alert(div.innerHTML);

...will show you <title>ow!</title>Ugh. A fully fledged DOM enables you to use absolute XPath references (/html/head/script[3]/@src for instance), discern where in the document a node is, find contents, attributes et cetera even of nodes that get squashed when doing the innerHTML hack, introspect at un-executed javascript code (especially inline code), and so on.

It's quite probable that you yourself won't ever need or want any of these properties, but others (both more and less advanced scriptwrights than yourself) do.

Creating document from string is available without GM API.
I recommend this article http://nanto.asablo.jp/blog/2009/10/29/4660197 (Japanese),
and this code snipet, http://gist.github.com/164430

Removed the 0.8.3 tag. This seems controversial. I hope to take a look at all the possibilities mentioned and move forward from there.

Like many other good tools, it would be a lot more beneficial having an html DOMParser implemented in the underlying browser than in Greasemonkey middle-ware.

Looks like Tim-Smart has worked on this:

3d9610d
be796a2.

I vote a wontfix on this, because I haven't the faintest idea how to fix it.

Johan: any input?

Same here. I'd welcome patches, if someone comes up with a solution that doesn't just enumerate all the current (and thus, over time, all future DOM enhancements) constants, but tackles the real problem.

Barring external implementations, I think real solutions to this issue would be on the other side of architectural changes like weaning off of the Mozilla javascript sandbox.

I vote -1. There are too many possibilities, that change over time. They are too specific to the author (you might use one template, someone else uses another).

It's not hard to keep a template file for yourself, and copy/paste from it. And this works for everyone, no matter what they might want the template to be today, and what they might want it to be tomorrow.

I want the possibility to add and save my templates but not for some to be shipped with Greasemonkey.

Actually the main issue is that, if I paste a template which require the libs I need, this does simply not work. I have to install this script and not just click on new script and paste my template. So this is really a pain to create a script which dependents on a lib.

Ah I see, the ability to add user-defined templates for new scripts would indeed be much better than my first failed understanding. The incompatibility of today's "New Script" dialog with @require (which is newer) is definitely worth fixing as well.

The incompatibility of today's "New Script" dialog with @require (which is newer) is definitely worth fixing as well.

It's true the new script dialog does not allow usage of several header features like @require and @resource (and perhaps @unwrap). Shouldn't we open a new ticket to deal with this?

That's a very good question. As I begin to think seriously about how to get this dialog to deal with all the possible different possible bits that the user might want to include, I start to realize that templates might be the best/only option.

I'd imagine that the dialog would then become a big (XUL of course, but HTML terms here because they're easy to understand) < textarea >, along with a < select > of existing templates, and a button to save the current text as a template (and delete the selected template?) Then there would be one (or more) default templates, perhaps the defaults would not be delete-able.

Perhaps a "simple" and a "complex" template, the latter with the "minimum" (name, namespace, description, include) that a script "should" use, the latter with all the options (resource, require, exclude, etc) which can be kept or removed. Whatever we end up with, this stub script is installed, and the editor is launched.

I'd imagine that the dialog would then become a big (XUL of course, but HTML terms here because they're easy to understand) , along with a of existing templates, and (...)

I'm sorry, i'm afraid you forgot a couple words in the above sentence. That makes it a bit of a guess game. ;-)
I assume you are proposing to use a text area to print/edit a template (instead of specific input fields as we have now), as well as a list of templates you can select/add/remove from, right?

I'm worried about something: dependencies.
It might be good to use a file-picker to get the target file path.
Also, ideally we'd like @require and @resource tags to feature a relative path (not an absolute local path): it's simpler, clearer and more portable.

I didn't, but markdown failed me. Inserted some spaces so that it doesn't eat my < > tags.

The dialog you are describing sounds great for me !

As a developper I think it will also be great to be able to update @require @resource @include @exclude from an edited file. I was thinking of a "reinstall" button in the "manage scripts" dialog. But as a developper, i think the best would be an option that i would enable when i develop which would reinstall all modified scripts at each page load.
I'll post another feature request for that.

I believe this behavior has no reason to be

I've definitely used this capability to good advantage in the past. The first example I can think of is the script I wrote to scrape your own movie ratings out of Netflix. It works because you're logged in, and can thus access your own pages.

I've been intending to survey how scripts actually use the GM_ apis (including how many actually use the cross-domain abilities of xhr) for some time, but I've hit external snags. I'll try to speed that up, as it would definitely help inform this.

I've definitely used this capability to good advantage in the past.

I'm not sure what exactly you are talking about. Just in case, i'm not criticizing the fact that cookies credentials can be used by GM_xhr, only that all userscripts can access all URI possible without the user's approval.
I fail to see why a userscript would require this capability and, if ever it did, it could always state it explicitly.

XHR cross-domain restrictions are here for a reason. GM_xhr breaks them also for a reason, although the "breach" should remain both explicit and as small as possible.

Edited the title to be more clear, because I keep seeing it and having to re-read it.

I can't disagree with this fundamental idea. I'm just unsure, yet, how to properly implement it. There's definitely some inspiration to be had here:
http://groups.google.com/group/greasemonkey-dev/msg/f3acf7a809365c0a

I think we'd have to do something like listing the host-names that could be connected to. Anything else (if not also this) would be too complex to present to the user, for it to add much in the way of security -- dialogs that users don't read and just click "OK" don't add security.

Eeew. One of the reasons I (and probably many others) love greasemonkey is because of GM_xmlhttpRequest's no-questions-asked-do-as-you-please privilege. The folks at W3C and IETF have great ideas from time to time, but I've never been a fan of the same origin policy. (Somewhat off topic, but I'm just curious, does anyone know how they feel about linking to externally hosted images?). Serious -1 from me on this.

Then again, you guys are super-concerned about security for reasons I have yet to fully understand, because as I currently understand it, it's mainly because you assume the general public is stupid, which is kind of depressing.

If you guys are really going to implement something like this, maybe a one-time question about cross-domain XHR's at greasmonkey's installation would be good enough. Please do not end up doing a ton of questions per script, because a ton of great scripts violate the same origin policy many times during their execution.

As for listing host names, yeah, that sounds good except it would be kind of annoying for you guys to implement and in terms of writing scripts (having to manage a hosts list per script).

I fail to see why a userscript would require this capability

As user "greasemonkey" mentioned, there are tons of scripts which use this feature to fetch external information and bring it to a site. The ones I can think of are torrent sites which can now have an IMDB rating next to the title and a Megaupload script which solves captchas.

Most scripts only "semi-violate" same-origin anyway (sub-domains, or external servers for use with the site). For example, youtube.com uses ytimg.com, and sites like megaupload often use www###.megaupload.

Traditionally, Greasemonkey is used to "fix" a site, wherein the users (rather than the owners/administrators of the site) implement the solution. Setting the "Access-Control" header requires the ability to control and change the server.

Also, of course, GM_xhr is quite a few years older.

-0

I'm suggesting to patch XMLHttpRequest somehow to allow x-domain requests. I'm not suggesting setting "Access-Control" header.

I see. I did not understand your statements.

Since this happened years ago, I don't have an answer for you. I'm pretty confident, however, that it is an accident of history, related to particular design quirks over the lifetime of greasemonkey. Today, it will remain, for backwards compatibility.

Can you produce any evidence that this behavior violates any specifications or documentation in any way? That's how it's supposed to work as far as I know.

I see, according to http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.4 that is how it's supposed to work, even for "HEAD" requests. In that case, would it be possible for GM_xmlhttpRequest to have an option not to follow redirects?

I would very much welcome an option not to follow redirects also. There is precious little point trying to save bandwidth and time doing a HEAD request if the redirect issues a GET.

As GM_xmlhttpRequest behaves as the underlying xhr (which apparently already follows the rfc SHOULD), this indeed isn't a bug.

As for the query about an override option, I wouldn't personally veto an override flag of some sort, but not rush to implement one either.

This is a bug in the underlying Mozilla implementation, which has been open for over three years:

https://bugzilla.mozilla.org/show_bug.cgi?id=343028

I would love to see it fixed, either in general or specifically for GM (which is where it really kills me). Being able to control following of redirects would at least provide a workaround, and be generally useful, as well.

Same problem here, same GM version.

User-agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6

Here is a test case userscript: http://userscripts.org/scripts/show/66759

oh I think this might be user error. I thought accelModifiers and accelKeys defined a global hot key, but it appears to just be a way to quickly execute the command while the menu of commands is open, and the accessKey is meant to help a user scroll through a long list of commands.

So you can have multiple commands with the same accessKey, but only one command per accelModifiers + accelKeys combination.

If this is correct then it appears to work for me.

Can we mark this as won't fix?

This issue definitely bears careful consideration. Something, at least the documentation, needs to change so that it's both clear what it's supposed to do, and that it actually does that.

I wrote a patch here, I'm not sure it's safe but I think that it is.

I get an access denied-error when trying to access any of xhr.upload's properties (with Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3, GM 0.8.6RC1 and your patch of course).
Also not sure if this is relevant:

Note: You need to add the event listeners before calling open() on the request. Otherwise the progress events will not fire.

(taken from MDC).
If that's true for upload events, your approach won't work for the upload-progress-event, since xhr.open and xhr.send are called before rtnReq is returned.

I get an access denied-error when trying to access any of xhr.upload's properties

Thanks for testing that Venero, I hadn't gotten around to that yet, would you mind making a gist of the test you used?.

Note: You need to add the event listeners before calling open() on the request. Otherwise the progress events will not fire.

(taken from MDC).

hmm.. we could accept new properties for the details object provided, which takes upload event listeners like it does onload/onerror, the only question would be what to call them.. maybe upload.onprogress? so the code would look like:

GM_xmlhttpRequest({
  method: "GET",
  url: "http://www.example.com/",
  onload: function(response) {
    ...
  },
  upload: {
    onprogress: function(e){...},
    onload: function(e){...},
    onerror: function(e){...},
    onabort: function(e){...}
  }
});

The gist is here.

hmm.. we could accept new properties for the details object provided, which takes upload event listeners like it does onload/onerror

Sounds like a good idea to me, and I guess the event names should be consistent with the request-events, so I think we should use onload, onprogress etc.
I guess a function similar to GM_xhr.setupRequestEvent could be used to set up the upload events. Since there's a few event-properties that can't be accessed (i.e. explicitOriginalTarget) without throwing an exception, I'd say the function should create a new object which will be passed to the callback instead of the original event, just like setupRequestEvent does.
The properties that object should have can be seen here.

the only question would be what to call them..

MDC: Progress events exist for both download and upload transfers.

I would go with something like:

GM_xmlhttpRequest({
  method: "GET",
  url: "http://www.example.com/",
  onload: function(response) {
    ...
  },
  uploadProgress: {
      onprogress: function(e){...},
      onload: function(e){...},
      onerror: function(e){...},
      onabort: function(e){...}
  }
});

I guess a function similar to GM_xhr.setupRequestEvent could be used to set up the upload events.

I say we modify GM_xhr.chromeStartRequest to attach all the events(both existing and the new upload/download progress events) with GM_xhr.setupRequestEvent.

I'd say the function should create a new object which will be passed to the callback instead of the original event, just like setupRequestEvent does.

Agreed

GM_xmlhttpRequest({
  method: "GET",
  url: "http://www.example.com/",
  onload: function(response) {
    ...
  },
  progress: {
    upload: {
      onprogress: function(e){...},
      onload: function(e){...},
      onerror: function(e){...},
      onabort: function(e){...}
    },
    download: {
      onprogress: function(e){...},
      onload: function(e){...},
      onerror: function(e){...},
      onabort: function(e){...}
    }
  }
});

Wouldn't details.onload and details.progress.download.onload listen to the same event?

I think adding onprogress, in:

GM_xmlhttpRequest({
  method: "GET",
  url: "http://www.example.com/",
  onload: function(response) {
    ...
  },
  onprogress: function(e) {
    ...
  }
});

Should be part of this issue as well.

Wouldn't details.onload and details.progress.download.onload listen to the same event?

I read the documentation more and apparently the download events are already called on the regular events so there's no need to implement them. Instead I'm going to use a uploadProgress object for the upload events.

I think adding onprogress, in:

Okay, I've got a patch for this issue in this commit: http://github.com/sizzlemctwizzle/greasemonkey/commit/4cd996cc1d8b6f480751d1830ee3bfa559060c8e

Here is a script to test the upload onload event(the other events are harder to test): http://gmconfig.googlecode.com/svn/trunk/test_all_events.user.js

I updated the branch for this issue.

One thing that still needs to be worked out is if any extra information is shown about the script during the install process then what is already present for installed scripts. The current install dialog shows where a script runs and does not run(@include/@exclude rules), as well as a warning to the user about installing scripts from untrusted sources.

One possible option is to add an extra button or link that would show these details(which is collapsed by default) about an installed script. This might be a good solution since when we moved to the addons dialog for user script management we also loss visibility of these rules for installed scripts as well.

We could possibly shove the untrusted scripts warning in a yellow banner when yet-to-be-installed scripts are present in the addons dialog.

Note: the forum seems to mess up the linebreaks, the code looks a bit messy that way

I don't see the benefit of adding completely unnecessary code, but if #1040 gets implemented you can have this code in your template.

As Sizzle says, I'd might see as a "template" feature but definitely not by default. -1

The console is indeed the right place/way to handle errors. Alerts are not. You can choose alerts for yourself, but if you get into an infinite loop of alerts .. it's not fun.

Yeah well i have a thing for alerts, the [i]way[/i] (crosses thumbs that BB code works here) it gives you the output is of minor importance. And yeah, maybe not by default, but as an option somewhere. You get my point.

It's hard to know much without more information from you, but I'll take a guess at why you're having a problem. Your function is bounded by the same domain policy for XMLHttpRequests so you have no problem with cookie permissions since they belong to the same domain you are making the request on. However, GM_xmlHttpRequest doesn't have those domain restrictions. It can make requests across domains, but it can only send cookies belonging to the domain you are requesting. I think you expect it to send cookies from the domain your script is running on but that doesn't happen. Again, this is just pure speculation. I won't know if I'm right without more information from you.

While probably true, it is not actionable for us non-Dutch people. If you suggest two appropriate phrases that would work better in the context (and that are not already in one of the other lines of the same file), change is more likely to happen sooner.

I have no idea of an alternative because you used most of them already (if not all).
Hoera would be a "pretty good" alternative, but the "De ballen" really makes no sense at all. Maybe just "Heey" or "Hallo" could be used?

A) We manage our translations via babelzilla.org so there is the best place to suggest translation improvements.
B) I've always disliked these things anyway. They serve no practical purpose, but they do actively add confusion and problems like this, even in English. I'd vote for removing them completely, regardless of language.

It's cute but not really functional +1 for removal.

Yup, +1

I see agreement. Changed title to match.

Well, now I'm only seeing this happen once, even without restarting. But it still does happen at least the once.

This is caused by this line in the parse function of content/config.js:

if (!script._namespace && uri) script._namespace = uri.host;

When I patched the parse function to read metadata from installed scripts, I found that we don't have a usable uri to use as a backup namespace for an installed script. I guess we need some other backup(or replacement) namespace for this case. Maybe we could just generate a random alphanumeric string of an arbitrary length?

This error also occurs if you remove the @name of an installed script for the same reason. A random string probs doesn't make as much sense for this. Maybe we could use something generic and combined with a random @namespace this would be okay. It's hard to come up a valid replacement for data that really should be required.

Seems to also happen when installing a script that never had a namespace.

There is legacy behavior for how to act when a script is missing a name and/or namespace. It should be preserved.

I've seen this in other, but not all, extensions. I have no idea what's going on. I hope it's a platform (firefox/mozilla) bug.

patch