Allow installing local files

[arantius]

In the past you could File>Open a .user.js and it would install. In 4.0 so far, it does nothing, just opens the file.

[Sxderp]

Perhaps this (match pattern docs)? Using * in the matches selector for the scheme only globs to http or https. Suggest adding an additional match for file://.

[ToadKing]

That or the <all_urls> selector.

[arantius]

Adding a match pattern to detect file:/// scripts just causes us to start an XHR which fails to read the contents at that URL, which is surprising.

[Sxderp]

I've been looking into this a bit. And I've come to two conclusions. The issue may have to do with disallowing filesystem access to WebExtensions, even if readonly through XHR (no source on this directly; edit: here at the very bottom). Or it could be related to same origin policy.

Starting in Gecko 1.9, files are allowed to read only certain other files. Specifically, a file can read another file only if the parent directory of the originating file is an ancestor directory of the target file.

It might be worth bringing up in a bug report to allow filesystem access to paths defined in the manifest with the file:// protocol.

[arantius]

Yeah, it's probably the "no files" rule. Don't know where to get official guidance on that but I know it's true now that I think about it. You only get very special exceptions like the downloads API for making new files.

We could try to figure a workaround like directly handling the drag/drop, or (LOL) reading the content of the tab that's already open. But then we'd fail to fetch relative icon/resource/requires, so it still won't go, without even more workarounds.

[Sxderp]

Could you not use storage.local to cache the content using the url as a key before navigating the to install page? You already have the content in a variable. Of course the cache would have to be cleared once installed / decidedly not installed. Would be more convenient if WebExtensions had some sort of temporary storage so they don't have to manually deal with eviction.

[Sxderp]

I've been messing around with what I've said and it's not as straightforward as I would have thought. Firstly, any user script has access to browser.storage.local so it's inherently an unsafe storage for addons like Greasemonkey. Secondly, the same goes for sending the content through a message to a background script. I'm not really sure how to secure that to ensure the message was sent only from script-detect.js. And due to the asynchronous nature of the scripts I'm not entirely sure if script-detect.js runs before user scripts (I'm going to do some tests on that).

And of course, unless I'm mistaken, background scripts don't receive any references to the DOM / content in any of the navigation listeners?

[Sxderp]

Turns out you can get the page content using onBeforeRequest with a match on ['*://*/*.user.js'] and then creating a StreamFilter. I've implemented some code testing this out in my recently published branch, no pull request at the moment since it doesn't fix anything. However, it does avoid the security concerns I brought up in my previous post.

Unfortunately it doesn't solve the file issue that was discussed. Some bugzilla tickets on it:
https://bugzilla.mozilla.org/show_bug.cgi?id=1341341
https://bugzilla.mozilla.org/show_bug.cgi?id=1266960

[erosman]

I was directed here from #2671

If this thread is about importing from local file .... then ...
Why are you using XHR to read local file? It causes all sorts of complications with origins and permissions.
The easy way is to use new FileReader() from the result of input type="file"

If this thread is about recognising file:///.....user.js URLs as scripts and installing them, that is the different issue and different solution.

[Sxderp]

The easy way is to use new FileReader() from the result of input type="file"

Ah, this could work. It's not as graceful as navigating to a file:// path and having the extension do everything for you, but it could work. The majority of the workflow could remain the same.

import script -> script selected -> contents cached in backend -> install dialog prompt -> retrieve content from backend -> continue install as usual

And it's much safer than the methods I was trying to use when attempting to keep the navigation workflow.

[kekkc]

Found a working webextension example using "input type="file"". Seems there's no need to cache, can be directly imported:
https://github.com/mdn/webextensions-examples/pull/171/files/6c066cfff4e8c662984f704cb17c8b39211ed062#diff-098de1750b345156f3cfd46f8199aa34

[Sxderp]

Seems there's no need to cache, can be directly imported

It's not that the cache is needed, but rather a safety measure. Just like when you navigate to a .user.js the dialog pops up telling you some information about the script. I think the same should happen when you do the import. Therefore you need to store, somewhere not in the normal database, the script contents so that should the user click the install button you still have it available and don't need to prompt the user again.

Whether this is cached using a storage API or just in a global object somewhere doesn't particularly matter for the workflow.

[Sxderp]

I'd also like to say that an import feature should become top priority[1] as it would help with people having migration issues. They can just import the files from 3.x.

[1] I'd look into it if @arantius has some ideas for how the process should work.

[erosman]

I have import./Export in a number of my addons if examples are needed.

The import is user initiated so there is no need for extra precautions/popups/notifications/warnings.

Add an import button (file input) to one of the dialogues
Once user clicks it, File picker opens up.. user select the required file and click open (all built-in HTTML5)
File is read and parsed
If it conforms to a USER Script, it is then added to the IDB
Then update the running listeners

That is all....

I do the same in Import/Export preferences, Theme data (up to 500kb) and many others in my add-ons with Import/Export function.

I'd also like to say that an import feature should become top priority[1]

Indeed .... that enables script writers to write new scripts and import to run or test and in case of the GM3 -> 4 upgrade, add the missed out scripts.

The code and function is very simple... few lines of code which can be done in an hours.
You can use my code as a base if you wish.

[Sxderp]

... The import is user initiated so there is no need for extra precautions/popups/notifications/warnings. ... If it conforms to a USER Script, it is then added to the IDB

I'm not sure about this though. I don't particularly like skipping over the standard install dialog. Perhaps @arantius can give some insight on what he'd like to see in the addon.

[erosman]

I'm not sure about this though. I don't particularly like skipping over the standard install dialog.

Standard dialogue is used when user is confronted with a user script from a remote source. A confirmation dialogue is then needed.

In case of user initiated Import:

• User decides to import a script
• User clicks the import button
• User navigates to the script selected by the user
• Is there a need to ask user again with a dialogue "Do you really want to install this script?" :)
  That seems annoying. A warning in case of errors is necessary though.

[Eselce]

• User navigates to the script selected by the user

But this is not the way it goes... Installation is by callback (trigger on *.user.js)...

[Sxderp]

But this is not the way it goes... Installation is by callback (trigger on *.user.js)...

Currently, yes. But it is possible to add an import button for local files. And not do installation on navigation for file://.

[Eselce]

Ah, okay, for local files, that is quite appropriate. Not for remote files! ;-)

[erosman]

But this is not the way it goes... Installation is by callback (trigger on *.user.js)...

As mentioned, we are talking about manual Import using file input

[Eselce]

Yes, I got it. That's okay for me for local files (see above)...

[S-ed]

I think it's possible to just drag&drop the .user.js on to Greasemonkey window to load the code of the script in to it?
Since this definitely works for Firefox itself (i.e. i could upload file to the website, like imgur, or megaupload by dropping it)

[bsto]

Is there a way (even a clumsy, non-drag-and-drop one) which allow me to import local GM scripts?

In exactly which "cache" directory should the scripts be finally placed? I found several "cache" files in Firefox profile

[erosman]

I think it's possible to just drag&drop the .user.js on to Greasemonkey window to load the code of the script in to it?

It is possible, like any other drop box (check my IMGoolge for an example). However, the straight forward file input would be the easiest method without the need for additional listeners and processes.

[kekkc]

Mozilla updated the doc (summary of the examples above):
https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Working_with_files

1. Open files in an extension using a file picker
2. Open files in an extension using drag and drop

With this implemented, we would have the same behavior as in GM3.

[Samizdata]

So, any possible suggestions for manually installing a script that isn't offered in an appropriate online form?

[kekkc]

@Samizdata Currently the easiest way is to get GM beta (https://addons.mozilla.org/en-US/firefox/addon/greasemonkey/versions/beta), open monkey menu --> new script.

If you have scripts that @require local files, you can only do it the hard way. E.g. grab Proxomitron ( http://www.proxomitron.info/files/ ), run it, configure FF to use proxy 127.0.0.1:8080, place your files in the Proxomitron HTML folder and access them afterwards within FF with "http://bweb..local.ptron/YOURFILE.user.js".

[Sxderp]

@kekkc, I think a new version with the remote resource problem fixed has been pushed to AMO.

[Samizdata]

Cheers. That's done it, @kekkc !

[erosman]

@kekkc, I think a new version with the remote resource problem fixed has been pushed to AMO.
Not yet.. as far as I can see

[Eselce]

@Sxderp Are we referring to #2707?

[Sxderp]

@Eselce, yes. That should resolve the issues present when creating a new script with the 'new script' button and then applying a @require tag to the script.

[ocumo]

To have a .user.js or a required script being served locally, there are various possibilities. But the easiest way is using this Python one-liner at your command line:

• for Python 2.x:
  python -m SimpleHTTPServer

• for Python 3.x:
  python -m http.server

...which immediately starts serving all files in the directory where you run it, on http://127.0.0.1:8000/

(note that 8000 is the default port; you can change that; see below)

That's how I put my local .user.js and require files in a local http server. I do this for Greasemonkey but also for Tampermonkey.

Python is installed by default in Linux and MacOs. If you would be using Windows and wouldn't have it installed and still plan on keep calling yourself a developer, then... seriously?! what's wrong with you? you have much more serious troubles than you can start to realize! I would suggest gardening 🌱 or knitting as hobbies rather than computers for you 😉 (hey, just kidding!)

There is no need to install any strange programs -- you can, but that's absolutely unnecessary. Python is not an "app", it's a fundamental programming language. But you don't even need to "speak" Python for this solution, so don't quit or rush to your knitting tools just yet!

Trivial procedure:

1. cd into the directory containing your .user.js and/or require files. Say, for example, requiredFile1.js and requiredFile2.js

2. From that directory: For Python 2.x, run

python -m SimpleHTTPServer

OR: for Phython 3.x, run

python -m http.server

3. Ensure your @require lines are like these:

// @require  http://127.0.0.1:8000/requiredFile1.js
// @require  http://127.0.0.1:8000/requiredFile2.js

...where requiredFile1.js and requiredFile2.js are whatever required local files you want to serve.

4. When your greasemonkey script fires, it will correctly pick up the requires, which are being served by Python.

Done.

To close your local server, just go to the console where you run the python command and press <Ctrl><C>.

Also, --and I hope this is ridiculously obvious to you--, please do not forget to run your Python HTTP server command line before expecting that Greasemonkey or Tampermonkey would be able to find the files...

Tip: If you want to use a port other than the default 8000, simply type the desired number (a valid port number) in your command, like this:

• for Python 2.x:
  python -m SimpleHTTPServer 12345

• for Python 3.x:
  python -m http.server 12345

...and naturally update the @require urls with that number instead of 8000.

[bsto]

Ok, Assume I click on the GM toolbar icon and select "New user script...."

The new corresponding tab is auto-named "Unnamed Script 821696"

Then I paste from a local file GM code into the tab pane and click the "save" icon on the upper left side.

Where do I find later this script? Read: How can later edit this script?

How can I change the name of the script to e.g. "foobar"?

[S-ed]

@bsto Name will be taken from the script @name.
All new scripts will appear above "New user script...".
To remove or edit one you click the title of the script, there will be sub menu.

[bsto]

Ok, thank you.

Just another question:

On Nov 25th user kekkc told us in his posting (see above) that Mozilla offered a way to drag & drop files (from WinExplorer).

So drag & drop of user files should be possible now.

Am I right?

When will it implemented in GM (available for drag & drop of *.user.js files)?

[arantius]

FWIW I think we can/should detect navigation to file://.../anything.user.js, and attach a page action that could open a UI with whatever kind of file browser we can make work.

[Sxderp]

We can detect navigation events but not get content (maybe in a content script)

Although I find it silly to navigate to a file:// just to open a file browser (I don't think we can do anything other than a file picker input).

[arantius]

Although I find it silly to navigate to a file:// just to open a file browser (I don't think we can do anything other than a file picker input).

Yes exactly, we're hamstrung. But we can detect intent and be as helpful as possible within our limited environment.

[erosman]

My 2 cents ...

FWIW I think we can/should detect navigation to file://.../anything.user.js,
We can detect navigation events but not get content (maybe in a content script)

As mentioned by Sxderp, it is possible but a bit messy ...

• Add a listener (like tabs.onUpdated.addListener since you would need the content anyway)
• Let the page load showing the script
• Inject content script to grab page content and pass to bg script
• Close the page/tab

Alternative ....

• Add a navigation listener
• Stop the page loading and display a notification to use the Import Option ... OR .... popup showing the import option which user has to click to launch the filepicker

Personally, if it were me, I would opt for not adding extra listeners and simply have the IMPORT option added to the browserAction popup

Additionally, I have not been using the .user.js format since GM4 anyway and I would reckon it can be left behind. ;)

[arantius]

Personally, if it were me, I would opt for not adding extra listeners and simply ..

This is what I meant. I might inject UI into the content page, though. We used to pop an infobar if you navigated to a script while GM was disabled; that sort of thing.

Additionally, I have not been using the .user.js format since GM4 anyway and I would reckon it can be left behind. ;)

What does this mean?

[erosman]

What does this mean?

GM3 required the scripts to be named as abc.user.js so that they could be recognised as GM script.

In GM4, scripts are saved to IndexedDB and the script name doesn't really matter as it gets the name from @name . Therefore I have been creating and saving my scripts (on computer) as abc.js (without the .user) and copy/paste into GM.

The manual IMPORT, I would imagine wouldn't need be tied to .user naming format.

[IzzySoft]

Any progress on this?

[pstein]

FireMonkey Extension can do this:

https://addons.mozilla.org/en-US/firefox/addon/firemonkey/