Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing styles from Stylish add-on for Firefox #77

Closed
schomery opened this issue Jun 5, 2017 · 62 comments
Closed

Importing styles from Stylish add-on for Firefox #77

schomery opened this issue Jun 5, 2017 · 62 comments
Labels

Comments

@schomery
Copy link
Contributor

schomery commented Jun 5, 2017

Update: please test the export addon for Stylish-Firefox: instruction below


in Stylish add-on for Firefox, the actual code is being stored instead of sections.

We can get all the stored styles by running the following code in the browser console (Ctrl + Shift + J)

var service = Cc['@userstyles.org/style;1'].getService(Ci.stylishStyle);
var styles = service.list(service.REGISTER_STYLE_ON_CHANGE, {}).map(s => ({
  enabled: s.enabled,
  id: s.id,
  md5Url: s.md5Url,
  originalMd5: s.originalMd5,
  name: s.name,
  updateUrl: s.updateUrl,
  url: s.url,
  code: s.code,
}));

I think it makes more sense to check if the imported JSON has a code key in its objects and if so convert them into sections before storing the imported styles. If I want to convert codes into sections in this script, it gets pretty messy.

@tophf any thoughts?

@schomery
Copy link
Contributor Author

schomery commented Jun 5, 2017

Okay, since we need to have an SDK add-on anyway to generate the JSON, let's do the parsing inside this SDK add-on.

Here is the first draft:
https://github.com/schomery/firefox-stylish-export
https://github.com/openstyles/firefox-stylish-export

When it is ready we can ask users to install it as a temporary add-on and generate the JSON.

@tophf
Copy link
Member

tophf commented Jun 5, 2017

Guess we should summon more people here from FF-related threads to test your export addon: @r-a-y, @TayliaM, @ivysrono, @Keith94, @BangDroid, @strel, @silverwind, @Mottie

@TayliaM
Copy link

TayliaM commented Jun 5, 2017

This was frustrating the heck out of me until I remembered I was using a FF Stylish fork now for everything but nightly. Works like a charm with Stylish. What a timesaver!

Edit to clarify: I very seldom use nightly. My nightly FF hadn't been updated recently and today after it did update I can no longer access the manage page for Stylus, and other extensions are borked as well. The export did work well on release WF and release FF from another laptop.

@Keith94
Copy link

Keith94 commented Jun 5, 2017

Here is the first draft; https://github.com/schomery/firefox-stylish-export

It's reported as incompatible with Nightly 55.0a1 :(

@schomery
Copy link
Contributor Author

schomery commented Jun 6, 2017

It's reported as incompatible with Nightly 55.0a1 :(

@Keith94 you need to load the extension in the debugging mode (about:debugging). It is not signed.

@schomery
Copy link
Contributor Author

schomery commented Jun 6, 2017

  1. Open about:debugging in a browser tab
  2. Download the XPI:
  1. Hit "Load Temporary Add-on" and point it into the downloaded XPI

There is no UI for now. If things go fine, you will get a desktop notification and a JSON file located in your Desktop directory.

@mzso
Copy link

mzso commented Jun 9, 2017

Well those steps are for installation, which I didn't need.
What I missed was the note at the end. Anyway, none of thos things happened, I didn't get any notifications or json files. Nothing happens. Tried via about:debugging too, just for completeness sake. Still nothing.

I only see a rather uninformative error on the console "20:34:23.267 TypeError: start is undefined 1 parse.js:45:11"

@tophf
Copy link
Member

tophf commented Jun 9, 2017

@mzso, thanks for the error message. It seems some of your styles have nested @-moz-document sections which is not supported in WebExtensions-based Stylus. Meanwhile try https://github.com/ExE-Boss/Stylish-to-Stylus and the other script mentioned there.

@mzso
Copy link

mzso commented Jun 9, 2017

Okay, so there's only the linked script? (I only see an empty java file)
Anyway I don't have whatever's required to run python scripts.
This is too much of a nuisance. I'll just stick to Stylish for now.

@schomery
Copy link
Contributor Author

@schomery
Copy link
Contributor Author

@tophf can you investigate this

@-moz-document domain(google.com) {

/** comment 2 **/
/** comment 2 **/
/** comment 2 **/
/** comment 3 **/ 

  body {
    border-color: #000;
  }
}

This cannot be imported using import button

@mzso
Copy link

mzso commented Jun 10, 2017

@schomery commented on 2017. jún. 10. 08:48 CEST:

@mzso please give the new version a try; https://github.com/schomery/firefox-stylish-export/blob/master/stylish-export.xpi?raw=true

There seem to be issues. First I tried on latest nightly but the manage tab doesn't appear in stylus.
I tried on the build of 05-23, which I use for everyday browsing. I could successfully export. (If Stylish was enabled, otherwise it failed silently.)
But on import I only got a spinning throbber while hovering over the manage tab.

(On export I got a single error for one stlye, which I don't actually use, have lying around disabled: 10:06:56.318 stylish-export:Torrentproject.se Dark start is undefined 1 index.js:45
)

@schomery
Copy link
Contributor Author

schomery commented Jun 10, 2017

First I tried on latest nightly but the manage tab doesn't appear in stylus.

This is related to #78

If Stylish was enabled, otherwise it failed silently.

Thanks! I added a notification to warn users if Stylish add-on is not accessible.

But on import I only got a spinning throbber while hovering over the manage tab.

Can you share your stylish.sqlite file for testing?

@Mottie
Copy link
Member

Mottie commented Jun 10, 2017

I used the Firefox export style addon and then exported the same style from Stylus and after normalizing the two exported json files, I found these differences (using Windows 10):

Stylus export Firefox Stlyish Export
updateUrl: "https://userstyles.org/styles/chrome/37035.json?..." updateUrl: "https://userstyles.org/styles/37035/github-dark.css?..."
originalDigest entry no originalDigest
regexps (unescaped) github.com/foo regexps (escaped) github\\.com\\/foo
Code includes \r\n Code includes \n
Code ends with no closing bracket (@-moz-document wrapper) Code ends with \n\n}

Here are the actual files: https://gist.github.com/Mottie/b1e0f12aeb265163504e36235a2251a2

I originally had them saved as a revision, but you still had to look at the diff using git, so I ended up separating them to make copy/pasting easier.

@tophf
Copy link
Member

tophf commented Jun 10, 2017

@tophf can you investigate this

I had to fix csslint's parser: 2e86c95

@mzso
Copy link

mzso commented Jun 10, 2017

@schomery commented on 2017. jún. 10. 10:37 CEST:

But on import I only got a spinning throbber while hovering over the manage tab.

Can you share your stylish.sqlite file for testing?

Okay, but it's filled with a bunch of disabled non-functioning/obsolete/tinkerware crap. I'll add the exported json too.

stylish-json.zip
stylish.zip

@schomery
Copy link
Contributor Author

@Mottie updateUrl is now converted to the Chrome URL. Thanks! Also, I changed the order of JSON keys to match with the Stylus keys.

Code ends with no closing bracket (@-moz-document wrapper)

This is fixed. Please give the new version a try.

regexps (escaped)

@tophf Stylus generates unescaped regexps, is this correct?

@schomery
Copy link
Contributor Author

@mzso please try the latest version of the exporter add-on and let me know if the JSON file is working as expected or not. I tested it and it successfully exported 36 styles.

@tophf
Copy link
Member

tophf commented Jun 11, 2017

Stylus generates unescaped regexps, is this correct?

Our backup file is a JSON and regexps are strings so they get the standard JSON escaping, in particular \\ for \. In Stylish for Firefox regexps are part of the style code so they use the standard string escape. In Stylus regexps use dedicated input boxes where the actual non-escaped expression is presented.

@schomery
Copy link
Contributor Author

In Stylus regexps use dedicated input boxes where the actual expression is presented.

The actual string should be github\.com\/foo not github.com/foo, right?

@tophf
Copy link
Member

tophf commented Jun 11, 2017

The actual string is what the user typed. A valid expression would be github\.com/foo

@schomery
Copy link
Contributor Author

this is what we have in the actual CSS

@-moz-document regexp("^https?://((gist|guides|help|raw|status|developer)\.)?github\.com((?!generated_pages\/preview).)*$"), domain("githubusercontent.com")
{
...
}

so why we get ^https?://((gist|guides|help|raw|status|developer).)?github.com((?!generated_pages/preview).)*$ from Stylus JSON?

@tophf
Copy link
Member

tophf commented Jun 11, 2017

why we get

Because the original expression is INVALID: \ inside a string should be \\.
Here's an example of a valid at-expression: regexp("^foo\\.bar")

Quoting the specification:

The values provided to the regexp() function must be enclosed in quotes. Escaped values provided to the regexp() function must additionally be escaped from the CSS. For example, a . (period) matches any character in regular expressions. To match a literal period, you would first need to escape it using regular expression rules (to \.), then escape that string using CSS rules (to \\.).

@schomery
Copy link
Contributor Author

right!

@tophf
Copy link
Member

tophf commented Jun 11, 2017

I just thought it's highly probable that many styles have invalid regexp escaping. I guess we could violate the spec and import them by checking whether the regexp string has single \ (which is invalid) and import it as is. This would require another csslint patch, though, but csslint is dead anyway so it's ok.

@tophf
Copy link
Member

tophf commented Jun 11, 2017

@schomery, Stylus editor "import" button already handles incorrectly-escaped regexps just fine. I haven't tested the export addon, though, nor intend to :-)

@mzso
Copy link

mzso commented Jun 11, 2017

@schomery commented on 2017. jún. 11. 08:43 CEST:

@mzso please try the latest version of the exporter add-on and let me know if the JSON file is working as expected or not. I tested it and it successfully exported 36 styles.

Exporting wasn't the issue last time. I could and can export fine. (As far as I can tell, and I attached the exported json too in my previous post, for you to check if needed)
It's importing the json that didn't succeed. And still doesn't.

I get the infinite spinner still and this in the console:

IndexedDB UnknownErr: ActorsParent.cpp:600  (ismeretlen)
undefined  storage.js:61:9
UnknownError  storage.js:46:18

@Mottie
Copy link
Member

Mottie commented Jun 12, 2017

Hehe I understand, what I meant was that regxr.com forces you to escape forward slashes. Anyway, I'm not trying to argue.

I haven't had a chance to test the Firefox Stylish exporter after the update. I'll do it when I get some free time later today.

@mzso
Copy link

mzso commented Jun 19, 2017

I gave this another go on FF55 beta, with the latest version of Stylus. But I still couldn't import the styles, and got the infinite spinner.

I even tried it in safe mode, with loading stylus as a temporary extension. Didn't work then either.

@Kinematics
Copy link

Was making another attempt at using Stylus, and needing to deal with importing. Finding a way to export from Stylish was annoying, but I finally found that the Stylish-Custom addon lets you expose an export feature. The exported styles are all .css files saved into a specified directory, with each css file named with the name of the style rule.

In the end, very simple.

(It also has an XML export, but that looks like it double-escapes HTML entities, which makes it annoying.)

So now I'm trying to import into Stylus, and it seems the only method available is to import a .json file? And then there's issues with escaping characters because it's a json file instead of a css file? Regardless, it doesn't work. If I try to load the suggested .xpi from about:debugging in the latest Nightly, I get the error:

There was an error during installation: Add-on @stylish-export is not compatible with application version. add-on minVersion: 38.0a1. add-on maxVersion: *.

No json file was created.

I'd just like to know if there's a reason the import process can't handle straight css files? The editor itself has options to 'import' Mozilla-style code, but all that means is that it brings up a window that you can paste stuff into. Clearly the addon has the means to handle that format, so it would be nice to just point it at a bunch of files and let it do the job.

I'll probably have to end up doing a manual copy-paste, but it would have been nice to have simple expectations 'just work'.

@tophf
Copy link
Member

tophf commented Jun 21, 2017

I'd just like to know if there's a reason the import process can't handle straight css files?

Meta info will be lost (update URL, customized advanced style settings on userscripts.org site). And the only means of detecting whether a style existed would be its name which is not reliable enough to update a style on such import, hence it would be added as a duplicate or skipped if its contents is identical. I'm not convinced this scenario is useful except for the mentioned edge case.

schomery added a commit to openstyles/firefox-stylish-export that referenced this issue Jun 21, 2017
@schomery
Copy link
Contributor Author

There was an error during installation: Add-on @stylish-export is not compatible with application version. add-on minVersion: 38.0a1. add-on maxVersion: *.

@Kinematics give "Stylish export to JSON" add-on another try. The issue with the latest Nightly is fixed.

@mzso
Copy link

mzso commented Jun 21, 2017

So will it only be possible to import into a fresh Firefox profile? It makes migration a lot more of a chore.
I still get the exact same error as before.

@Kinematics
Copy link

@Kinematics give "Stylish export to JSON" add-on another try. The issue with the latest Nightly is fixed.

Tried it, and it worked fine. Thanks.

Meta info will be lost (update URL, customized advanced style settings on userscripts.org site). And the only means of detecting whether a style existed would be its name which is not reliable enough to update a style on such import, hence it would be added as a duplicate or skipped if its contents is identical. I'm not convinced this scenario is useful except for the mentioned edge case.

Doesn't matter so much now that the export is working again, but I suppose mine could be an edge case, given that I have no styles from userscripts.org; all of mine are written by hand. Duplicates are irrelevant when just trying to get Stylus working on a first migration, and names are guaranteed to be unique simply due to file system naming requirements.

@Fyren
Copy link

Fyren commented Aug 7, 2017

It seems like the exporter addon is broken on nightly. No file is written to my desktop and in my browser console, I get:

00:57:36.061 1502089056060	DeferredSave.extensions.json	DEBUG	Starting write
00:57:36.126 stylish-export:Object { stack: "", 5 more… }  runner.js:162
00:57:36.130 stylish-export:Object { stack: "", 5 more… }  Promise-backend.js:925
00:57:36.071 1502089056071	DeferredSave.extensions.json	DEBUG	Write succeeded

If I poke around the object that's logged in those middle two log lines, I see the message field is service is undefined and the stack field is:

@resource://stylish-export/index.js:21:5
run@resource://gre/modules/commonjs/sdk/addon/runner.js:143:19
startup/</<@resource://gre/modules/commonjs/sdk/addon/runner.js:83:9
process@resource://gre/modules/Promise-backend.js:922:23
walkerLoop@resource://gre/modules/Promise-backend.js:806:7
Promise*scheduleWalkerLoop@resource://gre/modules/Promise-backend.js:739:11
schedulePromise@resource://gre/modules/Promise-backend.js:770:7
completePromise@resource://gre/modules/Promise-backend.js:707:7
promise callback*completePromise@resource://gre/modules/Promise-backend.js:698:7
process@resource://gre/modules/Promise-backend.js:958:5
walkerLoop@resource://gre/modules/Promise-backend.js:806:7
Promise*scheduleWalkerLoop@resource://gre/modules/Promise-backend.js:739:11
schedulePromise@resource://gre/modules/Promise-backend.js:770:7
completePromise@resource://gre/modules/Promise-backend.js:707:7
readURI/<@resource://gre/modules/commonjs/sdk/net/url.js:49:9
onStopRequest@resource://gre/modules/NetUtil.jsm:131:17

@mzso
Copy link

mzso commented Aug 7, 2017

@Fyren
Since they're dismantling XUL addon support don't expect it to ever work on FF57+

@Fyren
Copy link

Fyren commented Aug 7, 2017

Though that's not totally unexpected, it seems like the highest demand for an export tool is now that Stylish has also broken and people will look to move to Stylus.

@mzso
Copy link

mzso commented Aug 7, 2017

@Fyren commented on 2017. aug. 7. 10:36 CEST:

Though that's not totally unexpected, it seems like the highest demand for an export tool is now that Stylish has also broken and people will look to move to Stylus.

You should migrate before it breaks, not after. v56 is the last version you can expect to do so. Since (I'm certain) the export tool depends on deeper stuff that the WE API won't ever provide. Such as reading the "stylish.sqlite" file.

I would have migrated a while ago, but I'm hindered by stylus' inability to import what stylish-export has exported.

1 similar comment
@mzso
Copy link

mzso commented Aug 7, 2017

@Fyren commented on 2017. aug. 7. 10:36 CEST:

Though that's not totally unexpected, it seems like the highest demand for an export tool is now that Stylish has also broken and people will look to move to Stylus.

You should migrate before it breaks, not after. v56 is the last version you can expect to do so. Since (I'm certain) the export tool depends on deeper stuff that the WE API won't ever provide. Such as reading the "stylish.sqlite" file.

I would have migrated a while ago, but I'm hindered by stylus' inability to import what stylish-export has exported.

@schomery
Copy link
Contributor Author

schomery commented Aug 7, 2017

@Fyren as @mzso mentioned for the exporter add-on to work, you need to have a working Stylish add-on. Since both Stylish and the exporter is not working in FF57, use an older version of Firefox to create your JSON backup. Also, note that Firefox is changing their profile manager as of FF55; https://www.ghacks.net/2017/08/02/you-cannot-downgrade-firefox-55-profiles/

@jscher2000
Copy link

jscher2000 commented Aug 22, 2017

You can use sql.js to extract from a stylish.sqlite file in the browser. I have an example here:

https://www.jeffersonscher.com/res/stylishextract.html (loads 2MB script file)

That just makes an archival html document with the user styles' titles and code. If there is a particular format that would be good to generate perhaps I could add that to my page, or you could just take the current code and run with it on your site.


NOTE: sql.js takes up some 3.6MB under

[your-profile]/storage/temporary/https+++www.jeffersonscher.com/asmjs

You can delete it once you're done, either using your file browser or using the "Settings" button in the "Site Data" section of the Options/Preferences page to clear all data from www.jeffersonscher.com (select that domain in the list, click Remove Selected, click Save Changes).

@Mottie
Copy link
Member

Mottie commented Aug 22, 2017

Nice @jscher2000!

I don't know where that file is stored on a Mac, but in Windows you'll find it at:

C:\Users\{username}\AppData\Roaming\Mozilla\Firefox\Profiles\xxxxxxx.default\stylish.sqlite

If you are using a dev edition of Firefox, the profile folder will be \xxxxxxx.dev-edition-default\ (at least that's what it is for Nightly).

@silverwind
Copy link
Contributor

I don't know where that file is stored on a Mac

It's also visible in about:support > Profile Folder, btw.

@mzso
Copy link

mzso commented Aug 23, 2017

Hi!

Okay, so all of my attempts at migrating to stylus failed. Since importing only ever works on a new profile I imported the styles in such.
Then I did a file content search for some specific word in my styles and identified [mozilla profile]\storage\default\moz-extension+++4685ec5c-e55b-4928-867c-9d1a0802b6f4\ as the likely place where the styles are stored.
Copied it to my main profile, but I still get nothing.

Tried creating a style, thinking maybe a different place for the settings might be used. BUT that failed also.
Clicking save apparantly didn't save it and when I tried to navigate away (eg: " back to manage" button) I got a FF prompt to verify that I want to leave the "add style" page.

So it seems it not just importing fails but stylus is broken in it's totality. Can't create or import any styles, can't do anything.
So at this point I'm at a loss.

These errors showed up on the console when trying to save:

IndexedDB UnknownErr: ActorsParent.cpp:598  (ismeretlen)
undefined  storage.js:62:9
UnknownError  storage.js:47:18

@tophf
Copy link
Member

tophf commented Aug 23, 2017

IndexedDB UnknownErr: ActorsParent.cpp:598 (ismeretlen)

This error indicates your FF is broken.
Might be related to the known issue of cookies: when it's set to clear on exit FF also breaks/clears IndexedDB.

@mzso
Copy link

mzso commented Aug 23, 2017

@tophf commented on 2017. aug. 23. 10:17 CEST:

IndexedDB UnknownErr: ActorsParent.cpp:598 (ismeretlen)

This error indicates your FF is broken.

Might be related to the known issue of cookies: when it's set to clear on exit FF also breaks/clears IndexedDB.

I don't think it's broken in general all other addons work.
On the influence of the discussion in bug 1362878 I deleted the content of [mozilla profile]\storage\default\ and now I can import or create new styles.

However it looks like styles that mod the GUI are not applied. Are they not supposed to work?
I was hoping that since in FF <v57 they didn't remove any APIs or add restrictions (XUL addons still work), these styles would work.

EDIT:
I somehow managed to mix up two half comments. (An obsolet one and the updated one) Now I fixed it.

@tophf
Copy link
Member

tophf commented Aug 23, 2017

IndexedDB is one of the core browser API and the error indicates there's something wrong in the FF's internal code so I consider it's broken. Stylus simply may be the only extension that uses IndexedDB in your browser. Other developers think it's too complicated and opt for the simpler storage. Also, Stylus is a WebExtension and thus it can't apply userstyles to the browser UI, see #92 for more info and workarounds.

IIRC there's also a Private mode in FF which also breaks some core API that we use.

@citrusella
Copy link

This is quite late for the last messages to this issue, but I tested the export addon and it worked fine, except that one style I'd written was pegged as not mine... and somehow linked to a style on userstyles.org that isn't even the userstyles.org copy of it. 0_o

I suspect but can't conclusively prove that this is on Stylish's end rather than this one, so I'm posting here just in the less likely case that it's on this end.

@tophf tophf added the external label Dec 30, 2017
@stonecrusher
Copy link
Contributor

I guess this issue is solved or is it a placeholder for ambitions to automatic stylish -> stylus import function?
Jefferson's tool always worked very well for me and exporting from later Stylish seems to work also.

Imho the user style extractor makes the export addon obsolete (you can use it only up to FF 56!), but I'd keep it in the wiki as alternative and to honor all the effort.

@mzso
Copy link

mzso commented Nov 11, 2020

Hi!

So this method/addon should still work, right? (I see no indication that it shouldn't) I lagged around on the 56 branch until now. (Technically Waterfox Classic, but it makes for little difference) But I'm thinking of letting go because due to lack of webcomponents sites are breaking left and right.

Which would allow me to import styles from Stylish (technically Stylem, but also makes for little difference).
But what I also want is to export styles that affect GUI to separate .css files, so I can use them via UserChrome.css (Ones that are still applicable anyway). Can I do that via Stylus? No option for that is obvious to me.

@jscher2000
Copy link

But what I also want is to export styles that affect GUI to separate .css files, so I can use them via UserChrome.css (Ones that are still applicable anyway).

You might need to create your own separate files, but I suspect you need to start mostly from scratch. Firefox's UI structure has changed radically since Firefox 56. You may have better luck identifying what you want to change and then searching on the following subreddit for updated solutions:

https://www.reddit.com/r/FirefoxCSS/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests