Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute in frames #2574

Open
arantius opened this Issue Sep 21, 2017 · 48 comments

Comments

Projects
None yet
@arantius
Copy link
Collaborator

commented Sep 21, 2017

Greasemonkey 4 as of today only detects navigation events at the top level, so it effectively applies @noframes to every script.

@arantius arantius added this to the 4.x milestone Sep 21, 2017

Sxderp added a commit to Sxderp/greasemonkey that referenced this issue Oct 30, 2017

@Sxderp

This comment has been minimized.

Copy link
Contributor

commented Oct 30, 2017

webNavigation.onCommitted doesn't 'see' initial frame creation / page rendering, though if the frame navigations someplace other than its initial page then the listener will catch it. If options were to include the key 'allFrames': true the problem is somewhat resolved. Any frame in the static html page will have the script injected, although you then have origin/url matching problems. Further, if a frame is created using Javascript the script is not injected.

The easiest solution I could think of would be to replace webNavigation.onCommitted with webRequest.onResponseStarted with a filter of {'urls': ['<all_urls>'], 'types': ['main_frame', 'sub_frame']}.

I did some limited testing and no further changes were needed. I was able to execute a script within a frame and a frame created using Javascript.

Sxderp added a commit to Sxderp/greasemonkey that referenced this issue Nov 12, 2017

@ncehreli

This comment has been minimized.

Copy link

commented Nov 13, 2017

@arantius any plans to merge the fix from @Sxderp ?

This is affecting scenarios where the iframes are cross-origin objects so it's impossible to do any kind of modification without running GM on those particular iframes.

Thanks !

@arantius arantius modified the milestones: 4.x, 4.1 Nov 13, 2017

@arantius

This comment has been minimized.

Copy link
Collaborator Author

commented Nov 13, 2017

Missed this, will look into it soon.

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Nov 16, 2017

Just an observation during my testing of old scripts, don't know, if it helps...

Given a script, that is supposed to work on an iframe,
it looks as if it executes the changes on the page,
but then refreshes again to the unmodified page.

I can only see the flickering when I refresh the page very fast.

@Sxderp

This comment has been minimized.

Copy link
Contributor

commented Nov 16, 2017

Just an observation during my testing of old scripts, don't know, if it helps...

Is this with my patch or with the released version?

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Nov 16, 2017

Ugh, I think it was 4.0 release...
I WAS @4.1b3, but for the tests, I'm pretty sure, that I reinstalled the release version (until now!)

@canufarm

This comment has been minimized.

Copy link

commented Nov 16, 2017

Same here with iframes like Eselce describes. In some way it is executed, but then stopped after the iframe or page is loaded.

If I inject this script, I only get 1 and "self !== top":

console.log('1');
if (self !== top) {
   console.log('self !== top');
   setTimeout(function() {
      console.log('Timeout');
   }, 2000);  
} else {
   console.log('self === top');
}

"Timeout" is not shown in the log, neither all functions and binds are.

I use 4.1b3.

@dandrei

This comment has been minimized.

Copy link

commented Nov 17, 2017

I have the same issue running GM 4.0 on Quantum. I wrote a very simple dummy example with two pages: main.html and framed.html, and a GM script which gets loaded on every page and outputs the URL of the page it's loaded on.

Most of the time, I only get notifications about main.html, but in about 5% of cases, especially if I hold down F5, I might also get a notification about framed.html.

Is there any hack to reliably force GM 4.0 to execute inside iframes until a patch is out?

@cvzi

This comment has been minimized.

Copy link

commented Nov 18, 2017

I just found out that userscripts are reliably executed in <embed src="..."> but not in <iframe src="..">
I wrote a small test script:
https://openuserjs.org/scripts/cuzi/iframe_embed_Test_Greasemonkey_4

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2017

Some more details: In some cases, my scripts are completely executed in the frame (but the view is overwritten later by page scripts and so).
Sometimes, the synchronous parts of my script are ended, but the asynchronous parts are suddenly interrupted by page activity...
Hope, that helps!

@arantius arantius modified the milestones: 4.1, 4.2 Nov 23, 2017

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Dec 1, 2017

Has anyone more infos on this?

Just a short summary of this thread (issue):

  • Forget most of the postings, they don't apply
  • Probably, the scripts are executed always (but not till the end)
  • Unfortunately, the page is refreshed later - layout is recalculated, execution is aborted
  • This applies mostly (but not entirely) on async parts of the scripts

I'm not so much into these internals, but probably someone is...

@bobvh

This comment has been minimized.

Copy link

commented Dec 6, 2017

As a temporary fix, I've been swapping out iframes for embeds (example script), which does work to get a script corresponding to the frame to trigger (credit to @cvzi for figuring out that <embed src="..."> does work).

@RyanHanekamp

This comment has been minimized.

Copy link

commented Dec 13, 2017

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames. Since VM is open source, maybe see how they did it?

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Mar 24, 2018

You've described exactly what the problem is: It's like @noframes is activated. The frame content's URL does not activate your script correctly. Hopefully this is fixed soon (opening frames in extra windows is annoying)...

@GreG2MarS13

This comment has been minimized.

Copy link

commented Mar 24, 2018

Thanks Eselce.
And always since the update of Firefox, I am obliged in the header to declare all the scipts '.js' (with require) already used on the target site. (including jquerry)
And it's not the same.... It creates bugs or conflicts.
Do you also know this problem?

@Eselce

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2018

#2945 contains another example for the observation, that scripts are started in frames, but abandoned after some milliseconds.

@RyanHanekamp

This comment has been minimized.

Copy link

commented May 15, 2018

I'm hesitant to give the specific URLs I'm testing as they are outside of my control, but I'm finding some consistent but weird behavior to pass along.

I have a site with which I need to interact. It initially loads a frameset/frame with rows="100%,0" (one frame to fill the screen) on a different domain. This frame contains one frameset / 3 frames within the domain of the intermediate frameset/frame.

Some of the 1+3 child frames' GM scripts will "blip" into existence, then disappear after the initial cycle -- they never come back after any asynchronous operation. That fits some behavior described on this thread. Note that which ones "blip" and the described behavior below vary by browser / GM version, but it is not random; the patterns are weird but 100% repeatable for any given setup.

  1. The first frameset/frame will NEVER blip into existence. I've tried detaching and rebuilding the frame tag, both through window.document and unsafeWindow.document, both in the initial cycle and after a delay, but nothing will cause that frame's GM script to ever report anything to console.log. (The @include is *, with no @exclude or other URL filtering.)
  2. Some of the later behavior will differ between Firefox 52.8 / GM 4.1 and Firefox 60.0 / GM 4.3, but I can get frames' GM scripts in each case to "blip" into existence whether @noframes is set or not. This is with @includes *, no other URL filters. These should never blip with @noframes set. I have verified that window.top!==window, ie the browser knows (or should know) these these are in frames.
  3. In Firefox 52.8 / GM 4.1, The next frameset / 3 frames will ALWAYS blip into existence. In Firefox 60.0 / GM 4.3, they do not "blip" on initial frame load.
  4. In Firefox 60.0 / GM 4.3, clicking a link in one of the 3 frames that navigates another of the 3 frames (via a "target" attribute on the anchor link, not script) will "blip" -- not the new URL, but the old URL for the navigated frame. (This is one of the frames that blipped on initial load in item# 3.)
  5. Here's the weirdest part. In both browser setups --- Following the steps, we've opened the initial page with 2 layers of framesets, 4 total frames, and clicked a link in one frame to navigate a different frame on the same page. To give names to that, our initially-displayed second-level frameset had frames "top.htm", "menu.htm", and "start,htm". We've clicked a link in the "menu.htm" frame to cause the frame holding "start.htm" to navigate to "content.htm", with similar but slightly different behavior per browser setup, noted above. Now, we click a link inside of the "content.htm" frame, to navigate inside the same frame, same domain.

At this point, the script for "content.htm" will not only "blip"... it will also remain alive after a GM.xmlHttpRequest has completed -- an asynchronous event. At this point, "content.htm" is nowhere on the browser display, but its script will continue to operate.

So it appears to me that the reason GM scripts on pages within frames are not working is because the script is being loaded on page unload instead of page load. Setting @run-at to document-start and putting the script into a DOMContentReady event of unsafeWindow.document had no improvement. (Setting it on window.document never fires the event.)

-Ryan

@Sxderp

This comment has been minimized.

Copy link
Contributor

commented May 15, 2018

In Firefox 52.8 / GM 4.1, The next frameset / 3 frames will ALWAYS blip into existence. In Firefox 60.0 / GM 4.3, they do not "blip" on initial frame load.

In the transition to Firefox 57 Mozilla changed something. Whatever it was it changed (or broke) the way that Frames could be triggered. This was briefly talked about in some other issue. As a result the scripts would not run on initial frame load (57+).

The eventual switch to userScript or contentScript APIs should resolve all of this anyway.

@RyanHanekamp

This comment has been minimized.

Copy link

commented May 16, 2018

So people keep saying, yet Violentmonkey continues to execute in frames. VM doesn't properly separate userscripts from web page content (CSPs block userscripts, web pages can rewrite global objects etc.), or I'd have chucked Greasemonkey long ago. Perhaps these are related problems, but I'm using userscript engines so I don't have to explore Firefox's chrome deep enough to find out on my own.

@Cerberus-tm

This comment has been minimized.

Copy link

commented May 20, 2018

Hmm this is a difficult issue, some of my scripts no longer work because they can't work with iframes. So it seems it's not really possible to fix it, and we have to way for Mozilla to implement some API? Is there nothing we can do ourselves, like a work-around? I just need to do some stuff to a page in an iframe, often even from the same domain.

@RyanHanekamp

This comment has been minimized.

Copy link

commented May 20, 2018

My own workaround for the moment is to run Greasemonkey in top frames and Violentmonkey in child frames. I can't use Tampermonkey because it has an ambiguous homemade license, but if that's not a problem for you, then it might or might not work any better.

Bear in mind that VM drops the script into the page's own context. It's like GM's unsafeWindow without the safe equivalent. One of the most memorable times I had was debugging a script where the page content had defined a 'toJSON()" method on Array.prototype, causing JSON.stringify() to spit out invalid JSON inside my script. I've had to defensively trap and fix these as I've found them.

Another huge concern with VM is that it respects the content page's content-security-policy, so any directive that limits script sources will cause the VM script to never execute. You can see this in the browser console (but not the web console). That's why I can't run VM entirely and just get rid of GM. I hven't yet encountered a child frme with a CSP set, but when I do, it will be completely unusable.

@Cerberus-tm

This comment has been minimized.

Copy link

commented May 22, 2018

@RyanHanekamp Thanks for the tip! Perhaps I will use Violentmonkey for some scripts, then.

Does Violent also have something like synchronous GM_getValue? That's another problematic issue that breaks a ton of scripts in the new Greasemonkey. I still trust Greasemonkey the most, though, for various reasons, so I won't leave it.

As to iframes, I've been trying to replace them with object tags, which one can apparently access directly using Javascript, like so:

myObject= document.createElement('object');
myObject.setAttribute('id', 'myObject'); 
document.body.appendChild(myObject);
myObject.setAttribute('src', 'https://example.com');

Then, once the object is loaded:
document.querySelector('#myObject').contentDocument.defaultView.document.querySelectorAll('someElementInsideObjectPage')
At least this works for me in a script where the object is at the same host as the main page. You can also send messages from and to ( ... contentDocument.defaultView.postMessage('hello, object') ) the object.

@RyanHanekamp

This comment has been minimized.

Copy link

commented May 22, 2018

I'm not a VM expert, but I believe it does implement at least most of the original GM_* API. I would think it's better to adapt your scripts to asynchronous than to backtrack to a synchronous platform in the long run, though. Per my understanding, Greasemonkey did this as a performance boost within the new Quantum framework, which doesn't allow synchronous calls between background and content scripts.

As for the object solution, it won't solve my particular problem, but I'm glad others have found it useful. Aside from marshaling CSS / properties / etc and getting it to work with frames as well as iframes, I have to filter out objects in a capture process as potentially unsafe. There are ways around all of these problems, but VM was the easier interim until GM finally does what it says on the tin.

Also, if you have a same-origin frame/iframe, you can access the content ofthose directly as well. The harder part is cross-origin, which is why I need userscripts inside the frame. It sets up a window.postMessage() channel to talk back to the parent window.

@Cerberus-tm

This comment has been minimized.

Copy link

commented May 22, 2018

@RyanHanekamp Good to know that Violent Monkey still has the old, simple GM_*. I really wish Greasemonkey had kept the old, synchronous GM_getValue for backward compatibility, in addition to the new version. I might try to implement the new asynchronous function in a new script, but I'm not a programmer and I'm not sure whether I could make it work. And I am certainly unable to refactor the use of old GM_getValue in an ancient script of 2000 lines that I found online...so many scripts are broken now.

I understand why Violent was the better option for you. Let's hope Anthony or Sxderp or someone else finds out how to do implement this eventually. I really wish I could contribute, but I'm a total layman.

Oh, you can directly access the content of a same-origin iframe (without postMessage etc.)? I remember spending a lot of time trying to find a way, but it didn't seem possible. That's why I switched to postMessage as well.

@RyanHanekamp

This comment has been minimized.

Copy link

commented May 22, 2018

Frames and iframes have a contentWindow property that's equivalent to the window property. Both have a document property to access the DOM.

The hardest part of working with iframes (on the same origin) is detecting when its content is loaded, because you can't do squat until that happens. onload doesn't work how it looks. Firefox provides a DOMFrameContentLoaded event that fires for EVERY frame loaded, including grandchild / great-grandchild etc frames, which you can match to the original frame / iframe element with the event.target property. If you control the content of the frame/iframe, you can also have it talk back to the parent with either postMessage or calling a global method on the window.parent object.

Speaking of which... that's a potential workaround for this issue. If there is or could be a way to code a GM script to manually inject a userscript into a cross-domain window reference, it would take a lot more coding for the userscript creator, but it could get the job done. The pattern would be listen to DOMFrameContentLoaded, check if the event.target is first-generation, and manually inject the script if so. (Presuming that the first-generation frame's script can listen to DOMContentLoaded for second generation frames, and thus get a complete chain.) There would be no way to get @run-at dom-start behavior, and there might also be timing issues, but we could probably work around those for most use-cases.

@RyanHanekamp

This comment has been minimized.

Copy link

commented Jun 6, 2018

I've personally given up on this issue ever being resolved, and have instead moved on to directly coding an extension. Which works fine in all frames!

The difference between Greasemonkey and Violentmonkey on this point seems to be that Violentmonkey is being triggered from content scripts with all_frames set to true, while Greasemonkey has no install-time content scripts and relies entirely on the dubious ability of the background script to sniff when a tab's frame has navigated. (And Violentmonkey fails on CSP pages because it temporarily injects a SCRIPT tag instead of using the far safer tabs.executeScript().)

Put in a static content script with all_frames, run_at start, matches everything to notify the background process for start / document.DOMContentLoaded / document.Idle to trigger userscripts for each run_at, and you're good to go. A non-trivial but manageable amount of work to make this problem go away. I'd fix it myself, but I have no interest in plodding through your dev dependencies and could only produce the output code.

@poke

This comment has been minimized.

Copy link

commented Jun 7, 2018

@RyanHanekamp

and have instead moved on to directly coding an extension. Which works fine in all frames!

Would you be willing to share that extension code of yours?

@RyanHanekamp

This comment has been minimized.

Copy link

commented Jun 7, 2018

My extension isn't general purpose. The point is that using a static content_script in the manifest with all_urls, all_frames will run the script whenever any frame is loaded or navigated, and can even eval / Function constructor code just fine regardless of the page's Content-Security-Policy.

I have not tested with programmatically constructed frames / windows, but I would imagine they would run at initial creation regardless of the run_at setting, because such frames are initially created blank and then populated -- the engine would probably only see the initial creation. I also didn't test data: urls, which might require explicit matching -- I'm not sure if all_urls covers them, or just http/https.

It might not have to be a static content_script reference, either, but it appears uncertain from the documentation if dynamically-invoked content scripts load automatically when the page is navigated. My impression is that they're only injected into currently matching tabs / frames per the supplied match pattern, and navigation doesn't trigger a re-injection. But I see no downside to using a static content_script for this purpose.

Greasemonkey obviously can't include userscripts as static resources in the manifest, and content scripts don't appear to have direct access to tabs.executeScript (though I'm hardly an expert on this as I'm only a few days in), but what a static content script CAN do is message the background process to let it know that the frameId has been navigated, and to what URL. This would be more reliable than how I perceive the attempts mentioned on this thread to hook the right event in webRequest or webNavigation. The signal from the static content_script becomes the event we're looking for to trigger Greasemonkey's userscript loader / injector.

There would possibly be a delay for userscripts that strictly MUST run_at document_start. The call to the background script is asynchronous, and the document will have progressed by the time the userscript is invoked. This is quite possibly why Violentmonkey uses a temporary script tag instead of tabs.executeScript, as the script tag injection can be done directly from the content_script, synchronously. I would find being forced to work around uncertainty of the document's state for run_at document_start to be troublesome, but preferable to the script not running at all.

@arantius

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 7, 2018

Greasemonkey ... CAN [use a static content script to] message the background process to let it know that the frameId has been navigated, and to what URL ...

This is what we used to do for detecting .user.js navigations. Seems like an obvious and good solution: detect content by running a content script!

But it turns out even extension content scripts can be broken by CSP (#2631 and http://bugzil.la/1267027 and http://bugzil.la/1411641).

@RyanHanekamp

This comment has been minimized.

Copy link

commented Jun 7, 2018

I can execute my own plugin, including a Function() constructor from directly within the content_script, on Firefox 60.0.1 and 52.8.1ESR with the following CSP set:

frame-src data:; object-src 'none'; script-src 'none'; style-src 'unsafe-inline' data:; connect-src 'none'; media-src 'none';

#2631 has been closed, apparently because Firefox fixed its underlying bug. The first bugzilla is in reference to injecting script tags (the Violentmonkey method), not the content_script itself. The second is in reference to the sandbox attribute for CSP, not surprising because it forces the domain to never successfully complete a domain match even to itself. Kinda like NaN!==NaN.

@arantius

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 29, 2018

When this was first filed several months ago we did things differently. Today we use webNavigation.onCommitted to detect navigation, and in the test I'm running right now, for some reason I'm not seeing it trigger on a(n i)frame).

@louisabraham

This comment has been minimized.

Copy link

commented Dec 29, 2018

Hello, is this solved now?

I couldn't get a script I created for Tampermonkey to be launched on an iframe with Greasemonkey.

@Sxderp

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2018

The code for execution hasn't been changed on our side. So unless Mozilla has changed something on their end this is still broke. However #2663 should resolve it.

@raszpl

This comment has been minimized.

Copy link

commented Mar 17, 2019

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames.

Tampermonkey has problems with iframes in Chrome, at least for me.

@Cerberus-tm

This comment has been minimized.

Copy link

commented Mar 29, 2019

It might be worth noting that Violentmonkey and Tampermonkey work just fine inside embedded frames.

Tampermonkey has problems with iframes in Chrome, at least for me.

But it works for me in Firefox Violentmonkey. So I wonder how it can work there?

I've just noticed that a script using the old sync GM_GetValue still works fine in Violentmonkey as well. How is that possible? I thought Firefox had forced async GM.GetValue? I'm so confused now: presumably Violentmonkey had to sacrifice something else in order to still support sync and the other stuff?

@makyen

This comment has been minimized.

Copy link

commented Mar 29, 2019

@Cerberus-tm The change in Firefox meant that data can only be requested from extension storage or the background context asynchronously (the userscripts themselves are sent to the content script asynchronously). However, data could be provided to userscripts synchronously, if each GM4 content script prefetched and cached in the content script the data that's stored for each userscript being loaded in that page.

Such a cache would allow the content script to respond synchronously to the userscript's requests for data. Implementing a cache runs into issues with maintaining consistency across the various content scripts, but that's resolvable by having the content scripts listen for all changes to GM4's extension storage. In addition, if a userscript is storing a large amount of data, then caching it in every content script where the userscript is run also significantly increases the memory required for each content script.

TM and VM chose to do something similar to the above in order to not break compatibility with the original Greasemonkey APIs when those userscript managers were implemented for Chrome, which has the same restrictions wrt. asynchronous communication with extension storage, etc. Given that they are already doing it for Chrome, there was no reason for them to change when implemented for Firefox.

So, the FF57 cut-over to WebExtensions did force a rewrite of GM, but it did not force the adoption of the asynchronous APIs for GM.getValue, GM.setValue, etc. WebExtensions did make the use of async-based APIs easier to implement than synchronous would be, but it didn't make it required.

Personally, I feel that the choice, and other choices to break compatibility, were/are unfortunate. The lack of backwards compatibility with scripts which ran fine in GM3 and/or compatibility with TM result in many people choosing not to use GM4. My experience is that well more than 30 of the userscripts I use, all of which worked fine in GM3, don't work with GM4 (or at least didn't work prior to being rewritten to be compatible with GM4). There are still 28 userscripts I use daily which ran fine under GM3 which don't work with GM4.

@adamhotep

This comment has been minimized.

Copy link

commented Apr 24, 2019

I posted a workaround to this issue on Stack Overflow as an answer to how to Apply a Greasemonkey/‌Tampermonkey/‌userscript to an iframe. Basically, I'm waiting for the frame to load and then I'm operating on the window.frames array. I use a custom marker in each frame's body to denote that I've already seen the frame.

Perhaps Greasemonkey could implement a solution in a similar manner?

It'd be awesome if we also had a GM.waitFor(css_selector, action_function) like waitForKeyElements(), but that's an aside.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.