New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test sandbox #6

Open
csnover opened this Issue Apr 26, 2013 · 11 comments

Comments

Projects
None yet
9 participants
@csnover
Member

csnover commented Apr 26, 2013

Provide an opt-in sandbox interface for tests that gives them a fresh window context to do things to.

@lbod

This comment has been minimized.

Show comment
Hide comment
@lbod

lbod Nov 13, 2013

Contributor

@csnover I wondered if you had advice on this for users if it doesn't make it into intern?
I saw on SO you answered a question

so if you’re going to modify native objects for the purposes of testing, you’ll either need to restore them yourself later (in your suite teardown) or create your own sandboxing (by creating a new document or a new frame, depending upon what you are actually trying to test)

Is there a pattern end users can implement to achieve this (without teardown)? Would the test fixture need to create the iframe? If so how could topics from the test be published to ClientSuite? e.g. /suite/end and /client/end

Contributor

lbod commented Nov 13, 2013

@csnover I wondered if you had advice on this for users if it doesn't make it into intern?
I saw on SO you answered a question

so if you’re going to modify native objects for the purposes of testing, you’ll either need to restore them yourself later (in your suite teardown) or create your own sandboxing (by creating a new document or a new frame, depending upon what you are actually trying to test)

Is there a pattern end users can implement to achieve this (without teardown)? Would the test fixture need to create the iframe? If so how could topics from the test be published to ClientSuite? e.g. /suite/end and /client/end

wkeese added a commit to ibm-js/delite that referenced this issue Nov 21, 2013

Workaround lack of Intern sandboxing.
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@lbod

This comment has been minimized.

Show comment
Hide comment
@lbod

lbod Nov 21, 2013

Contributor

I've ran into a fundamental problem due to there being no sandboxing in intern.
Custom components i.e. polymer/web components, means all components will need to be name spaced

see https://dvcs.w3.org/hg/webcomponents/raw-file/d7d1b718de45/spec/custom/index.html#terminology

An element definition can only be registered with one document

Whilst name spacing the elements will work, it's more than cumbersome and sand boxing via an iframe would seem a much safer option all round

Contributor

lbod commented Nov 21, 2013

I've ran into a fundamental problem due to there being no sandboxing in intern.
Custom components i.e. polymer/web components, means all components will need to be name spaced

see https://dvcs.w3.org/hg/webcomponents/raw-file/d7d1b718de45/spec/custom/index.html#terminology

An element definition can only be registered with one document

Whilst name spacing the elements will work, it's more than cumbersome and sand boxing via an iframe would seem a much safer option all round

@csnover

This comment has been minimized.

Show comment
Hide comment
@csnover

csnover Nov 21, 2013

Member

The asymmetrical registration API in Web Components is probably a spec bug. Also, it’s kind of a bad spec. Inflexible, high risk of collisions (as you see).

Member

csnover commented Nov 21, 2013

The asymmetrical registration API in Web Components is probably a spec bug. Also, it’s kind of a bad spec. Inflexible, high risk of collisions (as you see).

@lbod

This comment has been minimized.

Show comment
Hide comment
@lbod

lbod Nov 21, 2013

Contributor

I don't agree it's a spec bug, it's clearly written in the draft; it is still draft though so could change but I wouldn't expect it to e.g. you wouldn't expect 2 HTMLElement elements defined in a document
However, this isn't entirely relevant to the issue, it's just an example of where teardown won't work in unit tests. If you have to unit test a large number of tests it's almost certain collisions of some sort will occur.

What would be helpful, if you could give me any pointers to implement sandboxing in intern unit tests whether it's an iframe or otherwise & I'm happy to do the work & submit a PR

Contributor

lbod commented Nov 21, 2013

I don't agree it's a spec bug, it's clearly written in the draft; it is still draft though so could change but I wouldn't expect it to e.g. you wouldn't expect 2 HTMLElement elements defined in a document
However, this isn't entirely relevant to the issue, it's just an example of where teardown won't work in unit tests. If you have to unit test a large number of tests it's almost certain collisions of some sort will occur.

What would be helpful, if you could give me any pointers to implement sandboxing in intern unit tests whether it's an iframe or otherwise & I'm happy to do the work & submit a PR

wkeese added a commit to ibm-js/delite that referenced this issue Dec 25, 2013

Workaround lack of Intern sandboxing.
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.

wkeese added a commit to ibm-js/delite that referenced this issue Dec 25, 2013

Workaround lack of Intern sandboxing.
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@nickcmaynard

This comment has been minimized.

Show comment
Hide comment
@nickcmaynard

nickcmaynard Mar 20, 2014

Definitely a +1 from me. In DOH, I addressed this issue by writing a wrapper page to test invididual tests. In Intern, I'm finding it very difficult.

Manually tearing down tests is a pain, particularly when you've done some DOM manipulations, or altered some global state of your app.

I'd hope to see this configurable on both suite and test function levels.

nickcmaynard commented Mar 20, 2014

Definitely a +1 from me. In DOH, I addressed this issue by writing a wrapper page to test invididual tests. In Intern, I'm finding it very difficult.

Manually tearing down tests is a pain, particularly when you've done some DOM manipulations, or altered some global state of your app.

I'd hope to see this configurable on both suite and test function levels.

wkeese added a commit to ibm-js/delite that referenced this issue Oct 23, 2014

Workaround lack of Intern sandboxing.
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@hakatashi

This comment has been minimized.

Show comment
Hide comment
@hakatashi

hakatashi Mar 20, 2016

@jason0x43 @dylans I'm very interested to work for solving this issue as my GSoC 2016 project. I'll apply, but I have several questions.

  • Are there any expectation of the way to implement this feature? At first glance I think iframe (for browsers) and child process spawning (for Node.js) is enough to accomplish this task. But I'm not sure this is the right way.
  • Are there any chance to make good use of my experience of coding with TypeScript? While Ideas List page is listing TypeScript as a part of knowledge prerequisite for this task, I couldn't find any working TypeScript code in this repository (except for typings files).

Thanks.

hakatashi commented Mar 20, 2016

@jason0x43 @dylans I'm very interested to work for solving this issue as my GSoC 2016 project. I'll apply, but I have several questions.

  • Are there any expectation of the way to implement this feature? At first glance I think iframe (for browsers) and child process spawning (for Node.js) is enough to accomplish this task. But I'm not sure this is the right way.
  • Are there any chance to make good use of my experience of coding with TypeScript? While Ideas List page is listing TypeScript as a part of knowledge prerequisite for this task, I couldn't find any working TypeScript code in this repository (except for typings files).

Thanks.

@dylans

This comment has been minimized.

Show comment
Hide comment
@dylans

dylans Mar 20, 2016

@hakatashi To answer the second question, there is a fairly substantial work in progress towards Intern 4 that is rewritten in TypeScript at https://github.com/theintern/intern/tree/experimental-ts

dylans commented Mar 20, 2016

@hakatashi To answer the second question, there is a fairly substantial work in progress towards Intern 4 that is rewritten in TypeScript at https://github.com/theintern/intern/tree/experimental-ts

@kitsonk

This comment has been minimized.

Show comment
Hide comment
@kitsonk

kitsonk Mar 20, 2016

Contributor

@hakatashi for the first point, I am not sure we would want to pre-suppose a solution. For Node.js I suspect running scripts in a sandbox context as a core API would produce better results. Also with browsers, an iFrame wouldn't be wholly different than loading a page in Selinium. I would expect that a part of the proposal would be to assess what different methods might work in discussion with the mentor/team, because I suspect there maybe challenges with any approach. I think the main thing would be focusing on making the consumable API as transparent as possible for the end user to not have to do a lot of work to "sandbox" themselves.

As @dylans says, I think we would consider supporting development of this feature in TypeScript even if we aren't quite ready with the rest of Intern being on TS.

Contributor

kitsonk commented Mar 20, 2016

@hakatashi for the first point, I am not sure we would want to pre-suppose a solution. For Node.js I suspect running scripts in a sandbox context as a core API would produce better results. Also with browsers, an iFrame wouldn't be wholly different than loading a page in Selinium. I would expect that a part of the proposal would be to assess what different methods might work in discussion with the mentor/team, because I suspect there maybe challenges with any approach. I think the main thing would be focusing on making the consumable API as transparent as possible for the end user to not have to do a lot of work to "sandbox" themselves.

As @dylans says, I think we would consider supporting development of this feature in TypeScript even if we aren't quite ready with the rest of Intern being on TS.

@hakatashi

This comment has been minimized.

Show comment
Hide comment
@hakatashi

hakatashi Mar 20, 2016

@dylans wow, thanks for the guidance. It seems relaxing.

@kitsonk Fmm... OK. So, I'll include a certain period of time to investigate and discuss the way to implement this feature. Maybe I was hasty to jump to the conclusions.

hakatashi commented Mar 20, 2016

@dylans wow, thanks for the guidance. It seems relaxing.

@kitsonk Fmm... OK. So, I'll include a certain period of time to investigate and discuss the way to implement this feature. Maybe I was hasty to jump to the conclusions.

@kitsonk

This comment has been minimized.

Show comment
Hide comment
@kitsonk

kitsonk Mar 20, 2016

Contributor

@hakatashi feel free to jump to conclusions! I am really glad to see interest.

Contributor

kitsonk commented Mar 20, 2016

@hakatashi feel free to jump to conclusions! I am really glad to see interest.

@hakatashi hakatashi referenced a pull request that will close this issue Aug 4, 2016

Open

Implement sandbox environment to the test API #656

@indolering

This comment has been minimized.

Show comment
Hide comment
@indolering

indolering Nov 18, 2016

I really like WebComponentTester's setup, in which they run each module its own iFrame and allow you to navigate over to the individual page.

indolering commented Nov 18, 2016

I really like WebComponentTester's setup, in which they run each module its own iFrame and allow you to navigate over to the individual page.

@morrinene morrinene added this to the Intern 4.0 milestone Feb 9, 2017

@jason0x43 jason0x43 removed this from the Intern 4.0 milestone Aug 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment