Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test sandbox #6

Open
csnover opened this issue Apr 26, 2013 · 11 comments
Open

Test sandbox #6

csnover opened this issue Apr 26, 2013 · 11 comments
Labels
effort-high This will take a while enhancement A new or improved feature

Comments

@csnover
Copy link
Member

csnover commented Apr 26, 2013 •
edited by morrinene
Loading

Provide an opt-in sandbox interface for tests that gives them a fresh window context to do things to.
@lbod
Copy link
Contributor

lbod commented Nov 13, 2013

@csnover I wondered if you had advice on this for users if it doesn't make it into intern?
I saw on SO you answered a question

so if you’re going to modify native objects for the purposes of testing, you’ll either need to restore them yourself later (in your suite teardown) or create your own sandboxing (by creating a new document or a new frame, depending upon what you are actually trying to test)

Is there a pattern end users can implement to achieve this (without teardown)? Would the test fixture need to create the iframe? If so how could topics from the test be published to ClientSuite? e.g. /suite/end and /client/end

wkeese added a commit to ibm-js/delite that referenced this issue Nov 21, 2013
@wkeese
Workaround lack of Intern sandboxing.
aff3541 
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@lbod
Copy link
Contributor

lbod commented Nov 21, 2013

I've ran into a fundamental problem due to there being no sandboxing in intern.
Custom components i.e. polymer/web components, means all components will need to be name spaced

see https://dvcs.w3.org/hg/webcomponents/raw-file/d7d1b718de45/spec/custom/index.html#terminology

An element definition can only be registered with one document

Whilst name spacing the elements will work, it's more than cumbersome and sand boxing via an iframe would seem a much safer option all round

@csnover
Copy link
Member Author

csnover commented Nov 21, 2013

The asymmetrical registration API in Web Components is probably a spec bug. Also, it’s kind of a bad spec. Inflexible, high risk of collisions (as you see).

@lbod
Copy link
Contributor

lbod commented Nov 21, 2013

I don't agree it's a spec bug, it's clearly written in the draft; it is still draft though so could change but I wouldn't expect it to e.g. you wouldn't expect 2 HTMLElement elements defined in a document
However, this isn't entirely relevant to the issue, it's just an example of where teardown won't work in unit tests. If you have to unit test a large number of tests it's almost certain collisions of some sort will occur.

What would be helpful, if you could give me any pointers to implement sandboxing in intern unit tests whether it's an iframe or otherwise & I'm happy to do the work & submit a PR

wkeese added a commit to ibm-js/delite that referenced this issue Dec 25, 2013
@wkeese
Workaround lack of Intern sandboxing.
8c38ed3 
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
wkeese added a commit to ibm-js/delite that referenced this issue Dec 25, 2013
@wkeese
Workaround lack of Intern sandboxing.
977fd85 
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@nickcmaynard
Copy link

Definitely a +1 from me. In DOH, I addressed this issue by writing a wrapper page to test invididual tests. In Intern, I'm finding it very difficult.

Manually tearing down tests is a pain, particularly when you've done some DOM manipulations, or altered some global state of your app.

I'd hope to see this configurable on both suite and test function levels.

wkeese added a commit to ibm-js/delite that referenced this issue Oct 23, 2014
@wkeese
Workaround lack of Intern sandboxing.
ce581c2 
See theintern/intern#6.

Even if we made a dui/register.clear() method, we still couldn't use the same
widget name in different test files because the native document.register() call
throws an exception if you try to register the same name twice.

Therefore, our only option is to make widget names unique across all the test files.
@csnover csnover added help-wanted effort-high This will take a while labels May 22, 2015
@hakatashi
Copy link

@jason0x43 @dylans I'm very interested to work for solving this issue as my GSoC 2016 project. I'll apply, but I have several questions.

  • Are there any expectation of the way to implement this feature? At first glance I think iframe (for browsers) and child process spawning (for Node.js) is enough to accomplish this task. But I'm not sure this is the right way.
  • Are there any chance to make good use of my experience of coding with TypeScript? While Ideas List page is listing TypeScript as a part of knowledge prerequisite for this task, I couldn't find any working TypeScript code in this repository (except for typings files).

Thanks.

@dylans
Copy link

dylans commented Mar 20, 2016

@hakatashi To answer the second question, there is a fairly substantial work in progress towards Intern 4 that is rewritten in TypeScript at https://github.com/theintern/intern/tree/experimental-ts

@kitsonk
Copy link
Contributor

kitsonk commented Mar 20, 2016

@hakatashi for the first point, I am not sure we would want to pre-suppose a solution. For Node.js I suspect running scripts in a sandbox context as a core API would produce better results. Also with browsers, an iFrame wouldn't be wholly different than loading a page in Selinium. I would expect that a part of the proposal would be to assess what different methods might work in discussion with the mentor/team, because I suspect there maybe challenges with any approach. I think the main thing would be focusing on making the consumable API as transparent as possible for the end user to not have to do a lot of work to "sandbox" themselves.

As @dylans says, I think we would consider supporting development of this feature in TypeScript even if we aren't quite ready with the rest of Intern being on TS.

@hakatashi
Copy link

@dylans wow, thanks for the guidance. It seems relaxing.

@kitsonk Fmm... OK. So, I'll include a certain period of time to investigate and discuss the way to implement this feature. Maybe I was hasty to jump to the conclusions.

@kitsonk
Copy link
Contributor

kitsonk commented Mar 20, 2016

@hakatashi feel free to jump to conclusions! I am really glad to see interest.

@hakatashi hakatashi mentioned this issue Aug 4, 2016
Closed
@indolering
Copy link

I really like WebComponentTester's setup, in which they run each module its own iFrame and allow you to navigate over to the individual page.

@morrinene morrinene added this to the Intern 4.0 milestone Feb 9, 2017
@jason0x43 jason0x43 removed this from the Intern 4.0 milestone Aug 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
effort-high This will take a while enhancement A new or improved feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement sandbox environment to the test API hakatashi/intern
9 participants
@indolering @dylans @csnover @lbod @jason0x43 @kitsonk @nickcmaynard @hakatashi @morrinene