New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to MiniTest #658

Closed
DAddYE opened this Issue Sep 3, 2011 · 32 comments

Comments

Projects
None yet
3 participants
@DAddYE
Member

DAddYE commented Sep 3, 2011

Since MiniTest is included in ruby 1.9 I really will be happy to include minitest/spec.

@nesquena, @achiu you approve? Community?

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Only if we add https://github.com/metaskills/mini_shoulda on top and only if we dont have to rewrite every single test in every single gem. If the combo of mini_shoulda and minitest allows minimal to no test changes then sure. I prefer the shoulda syntax over "it" and rspec dot notations plus its how all the tests already are and its wasted effort to port them all to an arbitrary new syntax.

Member

nesquena commented Sep 3, 2011

Only if we add https://github.com/metaskills/mini_shoulda on top and only if we dont have to rewrite every single test in every single gem. If the combo of mini_shoulda and minitest allows minimal to no test changes then sure. I prefer the shoulda syntax over "it" and rspec dot notations plus its how all the tests already are and its wasted effort to port them all to an arbitrary new syntax.

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

I want to remove shoulda dependency it's not a standard, we can plan to finish for 1.0. I will start to padrino-core and then others gems without breaking things.

Member

DAddYE commented Sep 3, 2011

I want to remove shoulda dependency it's not a standard, we can plan to finish for 1.0. I will start to padrino-core and then others gems without breaking things.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

I like shoulda and it doesn't hurt anything as I said mini_shoulda is a very lightweight wrapper over minitest which only aliases a few things. I think we should use that then all the tests should still pass even after we switch. If we are going to change to something completely different, I would prefer riot since @achiu is a contributor. But I don't want to waste our time switching except maybe to minitest + minishoulda (since it uses same syntax). What do you think @achiu? We have hundreds of tests, I really think our time can be better spent towards 1.0 then to just jiggle around all the tests syntax for no real benefit...

Member

nesquena commented Sep 3, 2011

I like shoulda and it doesn't hurt anything as I said mini_shoulda is a very lightweight wrapper over minitest which only aliases a few things. I think we should use that then all the tests should still pass even after we switch. If we are going to change to something completely different, I would prefer riot since @achiu is a contributor. But I don't want to waste our time switching except maybe to minitest + minishoulda (since it uses same syntax). What do you think @achiu? We have hundreds of tests, I really think our time can be better spent towards 1.0 then to just jiggle around all the tests syntax for no real benefit...

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

For instance I am 85% done with full YARD docs for helper gem. Rather than spending all that time to switch tests for no benefit. Please spend the time instead on better docs which really help.

Member

nesquena commented Sep 3, 2011

For instance I am 85% done with full YARD docs for helper gem. Rather than spending all that time to switch tests for no benefit. Please spend the time instead on better docs which really help.

@achiu

This comment has been minimized.

Show comment
Hide comment
@achiu

achiu Sep 3, 2011

Member

i would push for riot. Recently I submitted some patches to remove warnings and I hope to do the same for padrino so that we can run without warnings as well. If there is any test framework change, I'm definitely all for Riot. Otherwise, I would side we the fact we can use a minishoulda framework to keep the test consistent and passing.

Member

achiu commented Sep 3, 2011

i would push for riot. Recently I submitted some patches to remove warnings and I hope to do the same for padrino so that we can run without warnings as well. If there is any test framework change, I'm definitely all for Riot. Otherwise, I would side we the fact we can use a minishoulda framework to keep the test consistent and passing.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Agreed. I say we either switch to minitest+minishoulda (cause its so simple to do, same syntax) or to riot since @achiu is a core member and its cool. My vote still stands with stick with the existing ones and focus our time on better things like documentation or fixing bugs.

Member

nesquena commented Sep 3, 2011

Agreed. I say we either switch to minitest+minishoulda (cause its so simple to do, same syntax) or to riot since @achiu is a core member and its cool. My vote still stands with stick with the existing ones and focus our time on better things like documentation or fixing bugs.

@achiu

This comment has been minimized.

Show comment
Hide comment
@achiu

achiu Sep 3, 2011

Member

Also, I don't like the argument of changing to something standard. Padrino itself is hardly considered standard at the moment. In addition, changing the test suite to accomodate the general consensus isn't going to achieve a substantial increase in contribution. It's the users that use it and want to make it better as a whole, not because of the test framework, but because of the framework itself. Lastly, @nesquena is right, the focus really is to spend time on better documentation, YARD conversion, and creating a stronger test suite.

Member

achiu commented Sep 3, 2011

Also, I don't like the argument of changing to something standard. Padrino itself is hardly considered standard at the moment. In addition, changing the test suite to accomodate the general consensus isn't going to achieve a substantial increase in contribution. It's the users that use it and want to make it better as a whole, not because of the test framework, but because of the framework itself. Lastly, @nesquena is right, the focus really is to spend time on better documentation, YARD conversion, and creating a stronger test suite.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Yeah for instance, lets channel our energy into new integration tests too. I plan on porting the blog tutorial to an executable integration test as I think the key to our success is being able to release without breaking things and to feel more confident about every version.

Member

nesquena commented Sep 3, 2011

Yeah for instance, lets channel our energy into new integration tests too. I plan on porting the blog tutorial to an executable integration test as I think the key to our success is being able to release without breaking things and to feel more confident about every version.

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

I don't want to chose a new testing framework, but a "standard", I love rspec and (never used in a project) riot seems very very nice, but Im asking that because:

  • I need to benchmark test (almost the time duration)
  • I prefer to don't handle external dependencies

Why that? If we are using external testing stuff for me is quite complex to be sure what slow down things.
Second, I've made padrino compat with 1.9, RBX and JRuby and I got some problems with external deps, this because they can test almost the most used testing frameworks. For instance, try to run test under 1.9.3 we have problems with DM and If I remember correctly with webrat.
Finally there was the rake fucking problem, don't tell me why but I was unable to run test due to a problem with rdoc and others things, that's was why I changed a lot of things.

Switch to 'minitest/spec' require very very few changes, I don't want to change all assert and using it we can alias it to should and or even better rename it. Nothing big.

That's is only for my testing/benchmark reasons.

Member

DAddYE commented Sep 3, 2011

I don't want to chose a new testing framework, but a "standard", I love rspec and (never used in a project) riot seems very very nice, but Im asking that because:

  • I need to benchmark test (almost the time duration)
  • I prefer to don't handle external dependencies

Why that? If we are using external testing stuff for me is quite complex to be sure what slow down things.
Second, I've made padrino compat with 1.9, RBX and JRuby and I got some problems with external deps, this because they can test almost the most used testing frameworks. For instance, try to run test under 1.9.3 we have problems with DM and If I remember correctly with webrat.
Finally there was the rake fucking problem, don't tell me why but I was unable to run test due to a problem with rdoc and others things, that's was why I changed a lot of things.

Switch to 'minitest/spec' require very very few changes, I don't want to change all assert and using it we can alias it to should and or even better rename it. Nothing big.

That's is only for my testing/benchmark reasons.

@achiu

This comment has been minimized.

Show comment
Hide comment
@achiu

achiu Sep 3, 2011

Member

If the changes to minitest show very minimal need to convert our current test, then I would be OK with this. I am not that familiar with minitest/spec so I will need to look it over. I see that perhaps benchmarking this can be a useful addition to remove external dependencies though. How about 1.8.7/ree compatiblity? Is that still in consideration or is that something we are dropping? Ideally we should not need to alias methods, rather just use the test framework as is.

Member

achiu commented Sep 3, 2011

If the changes to minitest show very minimal need to convert our current test, then I would be OK with this. I am not that familiar with minitest/spec so I will need to look it over. I see that perhaps benchmarking this can be a useful addition to remove external dependencies though. How about 1.8.7/ree compatiblity? Is that still in consideration or is that something we are dropping? Ideally we should not need to alias methods, rather just use the test framework as is.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

If we alias a few methods we can use the exact same syntax we already use: https://github.com/metaskills/mini_shoulda/blob/master/lib/mini_shoulda/spec.rb . This is what I am saying. If we just use minitest and this small gem or do the aliases ourselves, we will get a huge speedup and wont have to change a shitload of tests.

Member

nesquena commented Sep 3, 2011

If we alias a few methods we can use the exact same syntax we already use: https://github.com/metaskills/mini_shoulda/blob/master/lib/mini_shoulda/spec.rb . This is what I am saying. If we just use minitest and this small gem or do the aliases ourselves, we will get a huge speedup and wont have to change a shitload of tests.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Also minitest does exist for 1.8.7 its just a separate gem (minitest) and I don't think we should drop support for them at all.

Member

nesquena commented Sep 3, 2011

Also minitest does exist for 1.8.7 its just a separate gem (minitest) and I don't think we should drop support for them at all.

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

never got problems with 1.8.7/REE, Obviously I totally Agree that we can do that only if it requires very few changes, I don't wanna waste my time so much :D

Member

DAddYE commented Sep 3, 2011

never got problems with 1.8.7/REE, Obviously I totally Agree that we can do that only if it requires very few changes, I don't wanna waste my time so much :D

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

This is literally the entire mini shoulda gem just to be clear:

class MiniTest::Spec < MiniTest::Unit::TestCase

  class << self
    alias :setup :before unless defined?(Rails)
    alias :teardown :after unless defined?(Rails)
    alias :should :it
    alias :context :describe
  end

  def self.should_eventually(desc)
    it("should eventually #{desc}") { skip("Should eventually #{desc}") }
  end

end
Member

nesquena commented Sep 3, 2011

This is literally the entire mini shoulda gem just to be clear:

class MiniTest::Spec < MiniTest::Unit::TestCase

  class << self
    alias :setup :before unless defined?(Rails)
    alias :teardown :after unless defined?(Rails)
    alias :should :it
    alias :context :describe
  end

  def self.should_eventually(desc)
    it("should eventually #{desc}") { skip("Should eventually #{desc}") }
  end

end
@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

OK then I think we are more or less on the same page again? haha. Let's use minitest and just pull in the code for aliasing above into our test helpers to avoid the external dependency but still be able to use our existing syntax. This is what I plan to do for all future projects of mine. minitest is super lightweight and fast and I can still use my shoulda syntax :)

Member

nesquena commented Sep 3, 2011

OK then I think we are more or less on the same page again? haha. Let's use minitest and just pull in the code for aliasing above into our test helpers to avoid the external dependency but still be able to use our existing syntax. This is what I plan to do for all future projects of mine. minitest is super lightweight and fast and I can still use my shoulda syntax :)

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

Hahah Nate, you are a shoulda lover :D, you love shoulda but not rspec? Why???? :D

Mine is only because I prefer to haven't external deps, yep, true minitest is not buitin in 1.8, but removing the actual shoulda (which is big) and very complex we can use instead minitest on rubies that haven't it.

Member

DAddYE commented Sep 3, 2011

Hahah Nate, you are a shoulda lover :D, you love shoulda but not rspec? Why???? :D

Mine is only because I prefer to haven't external deps, yep, true minitest is not buitin in 1.8, but removing the actual shoulda (which is big) and very complex we can use instead minitest on rubies that haven't it.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Yes thats why this works so well. We can remove the dependencies altogether and just use minitest with the simple aliases:

class MiniTest::Spec < MiniTest::Unit::TestCase

  class << self
    alias :setup :before unless defined?(Rails)
    alias :teardown :after unless defined?(Rails)
    alias :should :it
    alias :context :describe
  end

  def self.should_eventually(desc)
    it("should eventually #{desc}") { skip("Should eventually #{desc}") }
  end

end

and we can all be happy except maybe arthur cause we can't use riot haha :P

Member

nesquena commented Sep 3, 2011

Yes thats why this works so well. We can remove the dependencies altogether and just use minitest with the simple aliases:

class MiniTest::Spec < MiniTest::Unit::TestCase

  class << self
    alias :setup :before unless defined?(Rails)
    alias :teardown :after unless defined?(Rails)
    alias :should :it
    alias :context :describe
  end

  def self.should_eventually(desc)
    it("should eventually #{desc}") { skip("Should eventually #{desc}") }
  end

end

and we can all be happy except maybe arthur cause we can't use riot haha :P

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Oh and rspec is huge and complex. Are you kidding me, shoulda is like a 5 yr old child compared to how heavy rspec is. And its just as slow as shoulda I have used it on large projects its not fast. If I wanted to switch to something with an external dependency and a crazy syntax, it would never be rspec still. It would be something like riot or test/spec. rspec is basically the worst choice up for unit testing IMO.

Member

nesquena commented Sep 3, 2011

Oh and rspec is huge and complex. Are you kidding me, shoulda is like a 5 yr old child compared to how heavy rspec is. And its just as slow as shoulda I have used it on large projects its not fast. If I wanted to switch to something with an external dependency and a crazy syntax, it would never be rspec still. It would be something like riot or test/spec. rspec is basically the worst choice up for unit testing IMO.

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

Yep, totally agree with you, the only nice thing of rspec is the most comprehensive output, basically I choose it when output is most important. Don't tell me to use cucumber which I hate :D

Member

DAddYE commented Sep 3, 2011

Yep, totally agree with you, the only nice thing of rspec is the most comprehensive output, basically I choose it when output is most important. Don't tell me to use cucumber which I hate :D

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

I'll give you that, output is nice but its not enough for me to justify it over minitest which is so much faster. And yes I agree with you, I hate cucumber. Rspec + Cucumber (standard or not) are the worst options for unit and integration testing that I have seen so far in ruby. I don't care if they are popular or not. Minitest + Capybara for the win :)

Member

nesquena commented Sep 3, 2011

I'll give you that, output is nice but its not enough for me to justify it over minitest which is so much faster. And yes I agree with you, I hate cucumber. Rspec + Cucumber (standard or not) are the worst options for unit and integration testing that I have seen so far in ruby. I don't care if they are popular or not. Minitest + Capybara for the win :)

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 3, 2011

Member

Minitest + Capybara for the win :)

Amen Man!

Member

DAddYE commented Sep 3, 2011

Minitest + Capybara for the win :)

Amen Man!

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Good article: http://metaskills.net/2011/01/25/from-test-unit-shoulda-to-minitest-spec-minishoulda/ @DAddYE Are you working on this, I have been playing around and already got minitest/with aliases passing on padrino-helpers! I can probably finish this off? unless you are working on it too same time :)

Member

nesquena commented Sep 3, 2011

Good article: http://metaskills.net/2011/01/25/from-test-unit-shoulda-to-minitest-spec-minishoulda/ @DAddYE Are you working on this, I have been playing around and already got minitest/with aliases passing on padrino-helpers! I can probably finish this off? unless you are working on it too same time :)

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

(pushed to a minitest branch for now)

Member

nesquena commented Sep 3, 2011

(pushed to a minitest branch for now)

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

@achiu @DAddYE Check it: https://github.com/padrino/padrino-framework/commits/minitest I think I got every single test passing with almost no code changes. Shoulda is gone, just using minitest! Can keep all the same syntax tho thanks to the strategic aliases

Member

nesquena commented Sep 3, 2011

@achiu @DAddYE Check it: https://github.com/padrino/padrino-framework/commits/minitest I think I got every single test passing with almost no code changes. Shoulda is gone, just using minitest! Can keep all the same syntax tho thanks to the strategic aliases

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 3, 2011

Member

Can you guys run the tests, check the commits (I had to comment out two tests for weird reasons) and if they all pass and it looks good, please merge this back to master.

Member

nesquena commented Sep 3, 2011

Can you guys run the tests, check the commits (I had to comment out two tests for weird reasons) and if they all pass and it looks good, please merge this back to master.

@achiu

This comment has been minimized.

Show comment
Hide comment
@achiu

achiu Sep 3, 2011

Member

wow, nice job @nesquena!

Member

achiu commented Sep 3, 2011

wow, nice job @nesquena!

nesquena added a commit that referenced this issue Sep 5, 2011

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 6, 2011

Member

W00T, @achiu, can you add a generator for minitest? One for minitest/spec and one for minitest/unit

Member

DAddYE commented Sep 6, 2011

W00T, @achiu, can you add a generator for minitest? One for minitest/spec and one for minitest/unit

@achiu

This comment has been minimized.

Show comment
Hide comment
@achiu

achiu Sep 6, 2011

Member

@DAddYE yeah I will add that

Member

achiu commented Sep 6, 2011

@DAddYE yeah I will add that

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 6, 2011

Member

I would just take the liberty and presume they want to use minitest spec. Having two minitest components that are separate seems a bit redundant.

Member

nesquena commented Sep 6, 2011

I would just take the liberty and presume they want to use minitest spec. Having two minitest components that are separate seems a bit redundant.

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 6, 2011

Member

should just be -t minitest and thats it. if they dont want to use spec they can comment out the one line.

Member

nesquena commented Sep 6, 2011

should just be -t minitest and thats it. if they dont want to use spec they can comment out the one line.

@DAddYE

This comment has been minimized.

Show comment
Hide comment
@DAddYE

DAddYE Sep 7, 2011

Member

I agree with @nesquena, in the meanwhile should we remove support for old test/unit and shoulda ?

Member

DAddYE commented Sep 7, 2011

I agree with @nesquena, in the meanwhile should we remove support for old test/unit and shoulda ?

@nesquena

This comment has been minimized.

Show comment
Hide comment
@nesquena

nesquena Sep 12, 2011

Member

@achiu @DAddYE I say we keep shoulda (for now) since its still as valid as a tool as rspec even if its less popular but I do think a minitest component should also go into core as we discussed. I am going to create a separate ticket for that and close this.

Member

nesquena commented Sep 12, 2011

@achiu @DAddYE I say we keep shoulda (for now) since its still as valid as a tool as rspec even if its less popular but I do think a minitest component should also go into core as we discussed. I am going to create a separate ticket for that and close this.

@nesquena nesquena closed this Sep 12, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment