Skip to content

Conversation

btashton
Copy link
Contributor

@btashton btashton commented Oct 25, 2020

Summary

This creates a set of testdata tweak files that can be used to add additional test data on a per OS basis

Impact

This should let us get better coverage of tests that are only supported by some OS.

Testing

CI System

We can see this working here. The cxxtest is skipped on macOS even though it is in the sim.dat file.

====================================================================================
Configuration/Tool: sim/unionfs
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Building NuttX...
  Normalize sim/unionfs
====================================================================================
Skipping: sim/touchscreen
====================================================================================
Skipping: sim/cxxtest
====================================================================================

@btashton
Copy link
Contributor Author

@xiaoxiang781216 while I think the testdata files/format should probably be expanded to give us more flexibility, I think this can let us kick the problem down the road a while.

Signed-off-by: Brennan Ashton <bashton@brennanashton.com>
Copy link
Contributor

@xiaoxiang781216 xiaoxiang781216 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xiaoxiang781216 while I think the testdata files/format should probably be expanded to give us more flexibility, I think this can let us kick the problem down the road a while.

Yes, it's better to extend datlist format to filter out some host, could you review these patches:
apache/nuttx#2128
#72

@btashton
Copy link
Contributor Author

@xiaoxiang781216 while I think the testdata files/format should probably be expanded to give us more flexibility, I think this can let us kick the problem down the road a while.

Yes, it's better to extend datlist format to filter out some host, could you review these patches:
apache/nuttx#2128
#72

But with my approach we can apply modifiers to the config based on host which will be very hard to do otherwise.

I also have small test lists locally that I use. I would have to keep these updated which would also annoying.

@xiaoxiang781216
Copy link
Contributor

What's modifier you plan to apply?

@btashton
Copy link
Contributor Author

#72 (comment)

Pulling this comment in here so we can talk in one place.

Basically what I would like is to see in a single file what restrictions I have for a given host. That way I can not worry when I create a new test list about what I need to disable. And is also lets me easily see what configs for a host we may want to look at improving.

I also have wanted to set config flags for a given host, although this is awkward as it is.

@btashton
Copy link
Contributor Author

btashton commented Oct 26, 2020

I guess the biggest thing for me is not having to encode the restrictions in the the main file list.

Could we just do both and put your restriction syntax list in a single file and concat like I did? That could include your regex.

@xiaoxiang781216
Copy link
Contributor

I guess the biggest thing for me is not having to encode the restrictions in the the main file list.

Could we just do both and put your restriction syntax list in a single file and concat like I did?

The problem is that this PR make some restriction put into xxx_tweak.dat and other put into sim.dat, It's strange to mix both approach. My focus is more on the consistence, either approach is fine for me.

That could include your regex.

Yes, I think both method need regex to handle the complex case.

@btashton
Copy link
Contributor Author

Ok do you want to update to use the regex? I'm fine starting with your approach first.

@btashton
Copy link
Contributor Author

Closing in favour of apache/nuttx#2128

@btashton btashton closed this Oct 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants