Remove Blank Columns upon project creation/import

[jenyoung]

Is your feature request related to a problem or area of OpenRefine? Please describe.

Creating/importing a project allows for the removal of blank rows. It would be great to also be able to remove blank columns.

Describe the solution you'd like
Have a similar functionality to Store blank rows with a Store blank columns option.

Additional context
This was discussed on the google group 9/19/2018.

[thadguidry]

Dissimilar to #1472 the Remove Rows with empty cells

[thadguidry]

@wetneb @ostephens could we squeeze this into 3.2 milestone?

[wetneb]

@thadguidry I won't have time to work on this soon, but if someone wants to work to implement this before 3.2 I'm happy to get that on board, yes.

[tommowlam]

This will be particularly useful when importing XML into OpenRefine, where tags that are used for structure but have no direct content themselves, create lots of empty columns that have to then be 'manually' removed - eg:

    <poster>    
      <name>  
        <firstname>Tom</firstname>  
        <lastname>Mowlam</lastname>  
      </name>  
    </poster>

...gives empty 'poster' and 'name' columns on import.

[aquaruiz]

May I work on this bug?

[wetneb]

Sure, thanks a lot!

[aquaruiz]

Hey, I did some fixes but I don't get it how to rebuild to check if it works...
./refine build
./refine
Is this all?

[thadguidry]

@aquaruiz Yes, that's how you can do it manually (if not using Maven config and running from directly in your IDE)

[cbedard]

It doesn't look like this has been worked on. Can I work on it?

[thadguidry]

Yes, I think so @CameronBedard , I don't see @aquaruiz ever made a PR for us to look at and review for merge.

[wsmmxmm]

Hi team! I am an outreachy applicant. Please can I work on this issue if it is still open?

[wsmmxmm]

Hi Team,

Congratulations to me because I run OpenRefine from the command line successfully!

Before I start, I would like to describe this feature and ask some questions I'm not sure about.

• Functionality: Store blank columns ( it's similar to Store blank rows)
  
• Supported importer:
• Tabular importer: This part needs to be added.
• Tree-based importer: I'm not very sure if I need to do this because it's "the option would be less useful for the tree-based importers" from @wetneb in Added 'Store blank columns' importing feature  #3497 . Maybe later?
• Testing: I'm not very sure how to do that by "be testing TabularImportingParserBase directly". I plan to create a few different types of documents to test first when I'm done.

[wetneb]

Congratulations for running OpenRefine! That is an excellent start :)
For tree-based importers, I guess they can still produce some empty columns, for instance if importing the following payload:

[
    {
       "foo": "1",
       "bar": null
    },
    {
        "foo": "2",
        "bar": null
    }
]

So I guess it is probably worth offering in tree importers as well, after all.

So this would end up being an option that would be available to basically all importers (except perhaps the line-based importer which only creates one column anyway). It feels like there should be a better way than manually adding the option to all those importers with the corresponding tests (which is a lot of work). I guess it should ideally be treated as a post-processing step independent of the importer, but we have not got any such notion of post-processing step.

With that in mind I would challenge the "good first issue" tag that I added myself - but that does not mean it is not doable either.

[wsmmxmm]

well, sorry I'm not familiar enough with structure and code at the moment, so I can't suggest more about "post-processing step". It sounds like something that requires a lot of planning for the overall process.
I think I could continue to do this by manually adding the option with notes（I mean →“//”). This way we can easily find this section if we need to further action about "post-processing step". Through this process I could also become more familiar with the structure of each importer. Any other advice?

[wetneb]

That sounds good to me!