[Feature Request]: Read missing values from sav/readstat into column empty values

[JorisGoosen]

Description

SPSS files can contain well defined per column missing values. These are currently read by JASP and are interpreted correctly as missing values. But they are not stored in the column.

Purpose

Make use of added functionality from jasp-stats/jasp-desktop#5276 to store missing values properly from SPSS

Use-case

pretty obvious I think

[tomtomme]

All stuff related to "missing values"
#2575
#2500
#2437
#396

[JorisGoosen]

@tomtomme do you have an example sav file for this?
I dont really want to buy a license for SPSS just to test this

[tomtomme]

Not until tuesday. If you need it earlier you could produce one with pspp. It is open source.
I have no access to a pc atm.

[JorisGoosen]

No hurry!

[shun2wang]

Here are some dataset of SPSS made: https://github.com/diazbastian/spss-sample-documents/tree/master/data-files/created-with-spss/spss-examples

[tomtomme]

@JorisGoosen
Here is a nice big one with lots of labels and missing values defined
missings.sav.zip

The first 8 variables should cover most combinations of variable type and sort of missing value (range, single value, string etc.)

Besides the information of the "missing values" we should also import description, type etc.
So from the screenshot below the columns number:
1 Name
2 type (numeric or string or date etc.)
5 Description
6 Labels
7 Missing Values
10 type (scale, ordinal, nominal, nominal text)

[JorisGoosen]

Im going to leave out the support for the min<X<max missing values because it doesnt really fit with the way they work in JASP.

Unless of course this is an indispensable feature, then Im sure we could cobble something together, but it would probably require some extra stuff in JASP.

[shun2wang]

Here in readstat has a metadata for dataset files type and "missing", and also in readstat.h wich will including some custom missing value and system missing.

[tomtomme]

@JorisGoosen
It is lower prio I think. So from my perspective leave it out for now.

Did you close the issue on purpose? Is this already implemented?

[JorisGoosen]

I closed it on purpose indeed, see: jasp-stats/jasp-desktop@79ef8b5

[JorisGoosen]

Here in readstat has a metadata for dataset files type and "missing", and also in readstat.h wich will including some custom missing value and system missing.

I know, im using them ^^
https://github.com/jasp-stats/jasp-desktop/blob/ab341fc7a580bc20976185df67054873c7d15803/Desktop/data/importers/readstat/readstatimportcolumn.cpp#L69-L85