Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FRACTIONAL_SNOW_COVER: how to escape from signed bytes? #601

Closed
soxofaan opened this issue Dec 6, 2023 · 2 comments · Fixed by Open-EO/openeo-geotrellis-extensions#245
Closed
Assignees

Comments

@soxofaan
Copy link
Member

soxofaan commented Dec 6, 2023

original issue: https://discuss.eodc.eu/t/passing-user-defined-parameters-to-an-udf/658

user wants to use FRACTIONAL_SNOW_COVER and remap the snow cover values:

  • below 20% -> round down to 0
  • between 20% and 100% -> round up to 100
  • value 205 (special value for clouds) -> keep 205

currently she manages to do this with a UDF, but instead this should be possible with simple band math.
However, you get stuck in signed byte space, which wraps around the 205 value to -51

Minimal reproduction example:

import openeo

con = openeo.connect(
    "openeo-dev.vito.be",
)
con.authenticate_oidc()

scf = con.load_collection(
    "FRACTIONAL_SNOW_COVER",
    temporal_extent=["2020-04-01", "2020-04-07"],
    spatial_extent={"west": 11.2, "east": 11.3, "south": 47.15, "north": 47.20},
    bands=["FSCTOC"],
)

# Note that the raw data is ok: value 205 is properly available
# scf.download("raw.nc")

# Band math
fsctoc = scf.band("FSCTOC")
fsctoc = 1.0 * fsctoc

scf_test = 100.0 * (fsctoc >= 20) * (fsctoc <= 100) + 205.0 * (fsctoc == 205)

scf_test.download("thresholded.nc")

resulting netcdf file has values 0, 100 and -51 instead of desired 0, 100 and 205. For example (-51=blue in plot):

image

Note the explicit float multiplications to attempt triggering float conversions, without effect however.

@jdries
Copy link
Contributor

jdries commented Dec 6, 2023

The multiply by float can probably be made to work in org.openeo.geotrellis.OpenEOProcessScriptBuilder#xyFunction

var x_input: Seq[Tile] = evaluateToTiles(x_function, context, tiles).map(convertBitCellsOp)
var y_input: Seq[Tile] = evaluateToTiles(y_function, context, tiles).map(convertBitCellsOp)
val combinedCellType = x_input.headOption.map(_.cellType.union(y_input.headOption.map(_.cellType).getOrElse(BitCellType)))
x_input = x_input.map(_.convert(combinedCellType.getOrElse(BitCellType)))
y_input = y_input.map(_.convert(combinedCellType.getOrElse(BitCellType)))

jdries added a commit to Open-EO/openeo-geotrellis-extensions that referenced this issue Dec 8, 2023
…igned and unsigned, also force celltype conversion in every xyfunction

Open-EO/openeo-geopyspark-driver#601
@jdries jdries self-assigned this Dec 8, 2023
@jdries
Copy link
Contributor

jdries commented Feb 1, 2024

Fixed on openeo-dev, I get the expected values.
I can also remove the explicit tricks to try and switch to floating point, in this case I now get int32 as datatype, also with correct values.

@jdries jdries closed this as completed Feb 1, 2024
jdries added a commit that referenced this issue Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants