How to write data of a variable in a sub group with netcdfAll 5.5.2

[thomas-romuald]

I want to create a netcdf file with variable in several groups

In my code after NetcdfFormatWriter writer = builder.build() the inspect with eclipse of field ncout in writer object return:

netcdf /home/toto/git/sedoo-mtropics-rest/target/test-classes/netcdf/7bc45591-5b9f-a13d-90dc-f2a75b0a15cc.nc {
  group: R2_group {
    dimensions:
      time = 6939;
    variables:
      double time(time=6939);
        :standard_name = "time";
        :long_name = "Time of measurement";
        :calendar = "julian";
        :units = "seconds since 1970-01-01, 00:00:00";

      float R2(time=6939);
        :standard_name = "rainfall_amount";
        :long_name = "Rainfall";
        :missing_value = -9999.0f; // float
        :_FillValue = -9999.0f; // float
        :units = "mm";

  }

  group: R1_group {
    dimensions:
      time = 6939;
    variables:
      double time(time=6939);
        :standard_name = "time";
        :long_name = "Time of measurement";
        :calendar = "julian";
        :units = "seconds since 1970-01-01, 00:00:00";

      float R1(time=6939);
        :standard_name = "rainfall_amount";
        :long_name = "Rainfall";
        :missing_value = -9999.0f; // float
        :_FillValue = -9999.0f; // float
        :units = "mm";

  }

}

I have now the write data to the nested group variables

Using the following method source code here throw an exception.

public void write(String varName, Array values) throws IOException, InvalidRangeException

The variable cannot be found

java.lang.NullPointerException: null
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:770)
	at ucar.nc2.write.NetcdfFormatWriter.write(NetcdfFormatWriter.java:483)

Does anyone could help me to find out the right way to set data in a sub group with netcdfall version >= 5.5.2 ?

[haileyajohnson]

What variable are you trying to write that is throwing the exception? Can you share a code snippet?

[thomas-romuald]

For the structure has shown above, I try to write data like that.
Variables R1, R2 are Array (new ArrayFloat.D1(6939)) and variable time is an Array (new ArrayDouble.D1(6939))

	    	try (NetcdfFormatWriter writer = builder.build()) {
	    			writer.write("R1", values.get("R1"));
	    			writer.write("R2", values.get("R2"));
	    			writer.write("time", values.get("time"));
	         		writer.flush();
	         		writer.close();
	    	} catch (Exception e) {
	    			log.error("Cannot create NetCDF file", e);
	   	}

The first write throw an exception because the variable is not found by the lib. I guess it looks for variable in the RootGroup but i don't know how to specify that this variable is in the subgroup R1_group.
I'm puzzled as well how to deal with the variable time as two subgroup has it.

A colleague gave me a netcdf file with a similar structure so it must be possible but I'm not able to do it in Java so far.

I'll try to do a code snippet.

[thomas-romuald]

Here a simplified version of the code:

import java.io.File;
import java.util.Random;

import ucar.ma2.Array;
import ucar.ma2.ArrayDouble;
import ucar.ma2.ArrayFloat;
import ucar.ma2.DataType;
import ucar.nc2.Attribute;
import ucar.nc2.Dimension;
import ucar.nc2.Group;
import ucar.nc2.Variable;
import ucar.nc2.write.NetcdfFileFormat;
import ucar.nc2.write.NetcdfFormatWriter;

public class NetCDFConverterTest {
	
	private static final int LENGTH = 10;
	public static final float MISSING_VALUE_FLOAT = -9999f;

	public static void main(String[] args) {
		
		File netCDFFile = new File("/data/test.nc");
		
		NetcdfFormatWriter.Builder builder = NetcdfFormatWriter.createNewNetcdf4(NetcdfFileFormat.NETCDF4, netCDFFile.getPath(), null);
		
		Group.Builder rootGroup = builder.getRootGroup();
		Group.Builder subGroup = Group.builder().setName("R1_group");
		subGroup.setParentGroup(rootGroup);
			
		Dimension timeDimension = Dimension.builder("time", LENGTH).build();
		
		subGroup.addDimension(timeDimension);
		Variable.Builder timeVariableBuilder = Variable.builder().setName("time");
		timeVariableBuilder.setDataType(DataType.DOUBLE);
		timeVariableBuilder.addDimension(timeDimension);
		timeVariableBuilder.addAttribute(new Attribute("standard_name", "time"));
		timeVariableBuilder.addAttribute(new Attribute("long_name","Time of measurement"));
		timeVariableBuilder.addAttribute(new Attribute("calendar","julian"));
		timeVariableBuilder.addAttribute(new Attribute("units", "seconds since 1970-01-01, 00:00:00"));
		subGroup.addVariable(timeVariableBuilder);
 			
		Variable.Builder variable = Variable.builder().setName("R1");
		variable.setDataType(DataType.FLOAT);
		variable.addDimension(timeDimension);
		variable.addAttribute(new Attribute("standard_name", "rainfall_amount"));
		variable.addAttribute(new Attribute("long_name", "Rainfall"));
		variable.addAttribute(new Attribute("missing_value", MISSING_VALUE_FLOAT));
		variable.addAttribute(new Attribute("_FillValue", MISSING_VALUE_FLOAT));
		variable.addAttribute(new Attribute("units", "mm"));
		subGroup.addVariable(variable);
		
		rootGroup.addGroup(subGroup);
		
		// Create random data
		Array time = new ArrayDouble.D1(LENGTH);
		Array r1 = new ArrayFloat.D1(LENGTH);
		Random rand = new Random();
		for(int index=0 ; index<LENGTH ; index++) {
			r1.setFloat(index, rand.nextFloat());
			time.setDouble(index, rand.nextDouble());
		}
			
		// Create netcdf file
		try (NetcdfFormatWriter writer = builder.build()) {
			writer.write("R1", r1);
			writer.write("time", time);
 			writer.flush();
 			writer.close();
		} catch (Exception e) {
			System.out.println("Cannot create NetCDF file");
		}
	}
}

It creates this structure:

netcdf /data/test.nc {
  group: R1_group {
    dimensions:
      time = 10;
    variables:
      double time(time=10);
        :standard_name = "time";
        :long_name = "Time of measurement";
        :calendar = "julian";
        :units = "seconds since 1970-01-01, 00:00:00";

      float R1(time=10);
        :standard_name = "rainfall_amount";
        :long_name = "Rainfall";
        :missing_value = -9999.0f; // float
        :_FillValue = -9999.0f; // float
        :units = "mm";

  }

}

But fails when writing data.

[haileyajohnson]

Cool, I think all you need to change is to give the "full" name of the variable you're writing to, which is written "{subgroup}/{variable}", e.g. "R1_group/time" and "R1_group/R1"

[thomas-romuald]

Many thanks @haileyajohnson it worked!

I just had time for a quick test but it looked fine. I'll implement the proper solution next week.

Thank you again.