wrong mapping of union inside structure

[folkfreund]

Just migrated a project from JNA 3.2.7 to 3.5.2
One of the structures contains a union and other fields after that union.

With the new version the fields after the union are mapped to wrong addresses - as if all fields of the union were present. The memory dump by toString shows "deafbeef".

In other words:
The Union's function size() reports the correct value (8), but the next field of the structure is placed 0x20 bytes after the start of the union (0x20 is the sum of all fields inside the union)

[twall]

Seeing the native declaration and JNA mapping would help, and a test case would be superb.

On Apr 22, 2013, at 10:53 AM, folkfreund wrote:

Just migrated a project from JNA 3.2.7 to 3.5.2
One of the structures contains a union and other fields after that union.

With the new version the fields after the union are mapped to wrong addresses - as if all fields of the union were present. The memory dump by toString shows "deafbeef".

In other words:
The Union's function size() reports the correct value (8), but the next field of the structure is placed 0x20 bytes after the start of the union (0x20 is the sum of all fields inside the union)

—
Reply to this email directly or view it on GitHub.

[folkfreund]

The original code is far too complex for posting here. I'll try to create a test case.
By now I know, this effect does not occur in structures created on the Java side. In my case the structure is created by the DLL and passed to a JNA Callback implementing this interface:

public static interface IReqDoneListener extends Callback
  {
    void callback(REQ_PEND req);
  }

I will post my test case or any other results soon.

[folkfreund]

I couldn't find a possibility to upload files...
Here is my test code:

JNA_union_test.java

package jna_union_test;

import com.sun.jna.Callback;
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Structure;
import com.sun.jna.Union;
import java.util.Arrays;
import java.util.List;

/**
 * Test to reproduce issue #223: "wrong mapping of union inside structure"
 */

public class JNA_union_test
{
  public static class TheUnion extends Union
  {
    public int      u_integer;
    public float    u_float;
    public double   u_double;
  }

  public static class TheStruct extends Structure
  {
    public long     s_long;
    public TheUnion s_union;
    public int      s_valueAfterUnion;

    @Override
    protected List getFieldOrder()
    {
      return Arrays.asList(new String[] { 
        "s_long", 
        "s_union", 
        "s_valueAfterUnion"
      });
    }

    @Override
    public String toString()
    {
      String sizeInfo = "TheUnion Size="+s_union.size()+"\n----------------\n";
      return sizeInfo + super.toString();
    }
  }

  public interface TestDll extends Library
  {
    public static interface ITestCallback extends Callback
    {
      void callback(TheStruct result);
    }

    void invokeCallback(ITestCallback callback);
  }

  public static class DllListener implements TestDll.ITestCallback
  {

    @Override
    public void callback(TheStruct result)
    {
      System.out.println();
      System.out.println("Structure built by DLL");
      System.out.println("-----------------------");

      System.out.println(result.toString());
    }

  }


  public static void main(String[] args)
  {
    Native.setProtected(true);
    final TestDll testDll = (TestDll)Native.loadLibrary("JNA_test_dll", TestDll.class);

    if (testDll == null)
    {
      System.out.println("cannot load JNA_test_dll");
      return;
    }

    // first print the structure built on the Java side
    System.out.println("Structure built by Java");
    System.out.println("-----------------------");
    TheStruct theStruct = new TheStruct();

    theStruct.s_long = 0x12345l;
    theStruct.s_union = new TheUnion();
    theStruct.s_union.setType("u_integer");
    theStruct.s_union.u_integer = 0x4321;
    theStruct.s_union.write();
    theStruct.s_valueAfterUnion = 0x5555;
    theStruct.write();

    System.out.println(theStruct.toString());

    // nest print the structure as built on the native side,
    // (the DLL will create a struct with the same values as above)
    testDll.invokeCallback(new DllListener());
  }

}

And the code of the JNA_test_dll.dll

typedef union
{
    int       u_integer;
    float     u_float;
    double    u_double;
} TheUnion;

typedef struct
{
    long      s_long;
    TheUnion  s_union;
    int       s_valueAfterUnion;
} TheStruct;

TheStruct myResult;

void invokeCallback (void (*callback) (TheStruct *result) )
{
    if (callback)
    {
        myResult.s_long = 0x12345l;
        myResult.s_union.u_integer = 0x4321;
        myResult.s_valueAfterUnion = 0x5555;

        (* callback)(&myResult);
    }
}

The output says (commented):

Structure built by Java
-----------------------
TheUnion Size=8
----------------
JNA_union_test$TheStruct(auto-allocated@0x384760 (32 bytes)) {
  long s_long@0=12345
  JNA_union_test$TheUnion s_union@8=JNA_union_test$TheUnion(allocated@0x384768 (8 bytes) (shared from auto-allocated@0x384760 (32 bytes))) {
    int u_integer@0=4321
    float u_float@0=0.0
    double u_double@0=0.0
  }
  int s_valueAfterUnion@18=5555 // 16 bytes (= sum of all union members) after s_union
}
memory dump
[45230100] 0x00 : s_long
[00000000]
[21430000] 0x08 : s_union
[00000000]
[00000000]
[00000000]
[55550000] 0x18 : s_valueAfterUnion
[00000000]

Structure built by DLL
-----------------------
TheUnion Size=8
----------------
JNA_union_test$TheStruct(native@0x3180ea0) (32 bytes) {
  long s_long@0=12345
  JNA_union_test$TheUnion s_union@8=JNA_union_test$TheUnion(native@0x3180ea8) (8 bytes) {
    int u_integer@0=4321
    float u_float@0=2.4081E-41
    double u_double@0=8.4905E-320
  }
  int s_valueAfterUnion@18=0 // adresses wrong memory!
}
memory dump
[45230100] 0x00 : s_long
[00000000]
[21430000] 0x08 : s_union
[00000000]
[55550000] 0x10 : here is the contents of s_valueAfterUnion
[00000000]
[00000000] 0x18 : here Java accesses s_valueAfterUnion
[01000000]

[twall]

Usually what you do with github is fork the repo and apply changes to that repo, then post a pull request against the original.

"gist" also exists for uploading code snippets independent of a particular repo.

On Apr 23, 2013, at 5:54 AM, folkfreund wrote:

I couldn't find a possibility to upload files...
Here is my test code:

JNA_union_test.java

package jna_union_test;

import com.sun.jna.Callback;
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Structure;
import com.sun.jna.Union;
import java.util.Arrays;
import java.util.List;

/**

• Test to reproduce issue wrong mapping of union inside structure #223: "wrong mapping of union inside structure"
  */

public class JNA_union_test
{

public static class TheUnion extends Union

{

public int u_integer;

public float u_float;

public double u_double;

}

public static class TheStruct extends Structure

{

public long s_long;

public TheUnion s_union;

public int s_valueAfterUnion;

@OverRide

protected List getFieldOrder()

{

return Arrays.asList(new String[] {

"s_long",

"s_union",

"s_valueAfterUnion"

});

}

@OverRide

public String toString()

{

String sizeInfo = "TheUnion Size="+s_union.size()+"\n----------------\n";

return sizeInfo + super.toString();

}

}

public interface TestDll extends Library

{

public static interface ITestCallback extends Callback

{

void callback(TheStruct result);

}

void invokeCallback(ITestCallback callback);

}

public static class DllListener implements TestDll.ITestCallback

{

@OverRide

public void callback(TheStruct result)

{

System.out.println();

System.out.println("Structure built by DLL");

System.out.println("-----------------------");

System.out.println(result.toString());

}

}

public static void main(String[] args)

{

Native.setProtected(true);

final TestDll testDll = (TestDll)Native.loadLibrary("JNA_test_dll", TestDll.class);

if (testDll == null)

{

System.out.println("cannot load JNA_test_dll");

return;

}

// first print the structure built on the Java side

System.out.println("Structure built by Java");

System.out.println("-----------------------");

TheStruct theStruct = new TheStruct();

theStruct.s_long = 0x12345l;

theStruct.s_union = new TheUnion();

theStruct.s_union.setType("u_integer");

theStruct.s_union.u_integer = 0x4321;

theStruct.s_union.write();

theStruct.s_valueAfterUnion = 0x5555;

theStruct.write();

System.out.println(theStruct.toString());

// nest print the structure as built on the native side,

// (the DLL will create a struct with the same values as above)

testDll.invokeCallback(new DllListener());

}

}
And the code of the JNA_test_dll.dll

typedef union
{

int u_integer;

float u_float;

double u_double;
} TheUnion;

typedef struct
{

long s_long;

TheUnion s_union;

int s_valueAfterUnion;
} TheStruct;

TheStruct myResult;

void invokeCallback (void (*callback) (TheStruct *result) )
{

if (callback)

{

myResult.s_long = 0x12345l;

myResult.s_union.u_integer = 0x4321;

myResult.s_valueAfterUnion = 0x5555;

(* callback)(&myResult);

}
}
The output says (commented):

Structure built by Java

TheUnion Size=8

JNA_union_test$TheStruct(auto-allocated@0x384760 (32 bytes)) {
long s_long@0=12345
JNA_union_test$TheUnion s_union@8=JNA_union_test$TheUnion(allocated@0x384768 (8 bytes) (shared from auto-allocated@0x384760 (32 bytes))) {
int u_integer@0=4321
float u_float@0=0.0
double u_double@0=0.0
}
int s_valueAfterUnion@18=5555 // 16 bytes (= sum of all union members) after s_union
}
memory dump
[45230100] 0x00 : s_long
[00000000]
[21430000] 0x08 : s_union
[00000000]
[00000000]
[00000000]
[55550000] 0x18 : s_valueAfterUnion
[00000000]

Structure built by DLL

TheUnion Size=8

JNA_union_test$TheStruct(native@0x3180ea0) (32 bytes) {
long s_long@0=12345
JNA_union_test$TheUnion s_union@8=JNA_union_test$TheUnion(native@0x3180ea8) (8 bytes) {
int u_integer@0=4321
float u_float@0=2.4081E-41
double u_double@0=8.4905E-320
}
int s_valueAfterUnion@18=0 // adresses wrong memory!
}
memory dump
[45230100] 0x00 : s_long
[00000000]
[21430000] 0x08 : s_union
[00000000]
[55550000] 0x10 : here is the contents of s_valueAfterUnion
[00000000]
[00000000] 0x18 : here Java accesses s_valueAfterUnion
[01000000]

—
Reply to this email directly or view it on GitHub.

[folkfreund]

Thanks for the info - I'm new to github, but I will have to learn using it ;-)
I hope my previous post is sufficient for now.

Greetings, folkfreund

[twall]

When the callback receives the union, it has no way of knowing the correct type and thus can't perform an automatic read to synch Java fields with native memory. You need to either set a "safe" default type in your union ctor, or do the setType and read within your callback.

Try calling setType and read within your callback and see if that fixes things.

[twall]

The post-union field is definitely using an incorrect offset. The total structure size should be 20 bytes, not 32, and the field offset should be 16, not 24.