Issues with nested structs

[mantielero]

I've been playing trying to wrap something else and I have faced some issues. Not sure if this is my bad or an issue.

vapoursynth

I was trying to wrap vapoursynth. I did the following:

import os, strutils
import nimterop/[cimport, git, paths]


const
  baseDir = currentSourcePath.parentDir()/"build"
  srcDir = baseDir/"project"

static:
  cDebug()
  cDisableCaching()
  gitPull("https://github.com/vapoursynth/vapoursynth", outdir = srcDir, plist = """
include/VapourSynth.h
""",
    checkout = "tags/R46")

cImport(srcDir/"include/VapourSynth.h", recurse = true)

I was expecting to get the corresponding type for struct VSAPI.

libxlsxwriter

The same with libxlsxwriter. I used:

import os, strutils

import nimterop/[cimport, git, paths]


const
  # Location where any sources should get downloaded. Adjust depending on
  # actual location of wrapper file relative to project.
  baseDir = currentSourcePath.parentDir()/"build"

  # All files and dirs should be inside to baseDir
  srcDir = baseDir/"project"

static:
  # Print generated Nim to output
  cDebug()

  # Disable caching so that wrapper is generated every time. Useful during
  # development. Remove once wrapper is working as expected.
  cDisableCaching()

  # Download C/C++ source code from a git repository
  #gitPull("https://github.com/sekrit-twc/zimg", outdir =)
  gitPull("https://github.com/jmcnamara/libxlsxwriter", outdir = srcDir, plist = """
include/*
include/xlsxwriter/*
""",
    checkout = "tags/RELEASE_0.8.7")


when defined(windows):
  # Windows specific symbols, options and files

  # Dynamic library to link against
  const dynlibFile =
    when defined(cpu64):
      "libxlsxwriter.dll"    # TBC
    else:
      "libxlsxwriter.dll"    # TBC
elif defined(posix):
  # Common posix symbols, options and files

  when defined(linux):
    # Linux specific
    const dynlibFile = "libxlsxwriter.so(.2|.1|)"      #TBC
  elif defined(osx):
    # MacOSX specific
    const dynlibFile = "libxlsxwriter(.2|.1|).dylib"  # TBC
  else:
    static: doAssert false
else:
  static: doAssert false


#cImport(srcDir/"include/xlsxwriter.h", recurse = true)
#cIncludeDir(srcDir/"include/xlsxwriter")
#cImport(srcDir/"include/xlsxwriter/format.h", recurse = true)
cImport(srcDir/"include/xlsxwriter/format.h", recurse = true)

I was expecting typedef struct lxw_format to be converted into nim.

Any suggestion?

[genotrance]

For Vapoursynth, line 71 causes a dummy object to be created after which the second instance on line 242 gets skipped. C doesn't care but Nim doesn't have a simple way for forward declarations.

For libxlsxwriter, line 421 has a nested struct due to this definition. As you know, nimterop doesn't support nested structs yet.

[genotrance]

By the way, there's already a wrapper for libxslxwriter.

https://github.com/KeepCoolWithCoolidge/nimlibxlsxwriter

[mantielero]

I know, but it doesn't work (at least for me). It looks unmaintained (the error I am facing was reported last year).

[mantielero]

Just for the record, the way in which lxw_format gets implemented using nimgen is:

type
  INNER_C_STRUCT_temp-format.nim_168* {.bycopy.} = object
    stqe_next*: ptr lxw_format

  lxw_format* {.bycopy.} = object
    file*: ptr FILE
    xf_format_indices*: ptr lxw_hash_table
    num_xf_formats*: ptr uint16
    xf_index*: int32
    dxf_index*: int32
    num_format*: array[128, char]
    font_name*: array[128, char]
    font_scheme*: array[128, char]
    num_format_index*: uint16
    font_index*: uint16
    has_font*: uint8
    has_dxf_font*: uint8
    font_size*: cdouble
    bold*: uint8
    italic*: uint8
    font_color*: lxw_color_t
    underline*: uint8
    font_strikeout*: uint8
    font_outline*: uint8
    font_shadow*: uint8
    font_script*: uint8
    font_family*: uint8
    font_charset*: uint8
    font_condense*: uint8
    font_extend*: uint8
    theme*: uint8
    hyperlink*: uint8
    hidden*: uint8
    locked*: uint8
    text_h_align*: uint8
    text_wrap*: uint8
    text_v_align*: uint8
    text_justlast*: uint8
    rotation*: int16
    fg_color*: lxw_color_t
    bg_color*: lxw_color_t
    pattern*: uint8
    has_fill*: uint8
    has_dxf_fill*: uint8
    fill_index*: int32
    fill_count*: int32
    border_index*: int32
    has_border*: uint8
    has_dxf_border*: uint8
    border_count*: int32
    bottom*: uint8
    diag_border*: uint8
    diag_type*: uint8
    left*: uint8
    right*: uint8
    top*: uint8
    bottom_color*: lxw_color_t
    diag_color*: lxw_color_t
    left_color*: lxw_color_t
    right_color*: lxw_color_t
    top_color*: lxw_color_t
    indent*: uint8
    shrink*: uint8
    merge_range*: uint8
    reading_order*: uint8
    just_distrib*: uint8
    color_indexed*: uint8
    font_only*: uint8
    list_pointers*: INNER_C_STRUCT_temp-format.nim_168

where the original data contains:

#define	STAILQ_ENTRY(type)						\
struct {								\
	struct type *stqe_next;	/* next element */			\
}

and

typedef struct lxw_format {

    FILE *file;

    lxw_hash_table *xf_format_indices;
    uint16_t *num_xf_formats;

    int32_t xf_index;
    int32_t dxf_index;

    char num_format[LXW_FORMAT_FIELD_LEN];
    char font_name[LXW_FORMAT_FIELD_LEN];
    char font_scheme[LXW_FORMAT_FIELD_LEN];
    uint16_t num_format_index;
    uint16_t font_index;
    uint8_t has_font;
    uint8_t has_dxf_font;
    double font_size;
    uint8_t bold;
    uint8_t italic;
    lxw_color_t font_color;
    uint8_t underline;
    uint8_t font_strikeout;
    uint8_t font_outline;
    uint8_t font_shadow;
    uint8_t font_script;
    uint8_t font_family;
    uint8_t font_charset;
    uint8_t font_condense;
    uint8_t font_extend;
    uint8_t theme;
    uint8_t hyperlink;

    uint8_t hidden;
    uint8_t locked;

    uint8_t text_h_align;
    uint8_t text_wrap;
    uint8_t text_v_align;
    uint8_t text_justlast;
    int16_t rotation;

    lxw_color_t fg_color;
    lxw_color_t bg_color;
    uint8_t pattern;
    uint8_t has_fill;
    uint8_t has_dxf_fill;
    int32_t fill_index;
    int32_t fill_count;

    int32_t border_index;
    uint8_t has_border;
    uint8_t has_dxf_border;
    int32_t border_count;

    uint8_t bottom;
    uint8_t diag_border;
    uint8_t diag_type;
    uint8_t left;
    uint8_t right;
    uint8_t top;
    lxw_color_t bottom_color;
    lxw_color_t diag_color;
    lxw_color_t left_color;
    lxw_color_t right_color;
    lxw_color_t top_color;

    uint8_t indent;
    uint8_t shrink;
    uint8_t merge_range;
    uint8_t reading_order;
    uint8_t just_distrib;
    uint8_t color_indexed;
    uint8_t font_only;

    STAILQ_ENTRY (lxw_format) list_pointers;
} lxw_format;

[observant2]

Is there a workaround one could use? 🤔

[genotrance]

First quick option is to simply use c2nImport and see if c2nim can wrap it seamlessly.

Next is to extract the problematic structure, wrap it manually or with c2nim, and then use cOverride to define it. cOverride only works for cImport though.

If there are many such symbols, you can use the toast command line with the -d flag. It will dump all code that it couldn't wrap to a separate header file which can be converted with c2nim. Then you can just cOverride them specifically.

[genotrance]

libbson example:

https://github.com/mongodb/libbson/blob/master/src/bson/bson-types.h#L275

[genotrance]

The forward declarations issue is fixed in ast2. Here is what the vapoursynth wrapper looks like.

Nested structs are also now supported in ast2 and here is what the libxlsxwriter wrapper looks like.