Skip to content

Structures

Jump to bottom Edit New page
Tavis Ormandy edited this page Jul 10, 2020 · 16 revisions

Bash doesn't provide any way to interact with native data structures, so ctypes.sh will translate native structures into bash format and back again. This process is called packing and unpacking.

In many cases, ctypes.sh can automatically import structures from the libraries you want to use. This makes working with structures in bash simple and easy.

Automatic Structure Definition

To define a structure in bash, use the struct command.

$ struct stat fileinfo

The parameters are the name of a structure, followed by the name of the variable to contain the structure. If you're using structs in a script, you might want to verify that the struct was created successfully.

if ! struct stat fileinfo; then
    echo "failed to create stat structure, see the ctypes.sh wiki for troubleshooting"
    exit 1
fi

The structure should be defined in a library you have loaded via dlopen.

The example above creates the standard stat structure in the variable fileinfo. To access members of the structure, specify the member name in brackets, like ${fileinfo[st_size]}.

You can also access the members of nested structures, just use . between names. For example, the stat structure stores the access time in a nested structure, access it like this ${fileinfo[st_atim.tv_sec]}.

If automatic structure definition doesn't work, don't worry, there are troubleshooting steps below.

Let's look at a complete example.

$ struct -m statbuf stat passwd
$ dlcall __xstat 0 "/etc/passwd" $statbuf
$ unpack $statbuf passwd
$ echo ${passwd[st_mtim.tv_sec]}
long:1446019680
$ date --date=@1446019680
Wed Oct 28 01:08:00 PDT 2015
  1. In this example we use the -m parameter to allocate memory, and the pointer is saved as $statbuf. This is shorthand for struct stat passwd; dlcall -r pointer -n statbuf malloc $(sizeof stat)
  2. Call stat() on /etc/passwd (note that on Linux, __xstat is used instead of stat)
  3. Use unpack to translate the result into a form we can work with in bash.

From here we can access members and verify that we got the data we expected, such as the last modification time.

Some error checking was omitted in the example above for clarity, you can see a more complete version here.

If you need to know the size of a structure for use with allocations or other memory management routines, use the sizeof command. sizeof understands the names of some primitive types like int and long, and supports the same parameters as the struct command.

Unions

Most structures can be imported automatically, but structures containing unions can be ambiguous. That is, they could be imported in many different ways. By default, the first member of a union is always used.

If that is not what you want, you need to override the default parsing with the -u option.

Let's look at an example, consider a structure like this:

struct whatever {
 union {
  int a;
  float b;
 } blah;
};

If you want ${whatever[blah.b]} rather than ${whatever[blah.a]}, you need to override the parsing like this:

$ struct -u blah:b whatever whatever

Separate more unions with commas, for example struct -u blah:b,foo:x,bar.baz:y whatever foo

Anonymous Unions

It is common for programmers to use anonymous unions in C. This is only legal when the result is unambiguous, so you can just omit the union name. For example, the following structure members could be addressed like ${whatever[.a]} and ${whatever[.b]}.

struct {
 union { int a };
 union { long b };
}

Anonymous Structures

Some programmers use anonymous structures defined like this:

typedef struct {
  int foo;
} type_t;

Because the structure is anonymous, it can't be looked up by name. To use a structure like this ctypes.sh must process typedefs, so specify the -a parameter to struct and then use the type name instead.

Troubleshooting Automatic Structures

Automatic structure definition works by parsing the DWARF debugging data in libraries. ctypes.sh includes data for many standard UNIX types already, but if you are using another library you may need to install the debugging data.

  • On Fedora, RedHat or CentOS, install yum-utils then you can use debuginfo-install <library> or dnf debuginfo-install <library>
  • On Debian or Ubuntu, you may need to enable ddebs, this is documented here.
  • On FreeBSD, enable WITH_DEBUG_FILES in src.conf and recompile
  • If this is your own library, don't use strip or gcc -s

Many standard structures are part of glibc, try debuginfo-install glibc or apt install libc6-dbg.

If none of these options are possible, you can either define the structure manually, or build a small shared object.

Creating a shared object

If you do not have debugging data for your library, but you do have the include files for it, you can build a small shared object and ctypes.sh can import the structures from it.

Create a .c file that just creates the structure you are using, the entire file is shown below.

$ cat example.c
#include <yourlibrary.h>

struct yourstruct test;

Compile that c file with debugging data.

$ cc -g -shared -o example.so example.c

Now load that file into bash so that ctypes.sh knows to search it.

$ dlopen ./example.so
0x234180

Your structure should now be available with the struct command.

Caveats

Some features like bitfields and complex multi-dimensional arrays can be difficult to parse, if the result is incorrect or ctypes.sh cannot parse your structure correctly, please file a bug.

Technical Notes

The struct command creates an unusual bash data structure.

You're probably familiar with bash's indexed arrays (declare -a) and associative arrays (declare -A). Internally, bash stores associative arrays as hash tables, which means the order of elements is discarded. You can test this yourself:

$ declare -A hello
$ hello[foo]=1 hello[bar]=2 hello[baz]=3 hello[quz]=4 
$ echo ${hello[@]}
2 4 3 1

However, bash's internal API allows you to create associative arrays with an arbitrary bucket size. ctypes.sh uses this feature to create associative arrays that maintain their order, by setting the bucket size to 1.

This means the associative arrays created by struct are technically hash tables, but work like linked lists. Therefore, they will maintain the order of elements.

Because of this, arrays created by the struct command may act differently than the arrays created with declare -A. ctypes.sh does this so that the more natural associative array syntax can be used with structures.

Manual Structure Definition

If you cannot use automatic structure definition, you can define your structure manually in an indexed array.

    $ data=(int int long pointer)
    $ unpack pointer:0x9e2d90 data
    $ echo ${data[@]}
    int:1460145432 int:32586 long:10350560 pointer:(nil)

Packing and Unpacking

ctypes.sh provides two commands for interacting with structs, pack and unpack.

  • pack converts from an array of prefixed types to native format.
  • unpack converts a pointer to an array of prefixed types.

In its simplest form, you specify the types and a pointer to fetch them from

    $ data=(int int long pointer)
    $ unpack pointer:0x9e2d90 data
    $ echo ${data[@]}
    int:1460145432 int:32586 long:10350560 pointer:(nil)
Add a custom footer
Add a custom sidebar
Clone this wiki locally