# Implementing structs by value in Dart FFI

> A deep dive into API design and native calling conventions.




### A deep dive into API design and native calling conventions

In the [Dart 2.12](/blog/announcing-dart-2-12) release, we extended our [C-interop feature, Dart FFI,](https://dart.dev/guides/libraries/c-interop) with the ability to [pass structs by value](https://github.com/dart-lang/sdk/issues/36730). This article talks about what it took to add this feature to the Dart SDK. If you’re interested in low-level language implementation details or in platform conventions for passing structs by value, keep reading.

This article talks about both developing the API and figuring out the ABI (Application Binary Interface) for the struct-by-value feature. During the two years we worked on this feature (and other Dart FFI features), we discovered many constraints that required changing the API. The ABI journey was equally interesting, illustrating that you can take multiple approaches to nailing down the details of a hard problem.

## Pass by value and pass by reference in C/C++

Here’s a quick refresher if you don’t write code in C every day. Suppose that we have the following struct and functions in C:

```c++
struct Coord {
  double x;
  double y;
  Coord* next;
};

Coord TranslateByValue(Coord coord) {
  coord.x = coord.x + 10.0;
  coord.y = coord.y + 10.0;
  return coord;
}

void TranslateByPointer(Coord* coord) {
  coord->x = coord->x + 10.0;
  coord->y = coord->y + 10.0;
}
```

Then, we can use these functions in some simple C code. Let’s say we have a local variable `c1`:

```
Coord c1 = {10.0, 10.0, nullptr};
```


If we pass `c1` to `TranslateByValue`, then the argument is passed by value, which makes the callee effectively operate on a copy of the struct:

```
Coord c2 = TranslateByValue(c1);
```


This means that `c1` stays unchanged.

However, if we pass `c1` by reference with a pointer to the memory containing `c1`, then `c1` gets mutated in place:

```
TranslateByPointer(&c1);
```


`c1.x` now contains `20.0`.

## The API design journey

The original Dart FFI [prototype](https://dart-review.googlesource.com/c/sdk/+/80124) already had support for passing pointers to structs. However, we redesigned the API multiple times to accommodate various use cases and constraints.

### Initial design

Our initial design enabled allocating structs in memory, passing those pointers to C, and modifying the fields of the structs. With that approach, the `Struct` class extended the `Pointer` class:

```dart
@struct
class Coordinate extends Pointer<Void> {
  @Double()
  double x;

  @Double()
  double y;

  @Pointer()
  Coordinate next;

  /// generated by @ffi.struct annotation
  external static int sizeOf();

  static Coordinate allocate({int count: 1}) =>
    allocate<Uint8>(count: count * sizeOf()).cast();
}

```

```dart
final c = Coordinate.allocate()
  ..x = 10.0
  ..y = 10.0;

```

Dart FFI users wrote the preceding snippet, and Dart FFI internals generated an implementation of `sizeOf` and getter and setter implementations for `x`, `y`, and `next`.

However, two years ago we realized that [this design had an issue](https://github.com/dart-lang/sdk/issues/35840). By having the `Coordinate` extend `Pointer`, we could not distinguish between `Coordinate` and `Coordinate*`.

### Distinguishing between Coordinate and Coordinate*

We [introduced](https://dart-review.googlesource.com/c/sdk/+/101291) `Struct` to Dart FFI and made structs extend this class:

```dart
abstract class Struct<S extends NativeType> extends NativeType {
  final Pointer<S> addressOf;
}

```

Now a `Pointer&lt;Coordinate&gt;` in Dart represents a `Coordinate*` in C, and a `Coordinate` in Dart represents a `Coordinate` in C.

This meant that the `next` field had the type `Pointer&lt;Coordinate&gt;`, which made the `@Pointer` annotation redundant. So, we [got rid of Pointer annotations](https://dart-review.googlesource.com/c/sdk/+/108415).

```dart
class Coordinate extends Struct<Coordinate> {
  @Double()
  double x;

  @Double()
  double y;

  Pointer<Coordinate> next;
}

```

Because we now represented pointers to structs as `Pointer` objects, we started using the `allocate` factory on `Pointer`:

```
final c = Pointer<Coordinate>.allocate();
```


To get access to the fields of a `Pointer&lt;Coordinate&gt;`, we need an object of type `Coordinate`, because that object has the fields `x`, `y`, and `next`. For this, we had the `load` method on `Pointer` already.

```
c.load<Coordinate>().x = 10.0;
```


Of course, having to write `&lt;Coordinate&gt;` on calling `load` is verbose. (Having to write a type argument was the same for loading a Dart `int` out of a `Pointer&lt;Uint8&gt;`.) The reason we need this type argument on `load` is to specify to the Dart type system the return type of this method.

### Extension methods to the rescue

Dart 2.7 [introduced](/blog/announcing-dart-2-7-a-safer-more-expressive-dart) extension methods. With extension methods, we could [pattern match](https://dart-review.googlesource.com/c/sdk/+/118992/14/sdk/lib/ffi/ffi.dart) on the type argument `T` in `Pointer&lt;T&gt;`:

```dart
extension StructPointer<T extends Struct> on Pointer<T> {
  external T get ref;
}

```

Pattern matching on the type argument enabled us to [get rid of the verbosity](https://dart-review.googlesource.com/c/sdk/+/118993/16/tests/ffi/structs_test.dart) on call sites:

```
c.ref.y = 10.0; // ref is pattern matched to be of type Coordinate.
```


We could also use the extension method pattern matching to make the type argument of `Struct&lt;S&gt;` redundant, changing the definition of user structs to:

```dart
class Coordinate extends Struct {
  @Double()
  double x;

  @Double()
  double y;

  Pointer<Coordinate> next;
}

```

Before, the type argument `&lt;S&gt;` constrained the `Struct` field `Pointer&lt;S&gt;` `addressOf`. Instead, we changed the field to an extension getter:

```dart
extension StructPointer<T extends Struct> on Pointer<T> {
  external Pointer<T> get addressOf;
}
```

### Stop leaking backing storage

When returning a struct by value from C to Dart, we don’t want to `malloc` C memory to save the struct, because that would be slow *and* burden the user with freeing it. So, instead, the struct is copied to a `TypedData`, and the `Coordinate` can have either a `Pointer` or a `TypedData` as backing storage.

However, `addressOf`, which was introduced in the first redesign, had type `Pointer`. This type conveyed that it was always backed by C memory, but this was no longer true.

So, we [deprecated](https://github.com/dart-lang/sdk/issues/40667) `addressOf`.

### For optimizations

The last step is to require invocations of various Dart FFI methods, including the ones related to structs, to have [compile-time constant type arguments](https://github.com/dart-lang/sdk/issues/44621):

```dart
extension StructPointer<T extends Struct> on Pointer<T> {
  /// Must invoke with a constant [T].
  external T get ref;
}

```

The invocation of methods allows us to better optimize the code and is more aligned with C semantics.

Note that this last change triggers deprecation notices in Dart 2.12, and the change is enforced in Dart 2.13.

## The ABI discovery journey

Now that the API is in place, the next question is: *Where does C expect these structs when passed or returned by value?* This is known as the Application Binary Interface (ABI).

### Documentation

The natural thing is to look for documentation. ARM provides [Procedure Call Standard for the Arm Architecture — ABI 2019Q1 ](https://developer.arm.com/documentation/ihi0042/h)and [Procedure Call Standard for the ARM 64-bit Architecture (AArch64)](https://developer.arm.com/documentation/ihi0055/c/). However, the x86 and x64 official documentation [fell off the internet](https://stackoverflow.com/questions/18133812/where-is-the-x86-64-system-v-abi-documented), resulting in people searching for this information and resorting to unofficial mirrors or [reverse engineering](https://www.agner.org/optimize/calling_conventions.pdf).

A quick glance at the documentation shows a variety of locations for passing structs by value:

* In multiple CPU and FPU registers.

* On the stack.

* A pointer to a copy. (The copy is on the caller’s stack frame.)

* Partially in CPU registers and partially on the stack.

When passed on the stack, there are some further questions about what the required alignment is and whether all unused CPU and FPU registers are blocked off or backfilled.

When returning a struct by value, the struct can be passed back in two locations:

* In multiple CPU and FPU registers.

* Written to a memory location by the callee, in which case the caller passes in a pointer to that memory location. (This reserved memory is also on the caller’s stack frame.)

When a pointer to the result location is passed in, a further question is whether this conflicts with a normal CPU argument register.

### Refactor Dart FFI compilation

This initial investigation was enough to realize that we had to reengineer a part of the Dart FFI compiler pipeline. We used to reuse the `Location` type, which was originally intended for compiling Dart code to assembly.

However, in the Dart ABI, we never use non-word-aligned stack locations or more than two registers at the same time. An experiment trying to extend the `Location` type to support these extra locations ended in a huge complicated diff because `Location` is used a lot in the Dart virtual machine.

So, instead, we [replaced the compilation pipeline](https://dart-review.googlesource.com/c/sdk/+/129081) for Dart FFI.

### Explore the native ABIs

Let’s explore the ABIs a bit.

Suppose that we have the following struct and C function signature:

```c
struct Struct3Bytes {
  uint8_t a0;
  uint8_t a1;
  uint8_t a2;
};

Struct3Bytes MyFunction(Struct3Bytes, Struct3Bytes, Struct3Bytes,
                        Struct3Bytes, Struct3Bytes, Struct3Bytes,
                        Struct3Bytes, Struct3Bytes);

```

How do various ABIs pass these structs in `MyFunction`?

In Linux on x64, there are 6 CPU argument registers. The struct is small enough to fit in a single register, so the first 6 arguments go into the 6 CPU argument registers, and the last 2 go on the stack. The stack arguments are aligned to 8 bytes. And, the return value also fits in a CPU register ([larger example](https://github.com/dart-lang/sdk/blob/9d5846b86eed777933b6c7a2a15e41f2b779c650/runtime/vm/compiler/ffi/unit_tests/struct3bytesx10/x64_linux.expect)).

```text
rdi int64 Compound(size: 3)
rsi int64 Compound(size: 3)
rdx int64 Compound(size: 3)
rcx int64 Compound(size: 3)
r8 int64 Compound(size: 3)
r9 int64 Compound(size: 3)
S+0 Compound(size: 3)
S+8 Compound(size: 3)
=>
rax int64 Compound(size: 3)

```

So, what happens on Windows?

It’s [completely different](https://github.com/dart-lang/sdk/blob/9d5846b86eed777933b6c7a2a15e41f2b779c650/runtime/vm/compiler/ffi/unit_tests/struct3bytesx10/x64_win.expect). Windows has only 4 argument registers. However, the first register is used to pass the pointer to the memory location to write the return value to. And, all arguments are passed by pointer to a copy, because the size of the struct is 3 bytes, which is not a power of 2.

```text
Locations on Windows
Pointer(rdx int64) Compound(size: 3)
Pointer(r8 int64) Compound(size: 3)
Pointer(r9 int64) Compound(size: 3)
Pointer(S+0 int64) Compound(size: 3)
Pointer(S+8 int64) Compound(size: 3)
Pointer(S+16 int64) Compound(size: 3)
Pointer(S+24 int64) Compound(size: 3)
Pointer(S+32 int64) Compound(size: 3)
=>
Pointer(rcx int64, ret:rax int64) Compound(size: 3)

```

Let’s look at another example: ARM32 on Linux and Android. Suppose that we have the following struct and C function signature:

```c
struct Struct16Bytes {
  float a0;
  float a1;
  float a2;
  float a3;
};

Struct16Bytes MyFunction2(Struct16Bytes, float, Struct16Bytes);
```

These specific types of structs are called homogeneous composites, because they only contain identical elements. And, homogenous floats with up to 4 members are treated differently from normal structs. In this case, [Linux uses floating point registers](https://github.com/dart-lang/sdk/blob/9d5846b86eed777933b6c7a2a15e41f2b779c650/runtime/vm/compiler/ffi/unit_tests/struct16bytesHomogenousx10/arm_linux.expect) for the individual floating points in the struct.

```text
Multiple(s0 float, s1 float, s2 float, s3 float) Compound(size: 16)
s4 float
Multiple(s5 float, s6 float, s7 float, s8 float) Compound(size: 16)
=>
Multiple(s0 float, s1 float, s2 float, s3 float) Compound(size: 16)

```

On Android, SoftFP is used instead of HardFP. This means that floats are passed in integer registers rather than floating point registers. Moreover, we’re passing in a `Pointer` for the result. This results in a [curious situation](https://github.com/dart-lang/sdk/blob/9d5846b86eed777933b6c7a2a15e41f2b779c650/runtime/vm/compiler/ffi/unit_tests/struct16bytesHomogenousx10/arm_android.expect) in which the first argument is partially passed in integer registers and partially passed on the stack.

```text
M(r1 int32, r2 int32, r3 int32, S+0 int32) Compound(size: 16)
S+4 float
M(S+8 int32, S+12 int32, S+16 int32, S+20 int32) Compound(size: 16)
=>
P(r0 uint32) Compound(size: 16)

```

Getting any of this wrong will likely lead to segmentation faults at runtime. So, it’s paramount to get all the corner cases of the ABI on every hardware and OS combination correct.

### Explore through godbolt.org

Because the documentation is very terse, we figured out many corner cases through the compiler explorer [godbolt.org](https://godbolt.org). The compiler explorer shows C code and compiled assembly [side by side](https://godbolt.org/z/17jzjv):

<DashImage src="images/0SfIpbC1E-Ysaewzs.webp" alt="Compiler Explorer showing C code and compiled assembly side by side." />


The preceding screenshot shows that on Windows x86 `sizeof(Struct3Bytes)` is 3 bytes, because 3 is moved into the return register `eax`.

When we [change](https://godbolt.org/z/r1rG6s) the struct slightly, we can inspect whether the size is still 3:

```c
typedef struct {
  int16_t a0;
  int8_t a1;
} Struct3Bytes;

```

The size is not 3: `mov eax, 4`. Because the `int16` must be 2-byte aligned, the struct must be 2-byte aligned. That means that when allocating an array of these structs there is a 1-byte padding after every struct to ensure that the next struct is 2-byte aligned. Hence, this struct is 4 bytes in the native ABI.

### Explore through generated tests

Unfortunately, the compiler explorer doesn’t support MacOS and iOS. So, to make exploring manually more efficient (and to have a nice and huge test suite for this feature), we wrote a test generator.

The main idea is to generate tests in such a way that if they crash it’s possible to use [GDB](https://www.gnu.org/software/gdb/) to see what’s wrong.

One way to make it easier to see what is going wrong when hitting a segmentation fault is to make all arguments have predictable and easy-to-recognize values. For example, the following test uses consecutive integers, so that these integer values can be easily spotted in registers and on the stack:

```dart
void testPassStruct3BytesHomogeneousUint8x10() {
  final a0Pointer = calloc<Struct3BytesHomogeneousUint8>();
  final Struct3BytesHomogeneousUint8 a0 = a0Pointer.ref;
  final a1Pointer = calloc<Struct3BytesHomogeneousUint8>();
  // ...
  a0.a0 = 1;
  a0.a1 = 2;
  a0.a2 = 3;
  a1.a0 = 4; 
  // ...

  final result = passStruct3BytesHomogeneousUint8x10(
      a0, a1, a2, a3, a4, a5, a6, a7, a8, a9);
  print("result = $result");
  Expect.equals(465, result);

  calloc.free(a0Pointer);
  calloc.free(a1Pointer);
  // ...
}

```

Another way to make finding problems easier is to add prints everywhere. For example, if we don’t hit a segmentation fault during the transition from Dart to C, but we manage to garble all the arguments, then printing the arguments helps:

```c
int64_t
PassStruct3BytesHomogeneousUint8x10(Struct3BytesHomogeneousUint8 a0,
                                    Struct3BytesHomogeneousUint8 a1,
                                    // ...
                                   ) {
  std::cout << "PassStruct3BytesHomogeneousUint8x10"
            << "((" << static_cast<int>(a0.a0) << ", "
            << static_cast<int>(a0.a1) << ", " << static_cast<int>(a0.a2)
            << "), (" << static_cast<int>(a1.a0) << ", ") <<  // ...

  int64_t result = 0;
  result += a0.a0;
  result += a0.a1;
  result += a0.a2;
  result += a1.a0;
  // ...

  std::cout << "result = " << result << "\n";
  return result;
}

```

Adding a test is as easy as adding a function type in the [configuration file](https://dart-review.googlesource.com/c/sdk/+/168829/6/tests/ffi/generator/structs_by_value_tests_confguration.dart). The ability to add tests quickly has resulted in a [huge test suite](https://dart-review.googlesource.com/c/sdk/+/168829).

Sure enough, this test suite caught another curious case in a native ABI — this time on iOS-ARM64. Non-struct arguments on the stack on iOS on ARM64 aren’t aligned to word size but to their own size. Structs are aligned to word size, except that if the struct is a homogeneous struct with only floats, then [it is aligned to the size of the float](https://github.com/dart-lang/sdk/blob/b6b82dd3ac756b39e7fe9cab21f060fae74e358d/runtime/vm/compiler/ffi/native_type.cc#L180-L182).

## Summary

This concludes our journey through the API design and ABI discovery. With a good test suite and thorough code reviews, we [landed support](https://dart-review.googlesource.com/c/sdk/+/140290) for passing structs by value in Dart FFI in December 2020 on the master branch, and it is available in Dart 2.12! If you’re interested in using Dart FFI, you can get started with the [C interop documentation on dart.dev](https://dart.dev/guides/libraries/c-interop). If you have any questions or comments on the API design and ABI discovery, feel free to leave a comment below. We’d love to hear from you!

*Thanks to the Dart language team and the (rest of the) Dart virtual machine team for their contributions to this Dart FFI feature, and thanks to Kathy Walrath and Michael Thomsen for shaping this blog post!*

