Teaching old C code new tricks with Nim

8th September 2023 - Guide , Nim , Programming

Recently I was met with an interesting problem when wrapping a C library in Nim. The library in question was MAPM, an older but quite complete library for dealing with arbitrary precision maths. Unfortunately the library doesn’t have much in the way of error handling. If something goes wrong it almost always writes to stderr and returns the number 0. And to be fair, there isn’t a whole lot that can go wrong in this library. Pretty much every error scenario is bad input to functions like trying to divide by 0 or trying to get trigonometry results for impossible angles. However in the case where malloc/realloc isn’t able to allocate more data then it writes to stderr and then calls exit(100). This sounds pretty terrible, but as the author points out the alternative isn’t great either, and there are ways to work around it. I do wish that the author had opted to use error flags like many of the C standard library functions, this way it’d be easier to deal with these errors, but alas.

So what do we do? I could add range checks to all inputs in my wrapper, which works, but isn’t great for performance. I could of course disable these when the user compiles with -d:danger like the Nim compiler itself does. But this still doesn’t feel like a great solution. And besides, MAPM does all these checks itself, so we’d be checking everything twice! Initially I wondered if it would be possible to read from the programs own stderr, or to replace stderr with a stream we could read from before calling MAPM functions and swap it back afterwards. But this seemed like a lot of hassle for quite small benefit.

The solution: old C tricks

Luckily the library performs all this error handling with an internal function called M_apm_log_error_msg. This function takes two arguments, one which decides if it’s a fatal error and exit(100) should be called, and the other which contains the message to display. And as it turns out ld, the GNU linker which ships with gcc, has an option called --wrap and has this to say about it in the documentation:

--wrap symbol
        Use a wrapper function for symbol. Any undefined reference to symbol will be resolved to
        __wrap_symbol. Any undefined reference to __real_symbol will be resolved to symbol. This
        can be used to provide a wrapper for a system function. The wrapper function should be
        called __wrap_symbol. If it wishes to call the system function, it should call
        __real_symbol. Here is a trivial example:

        void *
        __wrap_malloc (int c)
        {
          printf ("malloc called with %ld\n", c);
        return __real_malloc (c);
        }

        If you link other code with this file using --wrap malloc, then all calls to malloc will
        call the function __wrap_malloc instead. The call to __real_malloc in __wrap_malloc will
        call the real malloc function. 

So by simply passing --wrap M_apm_log_error_msg all calls in the library to this function will be converted into a call to __wrap_M_apm_log_error_msg with the same signature as the original. This means that if we pass this to the linker while supplying that function in a C callable way the library will simply call back to us instead of calling the implementation from MAPM. And if we want to call the original from within our wrapper we could simply call __real_M_apm_log_error_msg. However in our case we simply want to replace the entire function.

Making it work in Nim

Armed with our new knowledge of --wrap lets investigate what the M_apm_log_error_msg function actually does, and see if we can convert it into something useful in Nim:

void    M_apm_log_error_msg(int fatal, char *message) {
    if (fatal) {
        fprintf(stderr, "MAPM Error: %s\n", message);
        exit(100);
    } else {
        fprintf(stderr, "MAPM Warning: %s\n", message);
    }
}

As we can see it has two modes, one which simply writes out a message to stderr and the other which also terminates the program. The error case is easy to reason about, just convert it into a Defect and throw it (although as we will see it’s not trivial to implement). The simply writing out a message to stderr case is a bit harder. As mentioned in the introduction functions which errors out in this way will write these messages but still return the number 0. There are two complicating factors here. First off we aren’t guaranteed that it will only call this function once, a motivating example from m_apm_set_string which parses a textual representation of a number into a MAPM number:

if (((int)ch & 0xFF) >= 100) {
    M_apm_log_error_msg(M_APM_RETURN,
    "\'m_apm_set_string\', Non-digit char found in parse");

    M_apm_log_error_msg(M_APM_RETURN, "Text =");
    M_apm_log_error_msg(M_APM_RETURN, s_in);

    M_set_to_zero(ctmp);
    return;
}

As we can see M_apm_log_error_msg is called multiple times to display more than a single line of the error message, if we had simply turned the first call into an exception we would lose the following two lines.

The second problem is that by throwing an exception at any of these log messages would disrupt the control flow. Let’s say we somehow found a way to only throw the exception after the third message, we would end up not calling M_set_to_zero. Of course in this case this is our result variable which we won’t use anyways because of the exception, but MAPM could do other cleanup before returning which might be more critical.

So in summary we need to collect the messages, and only after the call returns can we turn it into an exception. What I ended up doing was to simply create a global variable to hold the messages, and then a template to check if this variable was not empty and throw an exception. Something along these lines:

type MapmError = object of CatchableError

var messages = ""

{.passL:"-Wl,--wrap=M_apm_log_error_msg".}
proc mapmErrorHandler(fatal: cint, message: cstring) {.exportc: "__wrap_M_apm_log_error_msg.} =
  if fatal == 0:
      messages.add message & "\n"

template errChk(): untyped =
  if messages.len != 0:
        let ex = newException(MapmError, messages.strip)
        messages = "" # So that the buffer won't already have stuff if we catch the exception and continue using MAPM
        raise ex

As we can see here we need to pass -Wl,--wrap= instead of just --wrap, this is simply because we compile with GCC and the -Wl flag tells GCC that this should be passed on to ld. Apart from that this is pretty straight forward, add messages to the messages buffer, and after every call to a MAPM function we use errChk to check if the buffer has anything in it, and if so raise an exception with the collected messages. Since the exception is raised from Nim code this works well and even Valgrind is happy with our use of the managed messages variable inside a C function.

Breaking the flow from C

You might have noticed that I quite critically left out the path for fatal == 1 in the above sample. As we’ve seen with fatal == 0 we need to ensure that the original control flow is still the same. After all the fatal == 1 errors means that malloc or realloc has failed and we’re out of memory. Continuing to execute the program after this while the programmer had expected the program to exit is not a great idea and will just cause errors. So we need to break out of the function right away, and we have to make sure we don’t try to allocate any further memory in doing so. And I can hear you say “can’t we just throw the exception in mapmErrorHandler?” which seems like the sensible thing to do. The problem however is that since C doesn’t have exceptions Nim needs to deal with exceptions on its own. This means that after every call to a Nim function which can throw anything there is injected a small piece of logic to check if an exception was raised, and if so to actually deal with it. However in C this logic isn’t present, and since we’re in a function we have no way to force C to return up the chain to Nim (short of hacking something with jumping, but let’s not go there). What I ended up with was a combination of a emit, some memory copying, and then a normal raise. It looks a bit like this:

type
  MapmError = object of CatchableError
    MapmDefect = object of Defect

var
  messages = ""
    defect = newException(MapmDefect, newString(100)) # Pre-allocate the defect and some room for an error

{.passL:"-Wl,--wrap=M_apm_log_error_msg".}
proc mapmErrorHandler(fatal: cint, message: cstring) {.exportc: "__wrap_M_apm_log_error_msg.} =
  if fatal == 0:
      messages.add message & "\n"
    else:
        copyMem(defect.msg[0].addr, message, message.len) # Copy our message into our pre-allocated buffer
        raise defect # "Raise" our exception
        {.emit: "nimTestErrorFlag();".} # Actually check if the exception was raised and act accordingly

Since the raise statement doesn’t actually do much more than register our exception as thrown we need to manually insert the nimTestErrorFlag into our code to ensure that the exception is actually checked and acted upon immediately. Of course for Defect this doesn’t really matter terribly much since Nim by default won’t let you catch them anyways. Now if we run this and trigger some fatal error (I cheated this and simply inverted the fatal == 0 check) we can see that the program properly throws the Defect and writes out a stack trace for where it occured.

Final remarks and caveats

All in all this system works great, it allows a C library which was written before Nim existed to raise exceptions and convert stderr messages into proper exception messages. There are however a couple details which I’ve ignore here that should be mentioned. Caveat number one is that the C compiler is free to optimise away calls, so we apparently aren’t actually guaranteed that --wrap will get all occurrences of our function. I’ve tested this with MAPM and it doesn’t appear to be an issue, but with more aggressive compiler flags enabled this would be something to look out for. The second is that I mentioned that we get a nice stack trace in our defect case. We take care to pre-allocate our exception and string buffer, but there is no guarantee that Nim won’t allocate data for the stack trace. Of course if the malloc that was requested was very large it might be possible that Nim would still be able to allocate a smaller buffer for this. But your mileage may vary in this case.