Using NimScript as a configuration language (Embedding NimScript pt. 1)

31st May 2020 - Nim , Programming

 

Over the past couple of years I’ve been playing around with creating my own WM. I recently picked up the project again, and I quickly realised that it was time to re-do the configuration system. Up until this point I had been using a TOML file for the configuration. But it was proving a bit clumsy to use, and I found myself using lots of strings with custom formatted content for doing more complex things, not good! So I started to think about how I wanted to structure my new custom configuration format when I realised that it might be possible to use NimScript. For those of you who are unaware NimScript is the subset of Nim that is able to run within the VM that Nim uses for macros. It has pretty much all the same features, but it isn’t able to use C interop features (more on this later). But why would I use a full programming language for configuration? This article will explain using a programming language, and specifically NimScript as a configuration language, and part 2 will look at how to actually implement this in a Nim application.

Configuring with programming

When using a configuration language or custom format you are essentially writing a parser that reads a file to understand what the user wants certain values to be. But depending on how flexible your format is this might leave the user with a rather limited set of ways to express what they want. Currently I use the i3 window manager, and for that I have three almost identical configurations, one for my desktop with three mid-resolution screens, one for my work machine with two high-resolution screens, and one for my laptop with a single mid-resolution screen. This means that instead of having one configuration that simply changes certain values such as border width and workspace-to-monitor mappings I need to manually copy any changes I want replicated across to every script. Of course I could write myself a configuration configurator script, but that would need to be manually run, and would need to be able to read and parse the configuration.

However by using a programming language as our configuration language we can simply tell the user what they can change, and then let them change it however they like. It also allows us to set up callbacks into the script, so for example keybindings could be bound directly to NimScript procedures that could run multiple commands and react to context, with no extra work for us! This not only lets us save on time spent writing complex configuration parsing rules, but it also allows the user a lot more power over how they create their configurations.

This approach isn’t without its drawbacks though. Remember I mentioned that if I wanted to write a script that would configure my configuration file it would need to be able to read and parse the configuration? We get into a similar situation if we wanted to create e.g. a UI for editing a config written in a programming language. It would be nigh impossible to write a parser that would be able to tell you what each field was set to and what kind of code was behind the changes. Luckily this can be partially fixed in a couple of ways. One option if your program reads the configuration files in predetermined sequence would be to simply have your UI generate a “clean” file that is read after the user config. The UI would then read through all the same configuration files in order to have values to show in the UI, then it would be able to add in overrides to set things to certain values. It could of course also feature some kind of code editing or conditionals. Another option would be to leverage the scripting language itself. If the UI program moves the user config and spits out a generated file that starts with include userconfig it would be able to override whatever fields it wanted, and the user could still tweak their userconfig to their hearts desire. The UI program would then otherwise function in the same way and read this user config file to populate the fields.

But why NimScript?

Using a programming language for configuration is not a new idea. Vim has VimScript, Emacs has ELisp, and Awesome WM has Lua. In fact the Nim compiler and the Nim package manager nimble also uses NimScript as their configuration languages. Of course writing our own proprietary language like VimScript or ELisp would give us full control over what we can and cannot do in our language. But it also means we have to write our own language, which is a lot of work to get right. At least Lisps are comparatively simple to write, but even if we did write a small Lisp we would still need to provide proper error messages, maybe add in some optimisations, and overall we would just spend a lot of time on it. By re-using NimScript we get the benefit of all the good work that’s been done on Nim, similar to how Awesome benefits from using Lua.

But wouldn’t this mean that our users would have to learn Nim in order to configure our application? And this is where NimScript has another huge benefit over something like Lua. A friend of mine was using Awesome for a while and told me that “after teaching myself enough Lua to configure it it’s actually pretty nice”. Definitely not the smoothest user experience. With Nims strong macro system it’s fairly easy to set up convenience macros that make the most common simple tasks just as easy as writing a configuration file. Just look at the configuration files for the compiler for example, it has a -- template so instead of writing:

switch("opt", "size")
switch("define", "foo")
switch("forceBuild")

you can instead write

--opt:size
--define:foo
--forceBuild

which is exactly what you would write on the command-line. This means that for the common user it wouldn’t be any more complex to configure their application than normal. But for the power user they can simply extend their configuration with code!

Conclusion

With Nim and NimScript it’s easy to get a powerful language running within your application. And with Nims macro system it is easy to hide the complexity from the user until they want to use it. In the next article I’ll go through how exactly we can embed NimScript in applications, so if you’re sold on the idea head over to part 2!