Blog
Sorry, your browser does not support inline SVG.

Nim Meets YottaDB

Lothar Jöckel

We thank Lothar Jöckel for his first guest blog post, and hope there are many more to follow. If you would like to post on the YottaDB blog please contact us at info@yottadb.com.

TL;DR

If you’re a Nim developer interested in working with a powerful hierarchical NoSQL engine, or an M developer interested in working with powerful modern programming language then this is for you.

The combination of Nim’s modern language features with YottaDB’s battle-tested database engine creates a powerful stack for building high-performance, reliable systems. This binding bridges the gap between a database proven over decades of use in mission-critical applications and a contemporary systems programming language.

I’m pleased to announce nim-yottadb, a language binding that connects Nim with the YottaDB database. This gives you direct access to global and local variables, transactions, iteration, locks, and more – all from Nim.

A Simple Example

setvar:
    ^Users("john_doe", "profile", "name") = "John Doe"
    ^Users("john_doe", "profile", "email") = "john@example.com"

let userName = get: ^Users("john_doe", "profile", "name")
echo "Hello, ", userName

In this post, I want to walk you through:

  • What YottaDB is (at a high level)
  • Why binding it to Nim is interesting
  • What features nim-yottadb currently offers
  • How its API and DSL (Domain Specific Language) design works
  • Caveats, threading, and implementation notes
  • Examples and next steps

Key Features of YottaDB

YottaDB is a high-performance, schema-less, key-value database designed for extreme scalability and reliability, particularly in transaction-heavy environments. Its origin is the M (affectionately known as MUMPS) language and database, which has been battle-tested in mission-critical systems for decades.

Key-Value Data Model with a Hierarchical Twist

At its core, data is stored as sparse, multi-dimensional arrays. A “global” variable starts with a caret (^), can have subscripts, creating a natural tree structure. A global variable is just a key-value node that persists and is shared, i.e., it is in the database.

^Patients("Smith", "John", 2024, "Visit") = "Checkup"

This model is incredibly flexible (schema-less) and allows for efficient hierarchical data access.

Extreme Performance and Low Latency

YottaDB has a in-memory database engine with cooperating processes managing database files that use transaction journals for Durability. All data operations are performed directly in memory, making it exceptionally fast.

A daemonless database whose logic executes in the address spaces of processes accessing the database, with control structures in shared memory, eliminates connection overheads and minimizes resource contention. This yields massive scalability on multi-core servers.

Rock-Solid Reliability and ACID Transactions

YottaDB is proven in mission-critical applications where data loss is unacceptable (e.g., banks, hospitals). It provides fully ACID (Atomic, Consistent, Isolated, Durable), robust transaction processing system that uses write-ahead journaling. Database updates are first written to a journal file before being applied to the database, ensuring data can be recovered after a crash.

Tight Integration of Database and Programming Language

This is a hallmark of the M heritage. Database operations are simple commands within the M language. There is no separate query language (like SQL) or connection string. A simple command like SET ^Customer(123)="John" both updates the variable in memory and commits the change to the database. This eliminates object-relational mapping (ORM) overhead and makes the code very concise for data manipulation.

YottaDB brings this tight integration to languages other than M.

Mature and Robust Codebase

M technology has its roots in the 1960s MUMPS. YottaDB itself is a direct descendant of GT.M, which was first deployed in 1986. The code base has decades of use in mission-critical, high-availability systems, and is the database of record at some of world’s largest real-time core-banking systems.

Efficient Database Replication

Efficient database replication provides built-in, low-latency replication between database instances. This is crucial for creating hot-standby systems for disaster recovery. Replicas can also be used downstream from production systems for real-time analytics, reporting, etc.

Replication protects mission-critical applications like core banking systems, healthcare information systems, stock exchanges, and any other application that requires the ability to remain continuously available in the face of unplanned as well as planned events.

Nim

The Nim programming language is a statically typed, compiled systems language that has a unique and powerful set of features. It’s often described as having the performance of C or C++, the expressiveness of Python, and the safety of Rust or Ada.

Python-like Syntax with Static Typing

This is one of the most immediately appealing features. Nim’s syntax uses significant whitespace (indentation) like Python, making it clean and easy to read. Unlike Python, it’s statically typed, meaning type errors are caught at compile time, leading to more robust and performant code. The compiler does all the type checking before the program ever runs.

Looks like Python, but is statically typed and compiled!

proc greet(name: string, age: int): string =
    return "Hello, " & name & ". You are " & $age & " years old."

echo greet("Alice", 30)

Compiles to Efficient C, C++, and JavaScript

Nim doesn’t have a virtual machine. Instead, it compiles its source code down to another language. By compiling to C, C++, or even Objective-C, it achieves performance comparable to these native languages. The generated C code can be compiled on virtually any platform.

The JavaScript target allows you to write both your backend and frontend logic in the same language, enabling full-stack development with Nim.

Powerful Metaprogramming

This is one of Nim’s superpowers. You can generate code at compile time, reducing boilerplate and creating powerful Domain-Specific Languages (DSLs).

With Templates you perform simple syntactic substitutions (hygienic macros). They are like C macros but much safer and more integrated.

The most powerful feature is Macros. You can manipulate the Abstract Syntax Tree (AST) of your code at compile time. This allows you to implement new language features, validate complex conditions, or generate code based on custom logic.

Memory Safety and Control

Nim offers a pragmatic approach to memory management.

It comes with several built-in garbage collectors (e.g., deferred reference counting, mark-and-sweep, …). The default is fast and pause-free for most applications.

For systems programming or real-time applications where garbage collection pauses are unacceptable, you can use manual memory management (alloc(), dealloc()) or leverage the --gc:arc or --gc:orc (Owned Reference Counting) options, which provide deterministic, non-tracing memory management without a garbage collector.

Generics, Union Types, and More

Nim’s type system is both practical and expressive. Full support for generic programming, allows you to write flexible and reusable code for different types.

Sum Types (Variant Objects) let you define a type that can hold values of different, but fixed, types. This is excellent for modeling state.

With Distinct Types you create a new type that has the same underlying representation as an existing type but is considered incompatible with it (e.g., type Meter = distinct int and type Kilogram = distinct int prevents you from accidentally adding meters to kilograms).

Zero-Cost Abstraction and Efficiency

Nim is designed to be highly efficient.

  • No Runtime Overhead: Features like iterators, templates, and generics are resolved at compile time, resulting in code that is as fast as hand-written C.
  • Value Types: Structs are value types by default (stored on the stack), which is cache-friendly and fast.
  • Direct Control: You have low-level control over memory layout, pointers, and can even inline assembly code when needed.

Unified Function Call Syntax (UFCS)

This syntactic sugar allows for both method-call and function-call syntax, where a.f(b) is equivalent to f(a, b). This enables a fluent, “chaining” style of programming that is very readable, similar to what you find in Unix pipes or Rust.

Cross-Platform and Interoperability

  • Native Executables: Nim compiles to a single, dependency-free native executable, making deployment trivial.
  • Excellent C/C++ Interoperability: You can directly import and call C/C++ functions and libraries with minimal effort, making it easy to leverage existing codebases.
  • Cross-Compilation: It’s straightforward to compile for a different target platform (e.g., compile for Windows on a Linux machine).

Async/Await for Concurrency

Nim has a built-in async/await mechanism for writing highly scalable asynchronous I/O operations, similar to what you find in Python, JavaScript, or C#. This makes it well-suited for network servers and clients.

What nim-yottadb provides

Many environments use YottaDB already. With a Nim binding, you can write new components or tooling in Nim that integrate with existing YottaDB data.

The flexibility of Nim’s metaprogramming (macros, templates) enables a nicer API and DSL wrapper over the more “raw” C interface. You can mask lower-level details, make code more expressive, and reduce boilerplate.

Core (Simple-API)

The binding exposes a basic set of database operations, roughly mapping to YottaDB capabilities:

  • ydb_data — inspect node or subtree state (e.g. whether there is data, subtree, both or neither)
  • ydb_delete — delete a node or an entire subtree
  • ydb_delete_excl — delete local variables except some exclusions
  • ydb_get / ydb_set — read or assign the value of a local or global variable
  • ydb_incr — atomic increment (local or global)
  • ydb_lock — lock one or more global variables
  • ydb_lock_incr / ydb_lock_dec — manipulate a lock count
  • ydb_node_next / ydb_node_previous — traverse siblings or nodes in and out of order
  • ydb_subscript_next / ydb_subscript_previous — step through subscript ranges under a global
  • ydb_ci — call a routine (M / YottaDB “Call-In Interface”)

These correspond fairly directly to the YottaDB C API (or M primitives). The binding supports both single-threaded and multi-threaded modes (automatically selected when compiling with --threads).

Extensions & syntactic sugar

To make working with the binding more ergonomic, nim-yottadb adds:

Iterators

You can iterate over next/previous nodes or subscripts, looping over nodes instead of manually calling next node/sub.

YdbVar

YdbVar is a type with overloaded operators ($, []) so that a global can be referenced in a natural, array-like way.

DSL

Instead of using the simple API directly an alternative exists with a DSL (Domain Specific Language). The DSL offers Nim-style keywords/mnemonics for common operations. So instead of writing

ydb_setvar("^building", @["Room", "1", "size"], "22.5")

you can write

setvar: ^building("Room", 1, "size")=22.5

setvar / get

setvar:
    ^XX(1,2,3)=123
    ^XX("B",1)="AB"

Support for mixed type subscripts

setvar: ^X(id, 4711, "pi") = 3.1414

setvar: in a loop

for id in 0..<5:
    setvar:
        ^CUST(id, "Timestamp") = cpuTime()
        ^CUST(id, "loop") = id

increment

Increment a global in the database by 1

let nexttxid = increment: ^CNT("TXID")
let accid = increment: ^Customer(nexttxid, by=1000)

data

Test if a node or tree exists and has a subtree.

setvar:
    ^X(5)="F"
    ^X(5,1)="D"
dta = data: ^X(5)
assert YdbData(dta) == YDB_DATA_VALUE_DESC

delnode

Delete a node. If all nodes of a global are removed, the global itself is removed.

delnode: ^X(1) # delete node

deltree

Delete a subtree of a global. If all nodes are removed, the global itself is removed.

deltree: ^X(1)

lock

Lock upto 35 named lock resources. Other processes trying to lock a locked resource must wait until the lock is released. {} has to be used to lock multiple resources in one operation; an empty list releases all locked resources. If lock: is called again, the previous locks are automatically released first.

lock:
  {
    ^LL("HAUS", "11"),
    ^LL("HAUS", "12"),
    ^LL("HAUS", "XX"), # not yet existent, but ok
  }

Additional locks can be acquired or released, without affecting other locks held by a process, by prefixing lock resource names + or -.

The template withlock simplifies the locking further:

let amount = 1500.50
withlock(4711):
  setvar:
    ^custacct(4711, "amount") = amount
    ^booking(4711, "txnbr") = amount

On leaving the withlock block, the lock is automatically released.

nextnode / prevnode / nextsubscript / prevsubscript

Traverse a global/subscript in the collating sequence.

(rc, node) = nextnode: ^LL()
(rc, node) = prevnode: ^LL("HAUS", "ELEKTRIK", "DOSEN", "1")
(rc, subs) = nextsubscript: ^LL("HAUS", "ELEKTRIK")
(rc, subs) = prevsubscript: ^LL("HAUS", "FLAECHEN")

‘get’ with postfix

It is possible to enforce a type when getting data from YottaDB. By using a “postfix” an expected type can be defined and tested.

let i = get: ^global(1).int16
let f = get: ^global(4711).float32

If the value from the db is greater or smaller than the range defined through the postfix, a ValueError exception is raised.

The following postfixes are implemented:

  • int, int8, int16, int32, int64
  • uint, uint8, uint16, uint32, uint64
  • float, float32, float64

The .binary Postfix

The binary postfix allows binary data of virtually unlimited size to be read from the DB. setcan save data, theoretically upto 99,999,999 MB.

let dbval = get: ^tmp(4711).binary

Saving a Nim Object-Tree to the database

Based on the Nim object model, it is possible to store objects, even complex ones, in the database. A global variable is created for each type, e.g., Address, Customer, etc. Attributes are then stored with their corresponding names.

type
  Address* = object of RootObj
    street*: string
    zip*: uint
    city*: string
    state*: string

let address = Address(street: "Bachstrasse 14", zip:6033, city:"Buchs", state:"AG")
store(@["4711"], address)

The data is stored as

^Address(4711,"city")="Buchs"
^Address(4711,"state")="AG"
^Address(4711,"street")="Bachstrasse 14"
^Address(4711,"zip")=6033

Performance

In general, the Nim / YottaDB language binding has excellent performance. Simple tests on a MacMini M4 with a virtualized Ubuntu (2 Cores / 4GB Memory) gives the following figures where every test had 10 million different records.

upcount dsl 2439 ms. (Increment a Global)
set dsl 2479 ms. (Set global value)
nextnode dsl 1536 ms. (Iterator over all nodes)
delnode dsl 2774 ms. (Delete all nodes)

This means writing 4,100,041 records per second and traversing the nodes at 6,510,416 nodes per second. I think these are impressive numbers!

A Larger Example

The following program retrieves images from a directory, stores them in the database, and extracts them into another directory.

import os
import std/[times, strutils]
import yottadb

proc walk(path: string): seq[string] =
    for kind, path in walkDir(path):
        case kind:
        of pcFile, pcLinkToFile:
            result.add(path)
        of pcDir, pcLinkToDir:
            result.add(walk(path))

proc loadImagesToDb(basedir: string) =
    for image in walk(basedir):
        let image_number = increment(^CNT("image_number"))
        setvar:
            ^images($image_number) = readFile(image)
            ^images($image_number, "path") = image
            ^images($image_number, "created") = now()

proc saveImage(target: string, path: string, img: string) =
    if not dirExists(target):
        createDir(target)

    let filename = path.split("/")[^1]
    let fullpath = target & "/" & filename
    writeFile(fullpath, img)

proc readImagesFromDb(target: string) =
    var (rc, subs) = nextsubscript: ^images(@[""]) # -> @["223"], @["224"], ...
    while rc == YDB_OK:
        let img     = getblob(^images(subs))
        let path    = get(^images(subs, "path"))
        saveImage(target, path, img)
        (rc, subs) = nextsubscript: ^images(subs)

if isMainModule:
    loadImagesToDb("./images") # read from the folder and save in db
    readImagesFromDb("./images_fromdb") # read from db and save under this folder

Conclusion

The nim-yottadb binding successfully bridges two powerful technologies from different eras of computing. YottaDB brings decades of refinement in hierarchical data management and transaction processing, while Nim offers modern language features, metaprogramming capabilities, and performance characteristics that rival lower-level systems languages.

What makes this integration particularly compelling is how Nim’s DSL capabilities and clean syntax make YottaDB’s hierarchical data model feel natural and expressive. The ability to write database operations that look like native Nim code, while maintaining the performance and reliability of a battle-tested database engine, represents the best of both worlds.

The performance benchmarks demonstrate that this binding doesn’t sacrifice speed for convenience—Nim applications can leverage YottaDB’s capabilities with minimal overhead, making it suitable for the same high-performance, transaction-heavy use cases that YottaDB has traditionally served.

For developers working with existing YottaDB systems, nim-yottadb provides a path to modernize tooling and develop new components without abandoning proven database infrastructure. For Nim developers, it opens access to a unique class of hierarchical database that excels in scenarios where relational databases might struggle.

As the binding continues to evolve, it represents not just a technical achievement, but a practical solution for building robust, high-performance systems that need both modern development ergonomics and proven data reliability. Whether you’re extending legacy M applications or building new systems from scratch, nim-yottadb offers a compelling combination of performance, reliability, and developer experience.

The project is available on GitHub as nim-yottadb, and welcomes contributions from both the Nim and YottaDB communities.

About Lothar Jöckel

Lothar began his IT career in 1989 at McDonnell Douglas in the Health Software Division,  with Pick/Reality and the “Homer” system. In a varied career of many years, he worked on library systems, banking systems, military and intelligence systems, and was an early user of AI image recognition methods to determine the positions of trains in the Swiss Federal Railways (SBB). He is curious about new languages and frameworks, such as Rust and Nim.

Contact: lothar.joeckel@gmail.com

Credits

  1. Visualization of Nimber products of powers of 2. Copyright Watchduck and licensed under the Creative Commons Attribution 3.0 Unported license.
  2. Blog roll picture of Nim game. Copyright Uncopy and licensed under the Creative Commons Attribution 3.0 Unported license.

Published on October 07, 2025

See All Announcements See All General See All What's New

Back to Blog