Living Document Notice
Published 2026-09-16. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Deterministic File Descriptors - Eliminating Leaked Handles Across Worker Spawns

Summary

Spawning concurrent worker subprocesses in batch conversion pipelines often introduces subtle handle inheritance bugs. When parent orchestrators hold open file descriptors to configuration databases, network sockets, or staging logs, standard fork and exec mechanics pass those descriptors to child processes.

Leaked file descriptors lock files on disk, prevent clean directory unmounting, and exhaust system handle tables during high-volume document ingestion. Outrigger enforces strict handle closure contracts at process spawn boundaries, guaranteeing that child extractors run with isolated, deterministic file descriptor tables.

POSIX Descriptor Leaks and O_CLOEXEC Guarantees

Under POSIX standards, file descriptors opened without the FD_CLOEXEC flag remain open across execve calls. When an orchestrator opens thousands of markdown target files while spawning child parsers, each child inherits references to every open descriptor held by the parent.

This inheritance prevents the parent from closing file locks and causes EMFILE (Too many open files) errors during batch operations. Outrigger requires all internal file operations to specify O_CLOEXEC atomically upon opening:

#define _GNU_SOURCE
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>

int open_isolated_file(const char *pathname) {
    // Atomically set O_CLOEXEC during descriptor creation
    int fd = open(pathname, O_RDWR | O_CREAT | O_CLOEXEC, 0644);
    if (fd < 0) {
        perror("Failed to open file with O_CLOEXEC");
        return -1;
    }
    return fd;
}

Prior to executing a child worker binary, Outrigger iterates through /proc/self/fd to verify that no descriptors other than standard input (0), standard output (1), and standard error (2) remain unflagged.

Handle Verification Loop Prior to Execve

When executing untrusted third-party binaries that may ignore runtime conventions, Outrigger executes a pre-exec closure sweep. The routine closes all open descriptors above index 2, eliminating leaked sockets or database handles.

use std::os::unix::process::CommandExt;
use std::process::Command;

pub fn spawn_clean_worker(binary: &str, args: &[&str]) -> std::io::Result<std::process::Child> {
    let mut cmd = Command::new(binary);
    cmd.args(args);

    unsafe {
        cmd.pre_exec(|| {
            // Close all inherited file descriptors above stderr
            let max_fd = match nix::unistd::sysconf(nix::unistd::SysconfVar::_SC_OPEN_MAX) {
                Ok(Some(limit)) => limit as i32,
                _ => 1024,
            };

            for fd in 3..max_fd {
                let _ = nix::unistd::close(fd);
            }
            Ok(())
        });
    }

    cmd.spawn()
}

This sweep guarantees that child workers cannot read or write to parent database handles, preventing concurrent state corruption across processing threads.

Handle Leaks Benchmark Under Continuous Worker Churn

The table below contrasts handle leakage and operating system descriptor consumption across 10,000 worker spawn cycles when processing note partitions.

Applying atomic descriptor flags at open time eliminates handle table exhaustion while keeping process spawn overhead under 100 microseconds per iteration.

Win32 Process Creation Flags

On Windows platforms, handle inheritance is suppressed by setting bInheritHandles to FALSE inside CreateProcessW.

#include <windows.h>
#include <stdio.h>

BOOL spawn_isolated_windows_worker(LPCWSTR application_name, LPWSTR command_line) {
    STARTUPINFOW si;
    PROCESS_INFORMATION pi;
    ZeroMemory(&si, sizeof(si));
    si.cb = sizeof(si);
    ZeroMemory(&pi, sizeof(pi));

    // Explicitly set bInheritHandles to FALSE
    BOOL success = CreateProcessW(
        application_name,
        command_line,
        NULL,
        NULL,
        FALSE, // Prevent handle inheritance
        CREATE_NO_WINDOW | DETACHED_PROCESS,
        NULL,
        NULL,
        &si,
        &pi
    );

    if (success) {
        CloseHandle(pi.hProcess);
        CloseHandle(pi.hThread);
    }
    return success;
}