Fixed-Point Neural Networks: The Math Behind Q16.16

Neural networks run on floating-point arithmetic. This is so universal that most practitioners never question it. But floating-point has a property that matters enormously in safety-critical systems: it is not deterministic across platforms.

The same floating-point calculation can produce different results on x86 versus ARM, on GPU versus CPU, or even between compiler versions. For research and most production systems, these differences are negligible. For systems that require certification under DO-178C, IEC 62304, or ISO 26262, they represent a fundamental barrier.

Fixed-point arithmetic offers an alternative. By representing fractional values as scaled integers, fixed-point achieves bit-identical results across all platforms. The trade-off is reduced range and precision compared to floating-point. Understanding this trade-off is essential for evaluating whether fixed-point is appropriate for a given application.

This article explains the mathematics of Q16.16 fixed-point representation, demonstrates the core operations required for neural network inference, and examines the precision implications for safety-critical AI systems.

Why Floating-Point Varies

Floating-point arithmetic follows the IEEE 754 standard, which specifies representation and basic operations. However, the standard permits variation in several areas that affect reproducibility.

Intermediate precision. The x87 floating-point unit in x86 processors uses 80-bit extended precision for intermediate calculations, even when the source and destination are 32-bit floats. ARM processors typically use 32-bit precision throughout. The same sequence of operations can accumulate different rounding errors.

Fused multiply-add. Modern processors offer FMA instructions that compute a * b + c with a single rounding step instead of two. Whether the compiler uses FMA depends on optimisation settings, target architecture, and sometimes the order of operations in source code. FMA produces more accurate results, but different results from separate multiply and add.

Associativity. Floating-point addition is not associative: (a + b) + c may differ from a + (b + c). Compilers that reorder operations for performance can change results. Parallel reduction algorithms that sum values in different orders produce different results.

Transcendental functions. Functions like exp(), log(), and sin() are not specified bit-exactly by IEEE 754. Different math libraries use different approximation algorithms.

For neural network inference, these variations typically produce outputs that differ in the least significant bits. The classification result is usually the same. But “usually the same” is insufficient for systems where certification requires demonstrable reproducibility.

The Fixed-Point Alternative

Fixed-point arithmetic represents fractional values as integers with an implicit scaling factor. The Q16.16 format uses a 32-bit signed integer where:

• The upper 16 bits represent the integer part
• The lower 16 bits represent the fractional part
• The scaling factor is 2^16 = 65536

A value like 3.25 is stored as 3.25 × 65536 = 212992. The key insight is that all arithmetic operates on integers, using the standard integer ALU that behaves identically on every platform.

Range and Precision

Q16.16 provides:

Compare this to 32-bit floating-point:

The reduced range and precision are significant constraints. Neural network weights and activations must be scaled to fit within ±32768, and accumulated rounding errors must be managed carefully. The benefit is certainty: the same input always produces the same output.

Core Operations

Neural network inference requires four fundamental operations: conversion, addition, multiplication, and accumulation. Each has specific implementation requirements in fixed-point.

Conversion

Converting from floating-point to Q16.16 during model quantisation:

typedef int32_t fixed_t;
#define FX_FRAC_BITS 16
#define FX_SCALE (1 << FX_FRAC_BITS)  // 65536

fixed_t fx_from_float(float f) {
    return (fixed_t)(f * FX_SCALE);
}

float fx_to_float(fixed_t x) {
    return (float)x / FX_SCALE;
}

The conversion truncates toward zero. For safety-critical applications, explicit rounding may be preferred:

fixed_t fx_from_float_rounded(float f) {
    float scaled = f * FX_SCALE;
    return (fixed_t)(scaled >= 0 ? scaled + 0.5f : scaled - 0.5f);
}

Addition and Subtraction

Fixed-point values with the same Q format add directly:

fixed_t fx_add(fixed_t a, fixed_t b) {
    return a + b;
}

fixed_t fx_sub(fixed_t a, fixed_t b) {
    return a - b;
}

Overflow is the primary concern. Adding two Q16.16 values near the maximum can overflow the 32-bit representation. For safety-critical systems, saturation arithmetic prevents wrap-around:

#define FX_MAX  ((fixed_t)0x7FFFFFFF)
#define FX_MIN  ((fixed_t)0x80000000)

fixed_t fx_add_sat(fixed_t a, fixed_t b) {
    int64_t result = (int64_t)a + (int64_t)b;
    if (result > FX_MAX) return FX_MAX;
    if (result < FX_MIN) return FX_MIN;
    return (fixed_t)result;
}

Multiplication

Multiplying two Q16.16 values produces a Q32.32 result in 64 bits. The result must be shifted right by 16 bits to return to Q16.16:

fixed_t fx_mul(fixed_t a, fixed_t b) {
    int64_t product = (int64_t)a * (int64_t)b;
    return (fixed_t)(product >> FX_FRAC_BITS);
}

This operation truncates the lower 16 bits of the 64-bit product. For applications requiring rounding:

fixed_t fx_mul_rounded(fixed_t a, fixed_t b) {
    int64_t product = (int64_t)a * (int64_t)b;
    // Add 0.5 in the fractional position before shifting
    product += (1 << (FX_FRAC_BITS - 1));
    return (fixed_t)(product >> FX_FRAC_BITS);
}

Division

Division is less common in inference (weights are pre-computed), but when needed:

fixed_t fx_div(fixed_t a, fixed_t b) {
    int64_t dividend = (int64_t)a << FX_FRAC_BITS;
    return (fixed_t)(dividend / b);
}

Division by zero must be handled explicitly. Safety-critical code typically validates the divisor before the operation rather than relying on runtime exceptions.

Neural Network Operations

With the core arithmetic defined, neural network layers follow directly.

Matrix Multiplication

The fundamental operation in dense layers and convolution:

void fx_matmul(const fixed_t* A, const fixed_t* B, fixed_t* C,
               int M, int K, int N) {
    for (int i = 0; i < M; i++) {
        for (int j = 0; j < N; j++) {
            int64_t acc = 0;  // Accumulate in 64 bits
            for (int k = 0; k < K; k++) {
                acc += (int64_t)A[i * K + k] * (int64_t)B[k * N + j];
            }
            C[i * N + j] = (fixed_t)(acc >> FX_FRAC_BITS);
        }
    }
}

The 64-bit accumulator is critical. A matrix multiplication with K=1000 elements could overflow a 32-bit accumulator even if individual products fit. The shift happens once at the end, preserving precision during accumulation.

Convolution

2D convolution for image processing:

void fx_conv2d(const fixed_t* input, const fixed_t* kernel, fixed_t* output,
               int in_h, int in_w, int k_h, int k_w) {
    int out_h = in_h - k_h + 1;
    int out_w = in_w - k_w + 1;
    
    for (int oh = 0; oh < out_h; oh++) {
        for (int ow = 0; ow < out_w; ow++) {
            int64_t acc = 0;
            for (int kh = 0; kh < k_h; kh++) {
                for (int kw = 0; kw < k_w; kw++) {
                    int ih = oh + kh;
                    int iw = ow + kw;
                    acc += (int64_t)input[ih * in_w + iw] * 
                           (int64_t)kernel[kh * k_w + kw];
                }
            }
            output[oh * out_w + ow] = (fixed_t)(acc >> FX_FRAC_BITS);
        }
    }
}

Activation Functions

ReLU is trivial in fixed-point:

fixed_t fx_relu(fixed_t x) {
    return x > 0 ? x : 0;
}

More complex activations like sigmoid or tanh require lookup tables or polynomial approximations:

// Sigmoid approximation using piecewise linear segments
fixed_t fx_sigmoid_approx(fixed_t x) {
    // Saturate for large inputs
    if (x >= FX_FROM_INT(4)) return FX_FROM_FLOAT(0.9820f);
    if (x <= FX_FROM_INT(-4)) return FX_FROM_FLOAT(0.0180f);
    
    // Linear approximation in middle range
    // sigmoid(x) ≈ 0.5 + 0.25*x for |x| < 1
    if (x >= FX_FROM_INT(-1) && x <= FX_FROM_INT(1)) {
        return FX_FROM_FLOAT(0.5f) + fx_mul(FX_FROM_FLOAT(0.25f), x);
    }
    
    // Lookup table for intermediate ranges
    // ... implementation depends on precision requirements
}

The approximation introduces error relative to the true sigmoid function. This error must be characterised and shown to be acceptable for the application.

Max Pooling

Pooling operates on integer comparisons, which are exact:

void fx_maxpool_2x2(const fixed_t* input, fixed_t* output,
                    int in_h, int in_w) {
    int out_h = in_h / 2;
    int out_w = in_w / 2;
    
    for (int oh = 0; oh < out_h; oh++) {
        for (int ow = 0; ow < out_w; ow++) {
            int ih = oh * 2;
            int iw = ow * 2;
            
            fixed_t max_val = input[ih * in_w + iw];
            if (input[ih * in_w + iw + 1] > max_val)
                max_val = input[ih * in_w + iw + 1];
            if (input[(ih + 1) * in_w + iw] > max_val)
                max_val = input[(ih + 1) * in_w + iw];
            if (input[(ih + 1) * in_w + iw + 1] > max_val)
                max_val = input[(ih + 1) * in_w + iw + 1];
            
            output[oh * out_w + ow] = max_val;
        }
    }
}

Precision Analysis

The practical question is: does reduced precision affect model accuracy?

Quantisation Error

Converting a trained floating-point model to Q16.16 introduces quantisation error in every weight. For a weight w, the quantised value is:

w_q = round(w × 65536) / 65536

The maximum error per weight is ±0.5/65536 ≈ ±7.6 × 10^-6. For a layer with N weights, these errors accumulate. The total error depends on the statistical distribution of weights and the structure of the computation.

Empirically, Q16.16 typically maintains classification accuracy within 1-2% of the original floating-point model for common architectures. However, this varies significantly by model and task. Rigorous evaluation on the target application is essential.

Accumulation Error

Each multiplication introduces rounding error when the 64-bit product is shifted back to 32 bits. For a dot product of length K:

• K multiplications, each with error up to 1 LSB
• One final shift with error up to 1 LSB
• Total error bounded by K + 1 LSBs

For large K (thousands of weights per neuron), this can become significant. Techniques to manage accumulation error include:

1. Larger accumulators: Use 64-bit arithmetic throughout
2. Block accumulation: Sum in smaller blocks, normalise between blocks
3. Kahan summation: Track and compensate for rounding errors (increases complexity)

Overflow Prevention

The Q16.16 range of ±32768 constrains both weights and activations. Model quantisation must ensure:

• All weights fit within the representable range
• Activations cannot exceed the range during inference
• Accumulated values in matrix operations fit in 64-bit intermediates

This typically requires scaling the model. A common approach:

1. Analyse the trained model to find weight and activation ranges
2. Apply per-layer scaling factors to fit within Q16.16
3. Adjust bias terms to compensate for scaling
4. Validate accuracy after quantisation

Practical Considerations

Memory Layout

Fixed-point values are 32-bit integers. Memory requirements match float32 exactly. There is no memory advantage to Q16.16 over float32, unlike INT8 quantisation.

The advantage is computational: fixed-point operations use the integer ALU, which on some embedded processors is faster and more power-efficient than the FPU. More importantly, integer operations are deterministic.

No Dynamic Allocation

Safety-critical systems typically prohibit dynamic memory allocation after initialisation, as discussed in The Real Cost of Dynamic Memory in Safety-Critical Systems. The fixed-point implementations shown here use caller-provided buffers:

// Caller allocates all buffers
fixed_t input[256];
fixed_t kernel[9];
fixed_t output[196];

// Function uses only provided memory
fx_conv2d(input, kernel, output, 16, 16, 3, 3);

This enables static memory analysis and prevents heap fragmentation, both requirements for DO-178C Level A certification.

Testing Determinism

Verifying bit-exact reproducibility requires testing across platforms:

void test_determinism(void) {
    // Known input
    fixed_t a = 0x00028000;  // 2.5
    fixed_t b = 0x00018000;  // 1.5
    
    // Expected output (computed once, verified manually)
    fixed_t expected = 0x0003C000;  // 3.75
    
    fixed_t result = fx_mul(a, b);
    
    assert(result == expected);  // Exact equality, not approximate
}

These tests must pass on every target platform. Any difference indicates a determinism failure that must be investigated.

Trade-offs and Limitations

Fixed-point Q16.16 is not universally appropriate. Consider these factors:

For safety-critical systems requiring certification, the determinism guarantee often outweighs the precision limitations. For research or high-precision applications, floating-point remains the appropriate choice.

Implementation Reference

The certifiable-inference project provides a complete implementation of fixed-point neural network inference in C99, including:

• Q16.16 arithmetic with saturation
• Matrix operations
• 2D convolution
• Activation functions (ReLU, approximated sigmoid)
• Max pooling

The implementation passes determinism tests across x86, ARM, and RISC-V platforms. A live simulator demonstrates the inference pipeline.

Conclusion

Fixed-point arithmetic trades precision for determinism. The Q16.16 format provides sufficient precision for many neural network applications while guaranteeing bit-identical results across platforms.

The mathematics are straightforward: scale values by 65536, use integer arithmetic, and shift results after multiplication. The engineering challenge lies in managing overflow, quantising models appropriately, and validating that reduced precision does not unacceptably degrade accuracy.

For systems requiring certification under DO-178C, IEC 62304, or ISO 26262, fixed-point may provide a path that floating-point cannot. The ability to prove that identical inputs produce identical outputs simplifies verification and validation significantly.

As with any architectural approach, suitability depends on system requirements, precision constraints, and regulatory context. Fixed-point is not a universal solution, but for safety-critical AI where determinism matters more than dynamic range, it offers a mathematically sound foundation.

For a working implementation of these principles, see certifiable-inference or try the live simulator.