Building a superscalar cpu:
Losing my sanity in the process

The Origin Story

It started, as many bad ideas do, with a video game. Turing Complete. if you haven’t played it, it’s basically “Zachtronics meets computer architecture.” You start with NAND gates and work your way up to a functioning CPU. It’s educational, addictive, and dangerously misleading about how easy this whole “building a computer” thing actually is.

I was deep into the “save breaker” branch (pre‑2.0 alpha, for those keeping score), having built what I thought was a pretty solid 16‑bit architecture. The ISA was clean, the pipeline made sense, and everything worked beautifully in the game’s simulated environment.

Then I made the mistake that would consume the next several months of my life: “How hard could it be to build this in real hardware?”

The First Rude Awakening

The answer, as it turns out, is “very.”

The game’s ISA looked like this:

NAND dst, arg_a, arg_b
ADD dst, arg_a, arg_b
LD16 dst, [arg_b]
JMP label

Pretty standard stuff. The instruction format packed nicely into 32 bits:

[Mode:2][Imm:1][Opcode:4][Dst:4][ArgA:4][ArgB:16]

Everything had a nice, clean abstraction. In the game, memory was instantaneous, pipelines were perfect, and timing constraints didn’t exist.

In real Verilog? Not so much.

The Architecture

Let’s break down what I was actually trying to build:

The ISA

The ISA is slightly modified from what the game provided (specifically in I/O as i didn’t have a convenient universal input or output pin to work with).

• 16‑bit word width, 32‑bit instructions
• 4‑bit register addressing (16 registers, with zr, flags, and sp as special cases)
• ALU operations: NAND, OR, AND, NOR, ADD, SUB, XOR, LSL, LSR, CMP, MUL
• Memory operations: Load/Store into memory or persistent storage (ld, str, ld.p, str.p)
• Jump operations: Conditional jumps with signed/unsigned comparisons
• I/O operations: In, Out, CLK, STS, DEV

note: persistent isn’t actually persistent as it also just lives in bram but ignore that tiny detail.

The Pipeline

The game’s CPU was simple: fetch, decode, execute, repeat. I figured if save breaker provided levels for a pipeline. What’s the worst that could happen?

Stage 1: Instruction Fetch
Stage 2: Decode + Operand Fetch
Stage 3: Execute + Writeback

Simple, right? Wrong. It works in theory but then hold timing kicks you in the face.

The Memory Wall

Here’s where things got ugly.

In Turing Complete, you have “magic memory”. You read it, you get the data instantly. In real hardware, BRAM has latency. The program memory needed to feed two instructions per cycle to the pipeline, and it absolutely refused to cooperate. And you know what? You also don’t have super wide 64-bit reads out of the box.

The solution involved splitting the program memory into four banks and interleaving instructions across them:

// Four banks, 16 bits each
// Bank 0: instructions 0, 4, 8...
// Bank 1: instructions 1, 5, 9...
// Bank 2: instructions 2, 6, 10...
// Bank 3: instructions 3, 7, 11...

This way, each bank only needed to read one 16‑bit value per cycle, which the BRAM could handle. The instruction fetch became a 64‑bit wide read.

The first off-by-one

If you started reading at address 1, you’d get the last chunk from line 0 and the first chunk from line 1. The alignment between byte addresses and word boundaries became a source of endless pain: the dreaded “read across a line” bug that cost me several evenings of debugging.

The solution solution? Read twice the amount of banks and offset the read address (mod 4) within the 128-wide read.

The assembler had to interleave the hex output across four separate .mem files because Verilog’s $readmemh doesn’t support “read every Nth line” natively because of course it doesn’t.

# just write to the correct bank file
for bank in range(4):
    for i in range(bank, len(instructions), 4):
        write_to_bank(bank, instructions[i])

The second off-by-one

And you know because hardware just has to hate you for no reason, turns out the synthesizer wired timing in such a way that memory expects every input a clock cycle early. Now I need to add extra wiring from pipelines, stall signals and, because why not, fixing more timing violations that follow out of that

The Pipeline Nightmare

The actual pipeline implementation is where things got… interesting.

The Hazard Problem

With a 2‑stage pipeline fetching two instructions at once, you need hazard detection. Here’s the compact version of the hazard logic (yes, this is production code):

assign exec_b = (inst_a[30:29]!=2'b10) &        // A isn't a jump
                (inst_b[30:29]==2'b01) &        // B is an ALU op
                (inst_a[23:20] != inst_b[19:16]) & // A's dest != B's src A
                (inst_b[28] || (inst_a[23:20] != inst_b[11:8])); // B immediate or dest mismatch

Translation: “Execute instruction B if there’s no data dependency with instruction A or structural violation.”

In my software-centric mind i assumed the original code (chained ternaries) would just become a mux. For me to then find out a month later that no, it really just makes a stupid long critical path.

The Delay Line Approach

Instead of traditional pipeline registers, I used a series of delay lines:

DelayLine #(.WORD_WIDTH(INSTRUCTION_BITS)) fetch_delay (
    .clk(clk), .rst(rst), .in(inst), .out(intermediate)
);

DelayLine #(.WORD_WIDTH(2)) delay_mode (
    .clk(clk), .rst(rst), .in(mode), .out(mode_exec)
);

Each DelayLine is literally just a register, it’s a one‑cycle delay that saves brain power when reasoning about timing. You just chain them to create pipeline stages without fighting with complex control logic. Simple, effective, and it made the waveforms much easier to read.

Did it work? Yes, after fixing the clock crossing issues and the reset synchronization problems. Did I want to pull my hair out? Also yes.

The Assembler Rabbit Hole

While debugging the hardware, I realized the assembler needed some love too.

Macros vs. Real Instructions

The game had no macro system, but I wanted one. So I added macro definitions (%macro … %endmacro) that expand into multiple real instructions:

hlt      ; Expands to: label: jmp label
mov dst, src  ; Expands to: add dst, zr, src

The macro preprocessor became multi-pass surprisingly quick:

class MacroPreprocessor(BaseProcessor):
    def register_macros(self):
        # Track macro definitions
        # Capture all instructions between %macro and %endmacro
        # Does not allow nested macros

    def resolve_macros(self):
        # iteratively expand macro references
        # until no more invocations are created
        # Preserve line numbers for debugging

Local Labels, A Necessary Evil

Because macros may introduce the same label multiple times (think of a hlt macro that creates a spin‑loop label), I needed local labels like 1f and 2b. These are resolved relative to the nearest occurrence, allowing macros to be used without polluting the global namespace or colliding with multiple invocations of the same macro.

loop:
    add r1, r1, 1
    cmp r1, 10
    jne loop
    jmp 1f  ; Jump forward to label '1'
    jmp 0   ; jump to address 0, never reached
1:
    hlt

The resolver searches backward or forward based on the direction suffix (b for backward, f for forward), picking the closest matching label. It’s a lifesaver when you’re generating code from macros.

The Section System

I added sections (.text, .data, .bss, .vector) and an .org directive to control memory layout. The organizer then stitched everything together… somewhat:

class Organizer(BaseProcessor):
    def collect_section_starts(self):
        # Track where each section starts
        # Handle implicit section starts (no .org)

    def reorganize_sections(self):
        # Sort section shards by address
        # Resolve.. err.. error overlapping ranges
        # Generate phantom .org directives for implicit starts

The Output Formats

The assembler supports three output formats:

• HEX: “Human‑readable” with spaces between words
• BIN: Raw binary
• EXE: Custom format with a 2‑byte header for length

The .exe format became necessary because the bootloader needs to know how much data to read from the flash.

The Toolchain Disaster

The assembler relies on Python and a bunch of modules. Here’s the compilation pipeline:

Parser -> MacroPreprocessor -> Organizer -> Resolver -> Assembler

Each stage transforms the instruction list and adds more information:

1. Parser: Parses source lines into instruction/directive objects
2. MacroPreprocessor: Expands macro definitions and references
3. Organizer: Lays out sections and resolves addresses
4. Resolver: Resolves labels and forward references
5. Assembler: Encodes instructions and writes output files

The Verilog Side of Things

The Clock Generator

The CPU clock is derived from a 100MHz external clock using a simple divider:

reg [1:0] clk_reg;
always @(posedge real_clk) begin
    clk_reg = clk_reg + 1;
end
assign clk = clk_reg[1];  // 50MHz using the high bit (00 - 01 - 10 - 11 -> 0 0 - 1 1)

Who cares about variables if you can just use bit magic instead.

The Boot Module… Not a Bootloader

The boot module holds the CPU in reset for three cycles:

reg [$clog2(PULSES)-1:0] reset_pulse;
always @(posedge clk) begin
    if (reset_pulse < PULSES) reset_pulse <= reset_pulse + 1;
    else rst <= 0;
end

This is not a bootloader. It’s just a power‑on reset generator that stabilizes everything before the clock actually hits the CPU. It ensures all registers and memory banks are properly initialized before the first instruction is fetched. The actual bootloader (which loads programs from external flash) came as one of the first three builds, because $readmemh is a simulation‑only construct. Why this module? Because uninitialized registers just love to say screw you multiple times over.

The BRAM Initialization

Here’s where the .mem files come in:

initial begin
    if (i == 0) $readmemh(`FULL_PATH(`ProgDir, ram_0.mem), ram);
    if (i == 1) $readmemh(`FULL_PATH(`ProgDir, ram_1.mem), ram);
    // ...
end

The ProgDir define is set at compile time, pointing to the output directory of the assembler. This means recompiling the program requires re‑running the assembler and rebuilding the Verilog project.

The lesson: Always leave a door open for software updates, write your bootloader sooner rather than later.

The “Works On My Machine” Myth

Spoiler: it didn’t.

I simulated the CPU with Icarus Verilog, using a simple test program that counted from 0 to 255 and output the results over UART. Everything looked perfect.

Then I synthesized it for the actual FPGA (a Basys 3 board).

Timing violations everywhere.

One of the largest critical paths (or at least part of it) was the ALU, specifically the multiplier. It wasn’t pipelined because that would have caused even more hazards to handle.

wire [WORD_WIDTH-1:0] multiply_wire;
assign multiply_wire = arg_a[7:0] * arg_b[7:0];

The 8×8‑bit multiply was timing out at 50MHz. The fix? Add a (* use_dsp = "yes" *) attribute to force the synthesizer to use the dedicated multiplier blocks:

(* mult_style = "pipe_none", use_dsp = "yes" *)
wire [WORD_WIDTH-1:0] multiply_wire;

And reduce the input width to 8 bits because the full 16×16 multiply would be even worse. Why 8 bits? We’re discarding the upper 16 bits anyway, so why waste the nanoseconds?

What else? When timings where finally resolved and the waveforms looked sane, I ran a test program to just echo UART in back to the terminal (… without just tying rx to tx). Turns out IO wasn’t properly being stalled when a jump occurs before an IO instruction

.label target
...
jeq target
in  ; this one would still partially execute when the jump was taken
nop ; IO/memory instructions here as well

Well that was a fun (sic) weekend of lost sanity.

The Bugs That Made Me Question My Life Choices

The Stall Signal Bug

The busy module was generating a stall signal that held the CPU in reset for two cycles after every jump. Some parts of the cpu expected this ready signal to be available a cycle sooner.

The fix? Add a ready_early signal that’s one cycle ahead of the normal ready signal.

This particular bug was a nightmare: after a jmp, the next instruction (often a memory or IO operation) would run despite no clear indication that it should have. The jump is taken, the next instruction executes anyway, and the cpu continues from the jump address.

The Register Write Trick

The register file writes on the negative edge of the clock, while everything else reads on the positive edge. This gave us half a cycle for the write to propagate before the next read. How this somehow worked (even with a bypass mux for reading the register in a next instruction)? I don’t know, i just know it does… somehow.

always @(negedge clk) begin
    if (write_enable_1 && write_address_1 != 0) begin
        storage[write_address_1] <= write_data_1;
    end
end

What I Learned

1. Video game abstractions are dangerously accurate: Turing Complete’s CPU model is functionally correct, but the timing assumptions are completely different from real hardware.

2. Memory is always the bottleneck: Every performance problem in this project came back to the BRAM access time and the alignment nightmares.

3. The toolchain is as important as the hardware: Without good debug output and error messages, you’ll be lost in a sea of zeros and ones.

4. Don’t design your ISA without considering how it will be implemented: The 4‑bit register addressing is fine, but the 16‑bit immediate value means the instruction format is cramped. A 32‑bit immediate would have been more useful, but then we’d need an awkward 48‑bit instruction.

5. Off‑by‑one errors in pipelined designs are the worst kind of bugs: They’re hard to spot in simulation and even harder to debug on hardware.

The Future (If I Ever Touch This Again)

If I were to do this over again (and I won’t, I promise), I would:

1. Use a 3‑stage pipeline with proper hazard forwarding
2. Add an instruction cache to reduce BRAM pressure
3. Use a 16×16 multiplier and figure out the timing constraints later
4. Build a compiler because programming assembly is a pain.
5. Add proper debugging support (JTAG or similar)

But honestly? It works. The CPU boots, executes programs, and talks to the outside world via UART. For a project that started as “let’s port this game to hardware,” I’d call that a success.

The Final Code

The full source code is available. The current program being tested is an echo server that reads characters over UART and echoes them back.

Here’s a snippet of the assembly:

.text
.org 0x0000

main:
    mov r1, 0x1000  ; Initialize pointer
    mov r2, 0       ; Counter

loop:
    in r3            ; Read UART
    cmp r3, 0
    jeq loop         ; Wait for character

    out r3           ; Echo it back
    str.p r3, [r1]   ; Store with post-increment
    add r2, r2, 1
    cmp r2, 256
    jne loop

    hlt

The CPU also has a clk instruction that reads the current program counter, useful for function calls:

clk r4     ; Read current PC into r4

Acknowledgments

This project wouldn’t have been possible without:

• The Turing Complete game for making me believe this was a good idea
• The open‑source Verilog community for countless examples
• My cup of coffee, who provided moral support by sitting on the desk during critical moments
• The Vivado timing analyser, which gave me more grey hairs than I already had

Closing Thoughts

What started as a fun video game project turned into a Verilog nightmare, a custom assembler with macro support, and an FPGA implementation that actually runs code. It’s not pretty, it’s not fast, and it definitely won’t win any awards for clean design.

But it works. And that’s more than I can say for some of my projects over the years.

If you’re thinking of building your own CPU: Do it. Just don’t expect it to be easy, and definitely don’t expect it to work on the first try. Or the tenth. And prepare for off‑by‑one errors that will make you question your sanity.

This post is dedicated to everyone who spent hours debugging after thinking “how hard could it be”.