Self-Modifying Code

Every time a Commodore 64 runs a line of BASIC, it runs a program that rewrites itself. The routine that fetches each character of the BASIC program, called CHRGET, keeps its place in the program as the address inside one of its own instructions, and moves on by adding one to that address. Jim Butterfield, explaining it in his 1986 book on Commodore machine language, put it plainly: “This subroutine is self-modifying, that is, it changes part of itself as it runs. That’s not always a good programming technique, but it works well here.”

Self-modifying code treats a program’s instructions as data. A program writes a new address, a new value or even a new instruction into its own code, and the processor runs the changed version the next time it gets there. On the 8-bit machines of the 1980s it was one of the most common ways to make a routine faster and smaller. Richard Mansfield’s The Second Book of Machine Language (1984) passed on his COMPUTE! colleague Charles Brannon’s checklist for “turbo-charged code”, which asks: “Should an Indirect Y addressing mode be replaced by an even faster structure such as self-modifying Absolute addressing?”

Why it is faster on the 6502

The 6502 has no register that can hold a 16-bit address. To read from an address that changes, a program normally keeps the address in two bytes of zero page and uses indirect indexed addressing, written lda (pointer),y. Self-modifying code writes the address straight into an ordinary instruction instead:

; Copy a page of 256 bytes to the screen at $0400.
; A holds the high byte of the source address.
copy_page
        sta src+2        ; write the page number into the instruction below
        ldy #0
src     lda $ff00,y      ; $ff00 is a placeholder: its high byte is rewritten
        sta $0400,y
        iny
        bne src
        rts

lda $ff00,y assembles to three bytes: the opcode, then the low and high bytes of the address. src+2 is the high byte, so the sta at the top changes which page the loop reads.

MOS Technology’s programming manual gives the saving. lda (pointer),y takes 5 cycles and lda address,y takes 4, each with one more if the read crosses a page; for stores, sta (pointer),y takes 6 and sta address,y takes 5. A copy loop that does both saves two cycles a byte, which is what Bruce Clark’s memory-move routines on 6502.org promise: replacing “the (ZeroPage),Y addressing mode with the Absolute,Y addressing mode … will take 2 fewer cycles per byte”. It also frees the zero-page bytes, which are scarce on a machine whose BASIC and operating system use most of them.

CHRGET uses the same idea for a different reason. Butterfield sets out its first four instructions:

CHRGET  INC CHRGOT+1     ; add one to the low byte of the address
        BNE CHRGOT
        INC CHRGOT+2     ; and carry into the high byte
CHRGOT  LDA xxxx         ; the address is the pointer into the BASIC program

On the C64 the routine sits in zero page at $0073–$008A, and Commodore’s Programmer’s Reference Guide lists the pointer TXTPTR at $007A–$007B, which are the two address bytes of the LDA at CHRGOT, $0079. Butterfield notes that the VIC-20 has it at the same address, and “most PET/CBM computers” a few bytes lower in zero page. Because every character of a BASIC program passes through it, the Programmer’s Reference Guide suggests replacing it to add new commands to BASIC: “This technique is often called a wedge.”

Common patterns

Saving a value inside an instruction. A routine that has to put something back later can store it in the instruction that will restore it. Marko Mäkelä’s stable-raster routine on the Codebase64 wiki keeps the address of the interrupt handler it replaced as the operand of a jmp:

oldirq = * + 1  ; Placeholder for self-modifying code
        jmp *   ; Return to the original interrupt

A computed delay or branch. The Codebase64 example for AGSP, which must make a write on a cycle that depends on the scroll position, calculates how many NOP instructions to skip and stores the number as the offset of a branch instruction that jumps into a row of them.

Changing the instruction itself. On the NES, the processor’s interrupt vectors are in the cartridge ROM. The NESdev wiki shows how a game can still switch between several handlers for the vertical-blank interrupt: the vector points at three bytes of RAM, and the program writes JMP and an address there. While it changes the address, it first writes the opcode for RTI (return from interrupt), so that an interrupt arriving halfway through the change returns at once instead of jumping somewhere half-written.

Values inside an unrolled loop. Writers on C64 programming in the 1990s, many of them in the electronic magazine C= Hacking, described how they used it:

On the Z80 and the Spectrum

Spectrum programmers used the same tricks. In January 1989 Sinclair User answered a reader’s question about the Z80’s stack with a fast way to clear the screen. The usual method, LDIR, clears a byte at a time; pointing the stack at the end of the display file and pushing a register pair holding zero clears two bytes for each PUSH. The routine has to put the stack pointer back afterwards, and it keeps the old value in the operand of its own last LD SP instruction, “via self modifying code”:

        di                  ; interrupts use the stack, so keep them out
        ld (restore+1),sp   ; save SP as the operand of the LD SP below
        ld sp,22528         ; the end of the 6,144-byte display file
        ld de,0
        ld bc,3072          ; 3,072 pushes of two bytes
loop    push de
        dec bc
        ld a,b
        or c
        jp nz,loop
restore ld sp,0             ; the 0 has been overwritten with the old SP
        ei
        ret

Game programmers mentioned it when they described their work. Mike Follin of Software Creations, writing in Sinclair User in June 1987 about converting Geoff Crammond’s The Sentinel to the Spectrum, explained that “using an 8K buffer, some nice self-modifying code, and lots of LDI instructions” produced its vertical scrolling. John Mullins, converting Power Drift in 1989, needed “a good deal of self-modifying code” to stop the program losing track when the road climbed off the top of the screen. His solution, he told the magazine, “would cause a purist to have a heart attack if he saw the code, but it works and it works fast, which is the main thing!”

Where it does not work

In ROM. A program can only rewrite code held in RAM. That is why the NES example above keeps its three-byte jump in RAM, and why CHRGET, part of a BASIC that otherwise lives in ROM, sits in zero-page RAM.

With an instruction cache. The 68020 keeps recently run instructions in a cache on the chip. Stan Kelly-Bootle and Bob Fowler’s 68000, 68010, 68020 Primer (1985) warned that “if a self-modifying program is cached, wrong results may occur”: the processor goes on running the old copy from its cache after the program has changed the one in memory. Their advice was “to avoid self-modifying programs”, or to turn the cache off while such a routine runs. This caught out Amiga software when faster processors arrived. Writing about the new Amiga 3000 in July 1990, Amiga Computing noted that its 68030 could not be switched off, so programs that broke “a ban on self-modifying code” would fail: “a lot of disk protection systems rely on this, and so won’t work on the A3000.”

When code is translated. A program converted to another processor ahead of time no longer contains the instructions it expects to rewrite. Reviewing a tool that translated Z80 programs into 68000 code for the Atari ST in January 1989, ST Amiga Format listed self-modifying code first among the things “the Translator simply can’t, or at least doesn’t, translate”. Emulators that compile a guest program’s code face the same problem today, and have to notice when the program writes to code they have already translated.

See also

• 6502
• Lookup tables
• Sprite multiplexing
• Raster interrupts
• AGSP