Skip to content

Clip a Rectangle to a Viewport on NES

A partly off-screen object should expose only the part that lies inside the viewport.

A partly off-screen object should expose only the part that lies inside the viewport.

NES shows the working diagnostic row 05 03 0A 0A 01 01 02 03.
A completed native caller displays 05 03 0A 0A 01 01 02 03. The public field order below identifies these bytes; the capture is a state diagnostic, not a recording of a finished game.

Use it in a project

Both rectangles use unsigned byte coordinates and half-open edges [left,right)×[top,bottom). Require ordered edges; equal edges are valid empty rectangles. Return the intersection and VISIBLE=1 if it has area. Empty results are all zero. Invalid reversed edges return VALID=0.

Include component.inc once (MOS callers include the shared 6502 implementation through it). Reserve the public fields in this exact order: OUT_LEFT, OUT_TOP, OUT_RIGHT, OUT_BOTTOM, VISIBLE, VALID, LEFT, TOP, RIGHT, BOTTOM, VIEW_LEFT, VIEW_TOP, VIEW_RIGHT, VIEW_BOTTOM. They occupy 14 contiguous bytes. Also reserve the private workspace: T0, T1, T2, T3 (4 bytes). Array names in the explanation refer to the numbered consecutive bindings, such as C0, C1 and so on.

Call rectangle_clipping_init once at the intended reset boundary. It clears inputs as well as outputs and workspace. Set your starting state and configuration afterwards, then call rectangle_clipping_step whenever the described event occurs. The helper reads no device, waits for no frame and installs no interrupt handler.

Ownership Fields
Retain between calls none
Supply before service both sets of rectangle edges
Read after service intersection edges, VISIBLE and VALID

The intersection starts at the greater left/top edges and ends at the lesser right/bottom edges. Strictly positive width and height distinguish a visible area from touching edges.

Reusable initialisation and service

6502.inc
; SPDX-License-Identifier: MIT
; Clip a Rectangle to a Viewport. See README.md for the byte interface and ownership.
; init clears the public fields and private workspace; set inputs afterwards.
rectangle_clipping_init:
    lda #0
    sta OUT_LEFT
    sta OUT_TOP
    sta OUT_RIGHT
    sta OUT_BOTTOM
    sta VISIBLE
    sta VALID
    sta LEFT
    sta TOP
    sta RIGHT
    sta BOTTOM
    sta VIEW_LEFT
    sta VIEW_TOP
    sta VIEW_RIGHT
    sta VIEW_BOTTOM
    sta T0
    sta T1
    sta T2
    sta T3
    rts
rectangle_clipping_step:
; OUT_LEFT = 0
    lda #0
    sta OUT_LEFT
; OUT_TOP = 0
    lda #0
    sta OUT_TOP
; OUT_RIGHT = 0
    lda #0
    sta OUT_RIGHT
; OUT_BOTTOM = 0
    lda #0
    sta OUT_BOTTOM
; VISIBLE = 0
    lda #0
    sta VISIBLE
; VALID = 0
    lda #0
    sta VALID
; if LEFT > RIGHT:
    lda LEFT
    cmp RIGHT
    beq rectangle_clipping_L5
    bcs rectangle_clipping_L4
rectangle_clipping_L5:
    jmp rectangle_clipping_L2
rectangle_clipping_L4:
    jmp rectangle_clipping_L1
rectangle_clipping_L1:
; return
    rts
    jmp rectangle_clipping_L3
rectangle_clipping_L2:
rectangle_clipping_L3:
; if TOP > BOTTOM:
    lda TOP
    cmp BOTTOM
    beq rectangle_clipping_L10
    bcs rectangle_clipping_L9
rectangle_clipping_L10:
    jmp rectangle_clipping_L7
rectangle_clipping_L9:
    jmp rectangle_clipping_L6
rectangle_clipping_L6:
; return
    rts
    jmp rectangle_clipping_L8
rectangle_clipping_L7:
rectangle_clipping_L8:
; if VIEW_LEFT > VIEW_RIGHT:
    lda VIEW_LEFT
    cmp VIEW_RIGHT
    beq rectangle_clipping_L15
    bcs rectangle_clipping_L14
rectangle_clipping_L15:
    jmp rectangle_clipping_L12
rectangle_clipping_L14:
    jmp rectangle_clipping_L11
rectangle_clipping_L11:
; return
    rts
    jmp rectangle_clipping_L13
rectangle_clipping_L12:
rectangle_clipping_L13:
; if VIEW_TOP > VIEW_BOTTOM:
    lda VIEW_TOP
    cmp VIEW_BOTTOM
    beq rectangle_clipping_L20
    bcs rectangle_clipping_L19
rectangle_clipping_L20:
    jmp rectangle_clipping_L17
rectangle_clipping_L19:
    jmp rectangle_clipping_L16
rectangle_clipping_L16:
; return
    rts
    jmp rectangle_clipping_L18
rectangle_clipping_L17:
rectangle_clipping_L18:
; VALID = 1
    lda #1
    sta VALID
; T0 = LEFT
    lda LEFT
    sta T0
; if VIEW_LEFT > T0:
    lda VIEW_LEFT
    cmp T0
    beq rectangle_clipping_L25
    bcs rectangle_clipping_L24
rectangle_clipping_L25:
    jmp rectangle_clipping_L22
rectangle_clipping_L24:
    jmp rectangle_clipping_L21
rectangle_clipping_L21:
; T0 = VIEW_LEFT
    lda VIEW_LEFT
    sta T0
    jmp rectangle_clipping_L23
rectangle_clipping_L22:
rectangle_clipping_L23:
; T1 = TOP
    lda TOP
    sta T1
; if VIEW_TOP > T1:
    lda VIEW_TOP
    cmp T1
    beq rectangle_clipping_L30
    bcs rectangle_clipping_L29
rectangle_clipping_L30:
    jmp rectangle_clipping_L27
rectangle_clipping_L29:
    jmp rectangle_clipping_L26
rectangle_clipping_L26:
; T1 = VIEW_TOP
    lda VIEW_TOP
    sta T1
    jmp rectangle_clipping_L28
rectangle_clipping_L27:
rectangle_clipping_L28:
; T2 = RIGHT
    lda RIGHT
    sta T2
; if VIEW_RIGHT < T2:
    lda VIEW_RIGHT
    cmp T2
    bcc rectangle_clipping_L34
rectangle_clipping_L35:
    jmp rectangle_clipping_L32
rectangle_clipping_L34:
    jmp rectangle_clipping_L31
rectangle_clipping_L31:
; T2 = VIEW_RIGHT
    lda VIEW_RIGHT
    sta T2
    jmp rectangle_clipping_L33
rectangle_clipping_L32:
rectangle_clipping_L33:
; T3 = BOTTOM
    lda BOTTOM
    sta T3
; if VIEW_BOTTOM < T3:
    lda VIEW_BOTTOM
    cmp T3
    bcc rectangle_clipping_L39
rectangle_clipping_L40:
    jmp rectangle_clipping_L37
rectangle_clipping_L39:
    jmp rectangle_clipping_L36
rectangle_clipping_L36:
; T3 = VIEW_BOTTOM
    lda VIEW_BOTTOM
    sta T3
    jmp rectangle_clipping_L38
rectangle_clipping_L37:
rectangle_clipping_L38:
; if T0 >= T2:
    lda T0
    cmp T2
    bcs rectangle_clipping_L44
rectangle_clipping_L45:
    jmp rectangle_clipping_L42
rectangle_clipping_L44:
    jmp rectangle_clipping_L41
rectangle_clipping_L41:
; return
    rts
    jmp rectangle_clipping_L43
rectangle_clipping_L42:
rectangle_clipping_L43:
; if T1 >= T3:
    lda T1
    cmp T3
    bcs rectangle_clipping_L49
rectangle_clipping_L50:
    jmp rectangle_clipping_L47
rectangle_clipping_L49:
    jmp rectangle_clipping_L46
rectangle_clipping_L46:
; return
    rts
    jmp rectangle_clipping_L48
rectangle_clipping_L47:
rectangle_clipping_L48:
; OUT_LEFT = T0
    lda T0
    sta OUT_LEFT
; OUT_TOP = T1
    lda T1
    sta OUT_TOP
; OUT_RIGHT = T2
    lda T2
    sta OUT_RIGHT
; OUT_BOTTOM = T3
    lda T3
    sta OUT_BOTTOM
; VISIBLE = 1
    lda #1
    sta VISIBLE
    rts
component_end:

Machine contract

A/X/status may change; Y is preserved. The 2A03 has no decimal arithmetic mode. A direct JSR needs two return-address stack bytes. The helper makes no nested calls and adds no other stack use. Its named scratch is private workspace, not a pointer, a ROM variable or shared zero-page space that the game may borrow concurrently. The samples use fixed symbol bindings. Scope or rename every label and binding, including the component_end measurement marker, and allocate a separate block for each independent instance. Different helpers can also use common field names, so combining them needs separate namespaces or explicitly renamed bindings. Give each block one mutation owner.

The NROM caller owns PPU setup and background display, with NMI and DMC disabled. Table and packet sizes are bounded to fit the cartridge caller and internal RAM; the helper owns no PPU registers.

Adapt the caller

demo.asm supplies the bindings and include order. exercise.inc first runs table cases, then a separate client retaining state between calls. A game can call the public step entry directly and remove the diagnostic packet and display. Do not copy the diagnostic’s hardware takeover into an existing game loop.

Build with make ASM198X=/path/to/asm198x. The Makefile produces normal, custom, deliberately wrong and repaired programs. Tools must already be installed; the sample downloads nothing.

Bindings and include order

demo.asm
.segment "HEADER"
.byte "NES",$1a,2,1,0,0,0,0,0,0,0,0,0,0
OUT_LEFT=$80
OUT_TOP=$81
OUT_RIGHT=$82
OUT_BOTTOM=$83
VISIBLE=$84
VALID=$85
LEFT=$86
TOP=$87
RIGHT=$88
BOTTOM=$89
VIEW_LEFT=$8a
VIEW_TOP=$8b
VIEW_RIGHT=$8c
VIEW_BOTTOM=$8d
T0=$8e
T1=$8f
T2=$90
T3=$91
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$0200
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=0
BROKEN=0
COYOTE_STEPS=4
BUFFER_STEPS=4
TEXT_CAP=8
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=7
.include "caller.inc"
.include "exercise.inc"
component_start:
.include "component.inc"
.include "data.inc"
.include "cases.inc"
code_end:
.segment "VECTORS"
.word nmi,reset,irq
.segment "CHARS"
.include "../../input/controller-reading/edges-font.inc"
.res 8192-272,0

A client retaining state between calls

exercise.inc, excerpt
preview_rules:
.if CUSTOM
; LEFT = 0
    lda #0
    sta LEFT
; TOP = 0
    lda #0
    sta TOP
; RIGHT = 255
    lda #255
    sta RIGHT
; BOTTOM = 255
    lda #255
    sta BOTTOM
; VIEW_LEFT = 10
    lda #10
    sta VIEW_LEFT
; VIEW_TOP = 20
    lda #20
    sta VIEW_TOP
; VIEW_RIGHT = 30
    lda #30
    sta VIEW_RIGHT
; VIEW_BOTTOM = 40
    lda #40
    sta VIEW_BOTTOM
.if BROKEN
; VIEW_RIGHT = 255
    lda #255
    sta VIEW_RIGHT
.endif
    jsr checked_step
    jsr emit_state
    rts
.else
; LEFT = 2
    lda #2
    sta LEFT
; TOP = 3
    lda #3
    sta TOP
; RIGHT = 12
    lda #12
    sta RIGHT
; BOTTOM = 15
    lda #15
    sta BOTTOM
; VIEW_LEFT = 5
    lda #5
    sta VIEW_LEFT
; VIEW_TOP = 0
    lda #0
    sta VIEW_TOP
; VIEW_RIGHT = 10
    lda #10
    sta VIEW_RIGHT
; VIEW_BOTTOM = 10
    lda #10
    sta VIEW_BOTTOM
.if BROKEN
; VIEW_RIGHT = 255
    lda #255
    sta VIEW_RIGHT
.endif
    jsr checked_step
    jsr emit_state
.endif
    rts

Customise it

Clip a maximum-size rectangle into a small inset viewport. The maintained custom caller changes its inputs or client sequence rather than hiding a different implementation behind the same label. Compare every state of the two clients, not only their final display.

Custom client selection

custom.asm
.segment "HEADER"
.byte "NES",$1a,2,1,0,0,0,0,0,0,0,0,0,0
OUT_LEFT=$80
OUT_TOP=$81
OUT_RIGHT=$82
OUT_BOTTOM=$83
VISIBLE=$84
VALID=$85
LEFT=$86
TOP=$87
RIGHT=$88
BOTTOM=$89
VIEW_LEFT=$8a
VIEW_TOP=$8b
VIEW_RIGHT=$8c
VIEW_BOTTOM=$8d
T0=$8e
T1=$8f
T2=$90
T3=$91
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$0200
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=1
BROKEN=0
COYOTE_STEPS=2
BUFFER_STEPS=2
TEXT_CAP=4
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=49
.include "caller.inc"
.include "exercise.inc"
component_start:
.include "component.inc"
.include "data.inc"
.include "cases.inc"
code_end:
.segment "VECTORS"
.word nmi,reset,irq
.segment "CHARS"
.include "../../input/controller-reading/edges-font.inc"
.res 8192-272,0

The custom diagnostic row is 0A 14 1E 28 01 01 00 00. The full client trace in the source explains how it gets there.

Mistake and repair

The wrong caller replaces the viewport’s right edge with the coordinate ceiling. The clipped object leaks past the intended panel. Supply the actual viewport bounds.

The BROKEN branch lives in the sequential client, not the reusable helper. wrong.asm selects it; repaired.asm restores the normal call sequence. The repaired executable matches the normal program (for Spectrum, compare loaded CODE rather than the tape filename).

Select the deliberately wrong client

wrong.asm
.segment "HEADER"
.byte "NES",$1a,2,1,0,0,0,0,0,0,0,0,0,0
OUT_LEFT=$80
OUT_TOP=$81
OUT_RIGHT=$82
OUT_BOTTOM=$83
VISIBLE=$84
VALID=$85
LEFT=$86
TOP=$87
RIGHT=$88
BOTTOM=$89
VIEW_LEFT=$8a
VIEW_TOP=$8b
VIEW_RIGHT=$8c
VIEW_BOTTOM=$8d
T0=$8e
T1=$8f
T2=$90
T3=$91
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$0200
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=0
BROKEN=1
COYOTE_STEPS=4
BUFFER_STEPS=4
TEXT_CAP=8
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=7
.include "caller.inc"
.include "exercise.inc"
component_start:
.include "component.inc"
.include "data.inc"
.include "cases.inc"
code_end:
.segment "VECTORS"
.word nmi,reset,irq
.segment "CHARS"
.include "../../input/controller-reading/edges-font.inc"
.res 8192-272,0
NES shows the deliberate error row 05 03 0C 0A 01 01 02 03.
The wrong client completes with 05 03 0C 0A 01 01 02 03; its changed inputs and state explain the result.

Cost and alternatives

The measured helper extent is 288 code bytes; public state is 14 bytes and private scratch is 4 bytes. There are no immutable helper tables. Caller data, the test cases, return-address stack, alignment and display buffers cost additional space. These are assembled sizes, not cycle measurements.

Constant work. Clip before entering expensive pixel loops; complex shapes still require their own clipping rules.

This returns geometry, not a masked draw. Coordinates cannot represent an exclusive edge of 256; use a wider type for a full 256-pixel span.

Execution evidence

Four variants run on NTSC NROM, background only, DMC/NMI disabled, using Asm198x 0.0.72, Emu198x 0.44.2. Each executes 64 boundary/varied/error table cases, dirty initialisation and the separate retained-state client. An independent mathematical/container model checks every public byte after every call. This is bounded example coverage, not exhaustive correctness for every possible state.

The records retain loaded immutable code, completion, register sentinels, full packets, source/tool/firmware hashes and captures. The public gate validates these retained observations and rejects altered evidence; it does not run restricted firmware. No firmware is distributed. The eight displayed bytes are the first eight public fields, zero-padded when shorter; the packet includes every field.

Where it fits

Rectangle overlap: Half-open edge conventions must agree between collision and drawing.

The Craft · Maths for Games · Source and native evidence.