Filter Collision Pairs by Layer on C64

A projectile can cross its owner without hitting them, while still hitting an opponent in the same patch of screen.

Use it in a project

GROUP_A/B are eight-bit membership sets; MASK_A/B say which groups each object accepts. Both directions must accept the pair. Return PAIR_KIND=0 when either rejects it, 1 for an accepted sensor pair, or 2 when both accepted objects have nonzero SOLID flags. Preserve all inputs. This classifies policy; a separate spatial test must establish actual contact.

Include component.inc once (MOS callers include the shared 6502 implementation through it). Reserve the public fields in this exact order: PAIR_KIND, GROUP_A, MASK_A, GROUP_B, MASK_B, SOLID_A, SOLID_B. They occupy 7 contiguous bytes. Also reserve the private workspace: T0 (1 bytes). Array names in the explanation refer to the numbered consecutive bindings, such as C0, C1 and so on.

Call collision_layer_filter_init once at the intended reset boundary. It clears inputs as well as outputs and workspace. Set your starting state and configuration afterwards, then call collision_layer_filter_step whenever the described event occurs. The helper reads no device, waits for no frame and installs no interrupt handler.

Intersect each object’s membership with the other’s acceptance mask. Requiring both intersections avoids one-sided friendly-fire rules. Solidity is considered only after acceptance.

Reusable initialisation and service

; SPDX-License-Identifier: MIT
; Filter Collision Pairs by Layer. See README.md for the byte interface and ownership.
; init clears the public fields and private workspace; set inputs afterwards.
collision_layer_filter_init:
    lda #0
    sta PAIR_KIND
    sta GROUP_A
    sta MASK_A
    sta GROUP_B
    sta MASK_B
    sta SOLID_A
    sta SOLID_B
    sta T0
    rts
collision_layer_filter_step:
; PAIR_KIND = 0
    lda #0
    sta PAIR_KIND
; T0 = GROUP_A & MASK_B
    lda GROUP_A
    and MASK_B
    sta T0
; if T0 == 0:
    lda T0
    cmp #0
    beq collision_layer_filter_L4
collision_layer_filter_L5:
    jmp collision_layer_filter_L2
collision_layer_filter_L4:
    jmp collision_layer_filter_L1
collision_layer_filter_L1:
; return
    rts
    jmp collision_layer_filter_L3
collision_layer_filter_L2:
collision_layer_filter_L3:
; T0 = GROUP_B & MASK_A
    lda GROUP_B
    and MASK_A
    sta T0
; if T0 == 0:
    lda T0
    cmp #0
    beq collision_layer_filter_L9
collision_layer_filter_L10:
    jmp collision_layer_filter_L7
collision_layer_filter_L9:
    jmp collision_layer_filter_L6
collision_layer_filter_L6:
; return
    rts
    jmp collision_layer_filter_L8
collision_layer_filter_L7:
collision_layer_filter_L8:
; PAIR_KIND = 1
    lda #1
    sta PAIR_KIND
; if SOLID_A != 0:
    lda SOLID_A
    cmp #0
    bne collision_layer_filter_L14
collision_layer_filter_L15:
    jmp collision_layer_filter_L12
collision_layer_filter_L14:
    jmp collision_layer_filter_L11
collision_layer_filter_L11:
; if SOLID_B != 0:
    lda SOLID_B
    cmp #0
    bne collision_layer_filter_L19
collision_layer_filter_L20:
    jmp collision_layer_filter_L17
collision_layer_filter_L19:
    jmp collision_layer_filter_L16
collision_layer_filter_L16:
; PAIR_KIND = 2
    lda #2
    sta PAIR_KIND
    jmp collision_layer_filter_L18
collision_layer_filter_L17:
collision_layer_filter_L18:
    jmp collision_layer_filter_L13
collision_layer_filter_L12:
collision_layer_filter_L13:
    rts
component_end:

Machine contract

A/X/status may change; Y and decimal mode are preserved. Use CLD for binary arithmetic. A direct JSR needs two return-address stack bytes. The helper makes no nested calls and adds no other stack use. Its named scratch is private workspace, not a pointer, a ROM variable or shared zero-page space that the game may borrow concurrently. The samples use fixed symbol bindings. Scope or rename every label and binding, including the component_end measurement marker, and allocate a separate block for each independent instance. Different helpers can also use common field names, so combining them needs separate namespaces or explicitly renamed bindings. Give each block one mutation owner.

The diagnostic PRG owns screen, font and colour RAM and disables interrupts; it does not return to BASIC. The reusable helper does none of those things.

Adapt the caller

demo.asm supplies the bindings and include order. exercise.inc first runs table cases, then a separate client retaining state between calls. A game can call the public step entry directly and remove the diagnostic packet and display. Do not copy the diagnostic’s hardware takeover into an existing game loop.

Build with make ASM198X=/path/to/asm198x. The Makefile produces normal, custom, deliberately wrong and repaired programs. Tools must already be installed; the sample downloads nothing.

Bindings and include order

PAIR_KIND=$80
GROUP_A=$81
MASK_A=$82
GROUP_B=$83
MASK_B=$84
SOLID_A=$85
SOLID_B=$86
T0=$87
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$c000
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=0
BROKEN=0
COYOTE_STEPS=4
BUFFER_STEPS=4
TEXT_CAP=8
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=7
!source "caller.inc"
!source "exercise.inc"
component_start:
!source "component.inc"
!source "data.inc"
!source "cases.inc"
hex_font:
    !byte 0,0,0,0,0,0,0,0,0,56,40,40,40,56,0,0,0,16,48,16,16,56,0,0,0,56,8,56,32,56,0,0,0,56,8,56,8,56,0,0,0,40,40,56,8,8,0,0,0,56,32,56,8,56,0,0,0,56,32,56,40,56,0,0,0,56,8,8,8,8,0,0,0,56,40,56,40,56,0,0,0,56,40,56,8,56,0,0,0,16,40,56,40,40,0,0,0,48,40,48,40,48,0,0,0,56,32,32,32,56,0,0,0,48,40,40,40,48,0,0,0,56,32,48,32,56,0,0,0,56,32,48,32,32,0,0
code_end:

A client retaining state between calls

preview_rules:
!if CUSTOM {
; GROUP_A = 1
    lda #1
    sta GROUP_A
; MASK_A = 2
    lda #2
    sta MASK_A
; GROUP_B = 2
    lda #2
    sta GROUP_B
; MASK_B = 1
    lda #1
    sta MASK_B
; SOLID_A = 1
    lda #1
    sta SOLID_A
; SOLID_B = 0
    lda #0
    sta SOLID_B
!if BROKEN {
; MASK_A = 255
    lda #255
    sta MASK_A
}
    jsr checked_step
    jsr emit_state
    rts
} else {
; GROUP_A = 1
    lda #1
    sta GROUP_A
; MASK_A = 2
    lda #2
    sta MASK_A
; GROUP_B = 4
    lda #4
    sta GROUP_B
; MASK_B = 1
    lda #1
    sta MASK_B
; SOLID_A = 1
    lda #1
    sta SOLID_A
; SOLID_B = 1
    lda #1
    sta SOLID_B
!if BROKEN {
; MASK_A = 255
    lda #255
    sta MASK_A
}
    jsr checked_step
    jsr emit_state
}
    rts

Customise it

Make an accepted pair a sensor by clearing one solidity flag. The maintained custom caller changes its inputs or client sequence rather than hiding a different implementation behind the same label. Compare every state of the two clients, not only their final display.

Custom client selection

PAIR_KIND=$80
GROUP_A=$81
MASK_A=$82
GROUP_B=$83
MASK_B=$84
SOLID_A=$85
SOLID_B=$86
T0=$87
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$c000
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=1
BROKEN=0
COYOTE_STEPS=2
BUFFER_STEPS=2
TEXT_CAP=4
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=49
!source "caller.inc"
!source "exercise.inc"
component_start:
!source "component.inc"
!source "data.inc"
!source "cases.inc"
hex_font:
    !byte 0,0,0,0,0,0,0,0,0,56,40,40,40,56,0,0,0,16,48,16,16,56,0,0,0,56,8,56,32,56,0,0,0,56,8,56,8,56,0,0,0,40,40,56,8,8,0,0,0,56,32,56,8,56,0,0,0,56,32,56,40,56,0,0,0,56,8,8,8,8,0,0,0,56,40,56,40,56,0,0,0,56,40,56,8,56,0,0,0,16,40,56,40,40,0,0,0,48,40,48,40,48,0,0,0,56,32,32,32,56,0,0,0,48,40,40,40,48,0,0,0,56,32,48,32,56,0,0,0,56,32,48,32,32,0,0
code_end:

The custom diagnostic row is 01 01 02 02 01 01 00 00. The full client trace in the source explains how it gets there.

Mistake and repair

The wrong caller accepts every group for A. A pair excluded by MASK_A=2 becomes a solid candidate. Preserve the intended mask; overlapping objects do not automatically have permission to interact.

The BROKEN branch lives in the sequential client, not the reusable helper. wrong.asm selects it; repaired.asm restores the normal call sequence. The repaired executable matches the normal program (for Spectrum, compare loaded CODE rather than the tape filename).

Select the deliberately wrong client

PAIR_KIND=$80
GROUP_A=$81
MASK_A=$82
GROUP_B=$83
MASK_B=$84
SOLID_A=$85
SOLID_B=$86
T0=$87
STATE=$80
input_ptr=$70
cases_left=$72
ptr=$74
saved_y=$76
OUTPUT=$c000
preview=OUTPUT+1400
ready=OUTPUT+1408
CUSTOM=0
BROKEN=1
COYOTE_STEPS=4
BUFFER_STEPS=4
TEXT_CAP=8
PROPERTY_COUNT=16
STREAM_BYTES=13
CRC_POLY=7
!source "caller.inc"
!source "exercise.inc"
component_start:
!source "component.inc"
!source "data.inc"
!source "cases.inc"
hex_font:
    !byte 0,0,0,0,0,0,0,0,0,56,40,40,40,56,0,0,0,16,48,16,16,56,0,0,0,56,8,56,32,56,0,0,0,56,8,56,8,56,0,0,0,40,40,56,8,8,0,0,0,56,32,56,8,56,0,0,0,56,32,56,40,56,0,0,0,56,8,8,8,8,0,0,0,56,40,56,40,56,0,0,0,56,40,56,8,56,0,0,0,16,40,56,40,40,0,0,0,48,40,48,40,48,0,0,0,56,32,32,32,56,0,0,0,48,40,40,40,48,0,0,0,56,32,48,32,56,0,0,0,56,32,48,32,32,0,0
code_end:

Cost and alternatives

The measured helper extent is 106 code bytes; public state is 7 bytes and private scratch is 1 bytes. There are no immutable helper tables. Caller data, the test cases, return-address stack, alignment and display buffers cost additional space. These are assembled sizes, not cycle measurements.

Two set intersections and bounded branches. Filter inexpensive policy before expensive shape tests; for a large object set, still use a broad phase to reduce candidate pairs.

A zero group or mask disables the relevant acceptance. This is neither collision detection nor damage response, and the eight group bits are a game-owned convention rather than hardware sprite-collision bits.

Execution evidence

Four variants run on PAL stock-ROM PRG, using Asm198x 0.0.72, Emu198x 0.44.2. Each executes 64 boundary/varied/error table cases, dirty initialisation and the separate retained-state client. An independent mathematical/container model checks every public byte after every call. This is bounded example coverage, not exhaustive correctness for every possible state.

The records retain loaded immutable code, completion, register sentinels, full packets, source/tool/firmware hashes and captures. The public gate validates these retained observations and rejects altered evidence; it does not run restricted firmware. No firmware is distributed. The eight displayed bytes are the first eight public fields, zero-padded when shorter; the packet includes every field.

Where it fits

Rectangle overlap: Policy filtering and a geometric overlap test answer separate questions.

The Craft · Maths for Games · Source and native evidence.