Sampling on fabric: 64 MB is the real number, and multisampling is a note-on pointer #78

New issue

Open

opened 2026-09-09 02:05:26 +00:00 by bcox · 0 comments

bcox commented

2026-09-09 02:05:26 +00:00

Owner

The sample oscillator landed as a mono one-shot with kMaxSamples = 2. This records what
a serious sampling story looks like on top of it, and the decisions taken working it out
on 2026-09-08. None of it is implemented.

The budget

The DE25-Standard is an Agilex 5 A5ED013: 138,060 LEs, 358 M20K blocks, 188 DSP
blocks. 358 x 20,480 bits is 895 KiB of on-chip RAM, total, for everything. The board
adds 64 MB of FPGA-side SDRAM and 1 GB of DDR4 shared with the HPS, plus microSD
on the HPS.

So the tiers are the ones a 1990s sampler had: M20K is the voice working set, the 64 MB is
sample RAM, the DDR4 is bank staging the audio path must never read (Linux competes for
it; jitter is unbounded), microSD is the library.

The deterministic sample-RAM figure is 64 MB. Not 4 MB, not tens of GB. That is Akai
S3000XL / E-mu EIV territory.

Patch data already fills it, before any sampling

One patch carries 4 spectral tables x 128 KiB plus 2 sample slots x 128 KiB -- 768 KiB
of static table data, before mode tables. At the multitimbral target of 64 channels that
is ~49 MB of the 64. Sample memory and table memory are the same pool and always were,
and spectral tables alone nearly spend it.

The consequence is the one structural change here: one budget, one meter. TABL,
MODE and SMPL come out of a single pool; slots are bank-scoped and refcounted rather
than copied per patch (64 channels loading the same chiff 64 times is the failure mode);
the editor shows the number. The EPS made you watch a memory figure, and that affordance
is correct here for the same reason -- fixed supply, demand knowable in advance.

The posture: store excitations, synthesize resonances

A sampled Steinway is tens of GB because it stores 88 x 8 x six seconds of resonance.
The resonance is what modal_layer.h and waveguide.h already do, with continuous
velocity and no zone crossfades. What no compact description gets cheaply is the hammer.

A 30 ms hammer transient at 48 kHz / 16-bit is 2.9 KB.
88 keys x 8 velocity layers is 2 MB, against 64.

This is the LA thesis from the sample-oscillator design carried one step on, and it is
why docs/superpowers/specs/2026-09-08-inter-layer-routing-design.md is the sampling
architecture: a real recording driving a modeled resonator (Drive::kInput,
kDriveSite1) is a class of instrument no sampler can produce, at kilobytes.

Multisampling is a note-on pointer selection

VaLayer::NoteOn already resolves patch.sample[i] -- a const SampleSlot* -- into a
SampleBinding, reading the rate, the root note and the keytrack distance from whichever
slot it points at (dsp/va_layer.h:772-808). Multisampling is choosing that pointer from
a zone table instead of taking it as a constant.

Render does not change. There is no second shape and no second oscillator type. A
zone is {key_lo, key_hi, vel_lo, vel_hi, slot} -- six bytes; sixteen of them is ~100
bytes of patch.

Two things follow from only ever zoning attacks:

Hard switch, no crossfade. No second accumulator, no double read, no extra
bandwidth. Velocity switching over a 150 ms transient is nearly inaudible, where the
same switch on a sustain is why samplers crossfade at all.
4-6 zones across the keyboard, not 30. Transposition drags formants over a long
sustain; over a noise-dominated transient, six semitones costs almost nothing.

Zones are per-oscillator

Not layer-scoped. The case for sharing a zone set was two oscillators used as a stereo
pair, and it dies twice. There is no per-oscillator pan -- kPan is layer-scoped,
kFilter1Pan/kFilter2Pan are filter-scoped, and the oscillator mixdown accumulates
through one common gain -- so the pair is not expressible without a new control. And
stereo slots serve that case directly and more cheaply.

What is left is two oscillators with different recordings on the same boundaries. Those
want the same boundaries, not the same slots, and a few velocity units of drift between
them is inaudible. Per-oscillator scope also buys something real: osc B holding a
hard-hammer set that only appears above velocity 90 is a different zone map by design.

Stereo is a property of the slot

Not of the oscillator, and not a format-wide doubling. The recordings that should dominate
the bank are point sources -- hammer, chiff, mallet, bow scrape -- and their stereo comes
from the resonator afterward. SampleSlot gains a channel count, ReadSample reads one
word or two, a mono slot costs exactly what it costs today, and the zone table does not
care because a zone names a slot and the slot knows its own width.

VaOscillator::Render already returns Stereo and RenderSample already carries
left/right accumulators; it reads one word and feeds both. The second word goes to
right. The only arithmetic that grows is the sync-step correction: two subtractions
instead of one.

Capacity in frames, not words. A stereo slot is 256 KiB and holds the same 1.365 s
at 48 kHz. Stereo recordings being half as long as mono ones is an arbitrary rule
someone trips over later.
Interleaved, not planar. L,R,L,R puts one linear interpolation's four words in one
burst; planar is two transactions at unrelated addresses.

The audio-rate source fold

dsp/va_layer.h:1313-1329 takes the left channel only for its audio-rate sources:
tick_source[0..2] for within-tick cross-mod, source_a_/source_b_/source_c_ for the
matrix. With a mono sample that is exact. With a stereo slot, ring- or cross-modulating
from osc A would silently take the left channel of the recording, making modulation depth
a function of how the recording happened to be panned.

Decision: fold (L + R + 1) >> 1 -- the same fold the tap bus uses -- at all six sites,
guarded on patch_->osc[i].shape == VaShape::kSample. The guard is constant for the
note, like the kSample branch already at the top of Render. Folding unconditionally is
not available: left of a unison-spread saw is a real signal, not half of one, and every
existing patch using osc A as a cross-mod source would move, goldens included.

It lands with the stereo read, not before it. RenderSample is strictly symmetric
today -- pending_left_ = pending_right_ = next, one value into both accumulators -- so
left == right == x, and (x + x + 1) >> 1 == x for every integer x, negative included.
The fold is bit-exact today and no input distinguishes it, so there is nothing to write a
failing test against until a slot can hold two channels.

The test that will have something to fail against: a stereo slot holding a hard-panned
recording, used as a ring-mod source. Unfolded, modulation depth tracks the recording's
pan; folded, it does not.

Deferred, with reasons

Block ADPCM (IMA-style, per-block predictor reset so seeks stay exact) is 4:1 for a
shift-and-add state machine, and would make 64 MB behave like 256. It is the only item
here that meaningfully changes ReadSample. Defer until the pool is real and 64 MB is
known to bind; it is also lossy, so the float-oracle story needs a position first.
Streaming and prefetch. Bounded stream counts make this fit fabric: 128 streams with
64-word FIFOs is 16 KiB of M20K, under 2%, and 128 x 96 kHz x 2x rate x 2 B is ~49 MB/s
against a few hundred available. Capacity and bandwidth are both comfortable; jitter is
what bites, and a fixed round-robin schedule removes it by construction. But there is no
C++ counterpart -- the host has RAM -- so there is nothing to write now.
Loop points, SFZ/Kontakt import, round-robin and repetition libraries, a script layer,
disk streaming. Each needs a best-effort memory manager, which is the one thing fabric
refuses to be.

Open

kMaxSamples = 2 has to grow for zones, and by how much is unanswered. It is the
same question as the pool budget and should be answered with it, not before it.
Bank-scoped refcounted slots and the editor meter is the recommended next decision,
and the only structural one here.
The tap bus's channel count. 2026-09-08-inter-layer-routing-design.md says mono at
e4d324b and in its working tree, with a stated rationale; a mono-or-stereo bus was
discussed since. Whichever it is needs to be in that document before anyone implements
from memory. A stereo sample layer is unaffected either way -- its tap is the same fold.
2026-09-07-sample-oscillator-design.md says flatly "Not multisampling." Read in
place, that is a scope statement for that stage, sitting beside "not loop points" and
"not a sample layer kind", and the argument it actually makes is against a sample
player bolted alongside the synth -- which zones do not reintroduce. But the sentence
is there and should be re-argued in writing rather than drifted past.
README.md:531 still lists "Actual samples and extensive multisampling" under
Sounds/Features I'm not worrying about. That predates the sample oscillator. Worth
rewriting into the position above rather than deleting: "no multisampling of sustains,
short zoned excitations yes" is sharper than either the old line or its absence.

Device figures from the Agilex 5 E-Series product table, and the DE25-Standard pages on
RocketBoards and LinuxGizmos.

Addendum, 2026-10-05: what the feature survey says wants this

From docs/feature-survey.md, whose research was read mostly as archive.org OCR of the manuals by research agents and not re-checked here. The instruments there that need sampling as a family, past the zoned attacks this ticket is about: the Kawai K4 (PCM waves as oscillator sources, the synth's filters after them), the Ensoniq SQ-80 (sampled attack transients crossfading into synthesized waves, the nearest to this ticket's posture), the Akai EWI3030m (a PCM sound module behind a wind controller), and a drum kit with a sound per key, which is this ticket's zone table at sixteen or more zones with no transposition. The gap this ticket defers by name, sustaining loops, is what the K4 and the EWI module play. The formant-preserving pitch shift gaps.md lists under sampling is the granular ticket's (#141), not this one's.

The sample oscillator landed as a mono one-shot with `kMaxSamples = 2`. This records what a serious sampling story looks like on top of it, and the decisions taken working it out on 2026-09-08. None of it is implemented. ## The budget The DE25-Standard is an Agilex 5 `A5ED013`: 138,060 LEs, **358 M20K blocks**, 188 DSP blocks. 358 x 20,480 bits is **895 KiB of on-chip RAM, total, for everything**. The board adds **64 MB of FPGA-side SDRAM** and **1 GB of DDR4 shared with the HPS**, plus microSD on the HPS. So the tiers are the ones a 1990s sampler had: M20K is the voice working set, the 64 MB is sample RAM, the DDR4 is bank staging the audio path must never read (Linux competes for it; jitter is unbounded), microSD is the library. **The deterministic sample-RAM figure is 64 MB.** Not 4 MB, not tens of GB. That is Akai S3000XL / E-mu EIV territory. ## Patch data already fills it, before any sampling One patch carries 4 spectral tables x 128 KiB plus 2 sample slots x 128 KiB -- **768 KiB of static table data**, before mode tables. At the multitimbral target of 64 channels that is **~49 MB of the 64**. Sample memory and table memory are the same pool and always were, and spectral tables alone nearly spend it. The consequence is the one structural change here: **one budget, one meter.** `TABL`, `MODE` and `SMPL` come out of a single pool; slots are bank-scoped and refcounted rather than copied per patch (64 channels loading the same chiff 64 times is the failure mode); the editor shows the number. The EPS made you watch a memory figure, and that affordance is correct here for the same reason -- fixed supply, demand knowable in advance. ## The posture: store excitations, synthesize resonances A sampled Steinway is tens of GB because it stores 88 x 8 x six seconds of *resonance*. The resonance is what `modal_layer.h` and `waveguide.h` already do, with continuous velocity and no zone crossfades. What no compact description gets cheaply is the hammer. - A 30 ms hammer transient at 48 kHz / 16-bit is **2.9 KB**. - 88 keys x 8 velocity layers is **2 MB**, against 64. This is the LA thesis from the sample-oscillator design carried one step on, and it is why `docs/superpowers/specs/2026-09-08-inter-layer-routing-design.md` is the sampling architecture: a real recording driving a modeled resonator (`Drive::kInput`, `kDriveSite1`) is a class of instrument no sampler can produce, at kilobytes. ## Multisampling is a note-on pointer selection `VaLayer::NoteOn` already resolves `patch.sample[i]` -- a `const SampleSlot*` -- into a `SampleBinding`, reading the rate, the root note and the keytrack distance from whichever slot it points at (`dsp/va_layer.h:772-808`). Multisampling is choosing that pointer from a zone table instead of taking it as a constant. **`Render` does not change. There is no second shape and no second oscillator type.** A zone is `{key_lo, key_hi, vel_lo, vel_hi, slot}` -- six bytes; sixteen of them is ~100 bytes of patch. Two things follow from only ever zoning attacks: - **Hard switch, no crossfade.** No second accumulator, no double read, no extra bandwidth. Velocity switching over a 150 ms transient is nearly inaudible, where the same switch on a sustain is why samplers crossfade at all. - **4-6 zones across the keyboard, not 30.** Transposition drags formants over a long sustain; over a noise-dominated transient, six semitones costs almost nothing. ### Zones are per-oscillator Not layer-scoped. The case for sharing a zone set was two oscillators used as a stereo pair, and it dies twice. There is no per-oscillator pan -- `kPan` is layer-scoped, `kFilter1Pan`/`kFilter2Pan` are filter-scoped, and the oscillator mixdown accumulates through one common gain -- so the pair is not expressible without a new control. And stereo slots serve that case directly and more cheaply. What is left is two oscillators with different recordings on the same boundaries. Those want the same boundaries, not the same slots, and a few velocity units of drift between them is inaudible. Per-oscillator scope also buys something real: osc B holding a hard-hammer set that only appears above velocity 90 is a different zone map by design. ## Stereo is a property of the slot Not of the oscillator, and not a format-wide doubling. The recordings that should dominate the bank are point sources -- hammer, chiff, mallet, bow scrape -- and their stereo comes from the resonator afterward. `SampleSlot` gains a channel count, `ReadSample` reads one word or two, a mono slot costs exactly what it costs today, and the zone table does not care because a zone names a slot and the slot knows its own width. `VaOscillator::Render` already returns `Stereo` and `RenderSample` already carries `left`/`right` accumulators; it reads one word and feeds both. The second word goes to `right`. The only arithmetic that grows is the sync-step correction: two subtractions instead of one. - **Capacity in frames, not words.** A stereo slot is 256 KiB and holds the same 1.365 s at 48 kHz. Stereo recordings being half as long as mono ones is an arbitrary rule someone trips over later. - **Interleaved, not planar.** L,R,L,R puts one linear interpolation's four words in one burst; planar is two transactions at unrelated addresses. ## The audio-rate source fold `dsp/va_layer.h:1313-1329` takes the **left channel only** for its audio-rate sources: `tick_source[0..2]` for within-tick cross-mod, `source_a_`/`source_b_`/`source_c_` for the matrix. With a mono sample that is exact. With a stereo slot, ring- or cross-modulating from osc A would silently take the left channel of the recording, making modulation depth a function of how the recording happened to be panned. **Decision: fold `(L + R + 1) >> 1` -- the same fold the tap bus uses -- at all six sites, guarded on `patch_->osc[i].shape == VaShape::kSample`.** The guard is constant for the note, like the `kSample` branch already at the top of `Render`. Folding unconditionally is not available: `left` of a unison-spread saw is a real signal, not half of one, and every existing patch using osc A as a cross-mod source would move, goldens included. **It lands with the stereo read, not before it.** `RenderSample` is strictly symmetric today -- `pending_left_ = pending_right_ = next`, one `value` into both accumulators -- so `left == right == x`, and `(x + x + 1) >> 1 == x` for every integer x, negative included. The fold is bit-exact today and no input distinguishes it, so there is nothing to write a failing test against until a slot can hold two channels. The test that will have something to fail against: a stereo slot holding a hard-panned recording, used as a ring-mod source. Unfolded, modulation depth tracks the recording's pan; folded, it does not. ## Deferred, with reasons - **Block ADPCM** (IMA-style, per-block predictor reset so seeks stay exact) is 4:1 for a shift-and-add state machine, and would make 64 MB behave like 256. It is the only item here that meaningfully changes `ReadSample`. Defer until the pool is real and 64 MB is known to bind; it is also lossy, so the float-oracle story needs a position first. - **Streaming and prefetch.** Bounded stream counts make this fit fabric: 128 streams with 64-word FIFOs is 16 KiB of M20K, under 2%, and 128 x 96 kHz x 2x rate x 2 B is ~49 MB/s against a few hundred available. Capacity and bandwidth are both comfortable; jitter is what bites, and a fixed round-robin schedule removes it by construction. But there is no C++ counterpart -- the host has RAM -- so there is nothing to write now. - **Loop points, SFZ/Kontakt import, round-robin and repetition libraries, a script layer, disk streaming.** Each needs a best-effort memory manager, which is the one thing fabric refuses to be. ## Open 1. **`kMaxSamples = 2` has to grow for zones**, and by how much is unanswered. It is the same question as the pool budget and should be answered with it, not before it. 2. **Bank-scoped refcounted slots and the editor meter** is the recommended next decision, and the only structural one here. 3. **The tap bus's channel count.** `2026-09-08-inter-layer-routing-design.md` says mono at `e4d324b` and in its working tree, with a stated rationale; a mono-or-stereo bus was discussed since. Whichever it is needs to be in that document before anyone implements from memory. A stereo sample layer is unaffected either way -- its tap is the same fold. 4. **`2026-09-07-sample-oscillator-design.md` says flatly "Not multisampling."** Read in place, that is a scope statement for that stage, sitting beside "not loop points" and "not a sample layer kind", and the argument it actually makes is against a sample *player* bolted alongside the synth -- which zones do not reintroduce. But the sentence is there and should be re-argued in writing rather than drifted past. 5. **`README.md:531` still lists "Actual samples and extensive multisampling"** under *Sounds/Features I'm not worrying about*. That predates the sample oscillator. Worth rewriting into the position above rather than deleting: "no multisampling of sustains, short zoned excitations yes" is sharper than either the old line or its absence. Device figures from the Agilex 5 E-Series product table, and the DE25-Standard pages on RocketBoards and LinuxGizmos. ## Addendum, 2026-10-05: what the feature survey says wants this From `docs/feature-survey.md`, whose research was read mostly as archive.org OCR of the manuals by research agents and not re-checked here. The instruments there that need sampling as a family, past the zoned attacks this ticket is about: the Kawai K4 (PCM waves as oscillator sources, the synth's filters after them), the Ensoniq SQ-80 (sampled attack transients crossfading into synthesized waves, the nearest to this ticket's posture), the Akai EWI3030m (a PCM sound module behind a wind controller), and a drum kit with a sound per key, which is this ticket's zone table at sixteen or more zones with no transposition. The gap this ticket defers by name, sustaining loops, is what the K4 and the EWI module play. The formant-preserving pitch shift `gaps.md` lists under sampling is the granular ticket's (#141), not this one's.