Tymbal: A General Purpose Synthesizer System

Introduction

This repo implements Tymbal, which is a general purpose sound synthesizer system.

By "general purpose sound synthesizer system" I mean that it has no fixed synthesis engine; it is deliberately not a modular synthesizer, but it has a variety of heterogeneous synthesis engines in it, and "a patch" may mix these at will. Think of it as highly flexible synth with a ridiculous mod matrix, no-compromise engine (compromise on voice count before anything else), and pre-wired with a bunch of musically-useful (or sound-design-useful) templates.

The tymbal is the membrane that a cicada vibrates to make its characteristic sound.

The Tymbal synthesizer is "free as in libre" and is released as CC0: no claims are made, and no patent indemnity is expressed or implied. Use this, or don't, at your own risk.

CC0 means some distros or companies won't use this code. That's A-OK by me.

Inspirations

The design "north star" for the system is the New England Digital Synclavier, specifically its ability to act as a "do it all machine." Secondary inspirations include:

• Fairlight CMI
• Yamaha FM-X and the FS1R formants
• Kurzweil VAST
• Buchla-style west coast synthesis
• Nord Modular G2, Nord Lead
• Access Virus
• CS-80
• Korg Modwave
• The "Sculpture" plugin from Logic
• ASM Hydrasynth
• Kyma/Capybara
• Lemur + Loris

By "inspirations" I mean that I built engines that can (aspirationally) achieve the same results as each of these, without reference to any of their implementations. I have used other instruments for comparison purposes and features, but I have no access to any source code for any of the above (and I doubt Claude Code does either).

Right now at 48kHz it can struggle with a dozen layers (of the most expensive engine in its most expensive settings corner). It can handle a few dozen layers of a cheaper patch. The goal is 96K and 200 full voices (1600 sounding layers) -- the eventual multitimbral target, not the single-channel CLAP plugin's own pool, which is smaller (see plugin/instrument.h's kMaxVoices).

When there is a tension between audio quality, flexibility, and performance, we will choose audio quality first, flexibility second, and performance (voice count) last. Math is internally done at 32 bits, except where it needs to be temporarily wider to avoid more than one narrowing at the end.

The system is designed with fixed-point integer math because the intent is to run it on an FPGA (the current target is the Terasic DE25-Standard). The FPGA isn't implemented yet, but the DSP was designed so that it would translate directly.

The CLAP plugin uses a SIMD floating point implementation for at least one engine, for performance purposes, but this is not bit-identical with the fixed-point integer math. The difference is down at around -117dB, though. That part won't exist on the FPGA; it's just for me to get a decent voice count in Bitwig in the meantime.

The engine will eventually respond to all MIDI 2.0 / MPE controls (even if not mapped by default). It doesn't yet respond to most of them, but it will. It can be triggered at sample-accurate resolution, with a fixed 32-sample delay.

LLM-Based Development Process

I'm using Claude Code to do most of this implementation. If that bothers you, you are entirely free to stop reading now and to not use this project.

Is it slop? I don't think so, any more than compiler output is slop. The repo has extensive unit tests, end-to-end tests with blessed bit-for-bit results, and has ASan, TSan, and UBSan configurations. And I manually test and use the features in Bitwig.

Is it boring uninspired work? IMHO also no; the LLM is implementing the DSP but I'm telling it what to do and why. ("No, don't fix the aliasing there, it's musically useful and is the whole point of this shaper block," or "why the hell did you think identical S&H sequences for two LFOs was OK?")

Is this creative work? Sure. I'm putting in all kinds of shit that's there just because I want it or think I can use it or think someone else can and I know how to do it. What I'm not spending creative effort on is reimplementing polyblep or FFT.

Collaboration

If you want to collaborate and contribute back to this, get in touch. I'll take ideas or patches under advisement.

If you want to fork this and do your own thing, awesome. Let me know about it and I'll add a link to your fork in this README. If you want.

Code of Conduct

If you abuse me or anyone else, or are a transphobe, homophobe, racist, bigot, ableist, or any flavor of fascist, fuck off.

And if you don't like Star Trek then I weep for you, but you're still welcome.

Credits

All of the synthesis techniques represented here, and all of the DSP implementations, come from open information about how they work, or on academic papers describing the techniques used.

See the bibliography.

Nostalgia

As it turns out, many of these techniques were pioneered at the University of Illinois at the Computer Music Project, while I was there (not by me, but I was in the room), or at the CERL Sound Lab across campus. The folks developing these included (from memory):

• Jim Beauchamp
• Doug Jones
• Rob Maher
• Carla Scaletti
• Kurt Hebel
• Lippold Haken
• Kelly Fitz
• Paul Christensen

(And a few others who I never personally met.)

(The MQ technique itself was not from there; it was from McAulay & Quatieri, and further developed at CERL as Lemur and then Loris.)

While I was at the CMP, I worked on synthesis with M4C (think Csound), granular synthesis, and algorithmic composition (the latter with Sever Tipei). Rob was implementing MQ pitch shift while I was doing granular pitch shift, and we traded notes. Or, he gave me pointers, more like; he was a PhD candidate and I was a freshman.

At one point Kelly and Paul and I took a Kyma class from Carla and Kurt (there were about six of us, on a prerelease) and IIRC they joined CERL afterward. Or maybe right before that. I remained at the School of Music until I moved to the ECE department later.

State Of The Union

Right now we build a CLAP plugin, a set of preset patches, a resynth tool, various testing tools, and a standalone patch editor. The patch editor uses basic imgui UI stuff and is not very pretty.

The CLAP plugin has no UI at all; in Bitwig you just see (and automate!) eight macro knobs, which you can name in the editor and are visible in the mod matrices as sources.

Performance is terrible. Play an arpeggio on the DX7 EP patch and you'll see what I mean.

How To Build

Install Bazel 9.2.x either directly or via bazelisk.

bazel build //...

You're best off building with --config=opt. bazel run --config=opt //tools:patch_editor will run the patch editor. There are run targets to install the CLAP plugin and factory patches.

Use tools/refresh_compile_commands.py to generate or update a compile_commands.json file for clangd. You must have built a fastbuild build at least once first.

Overall Layout

This paragraph is entirely aspirational: The instrument offers 64 "channels"; each channel may load its own independent patch and be controlled independently from the other channels. The instrument offers a mixing board with a strip for each channel in use, with inserts and sends. Each channel gets two LFOs (shared by all voices on that channel). (Effectively a channel is an instrument. These are two instrument-wide LFOs like on a CS-80.)

Each voice has its own unison/mono/poly property, glide settings, etc. (Aspirational.)

The voice also owns a "layer mixer" with a strip for each layer of the patch. The layer strip has a tilt EQ, a compressor, a L/R balance, and sends. The layer mixer owns a global effects slot (or two?) where the layer sends are routed (sends, effects, and compressor aspirational). A chorus/flange/delay/etc type effect can be applied as a single optional insert for each layer strip, in the voice's layer mixer (aspirational). The voice gets two voice-wide LFOs and two note-on-random values.

Each voice has up to 8 layers. Each layer can use any engine we offer. Some layers are mono (in which case they get a panner, and the voice's layer mixer's balance is applied on top of the pan), and some are stereo (in which case the voice's layer mixer's balance applies).

Tasks like channel->voice routing will be handled at the HPS layer, rather than on-fabric, as will MIDI/USB I/O. The audio path is all intended to run on the FPGA, including voice allocation and audio output (via fabric-attached ADAT TOSlink). (Aspirational!)

At some point the HPS will gain a sequencer that can do sample-accurate voice timing (tighter than MIDI 2.0). This will be at least a year in the future. The sequencer will hopefully also do disk audio tracks, but it will not be a substitute for Cubase or Pro Tools. Think like a cross between Page R and an MPC. Again: goal, not here now.

Voice Allocation and Degradation

The stated synth goal is 200 voices of polyphony. We're not there yet. When a new voice is needed and there are no unused voices, the oldest-started voice is chosen as a victim. (Working. Aspirational: released-but-not-yet-silent before unreleased, and see degradation below.)

Aspirational: Each layer knows how expensive it is, and some of them can make real time quality tradeoffs. The FM layer could reduce the rendered partial count to make a voice cheaper to render by a known amount; this itself could change the oversampling computation to make it cheaper still. VA patches with supersaws could reduce the saw count, or we could reduce oversampling.

The general idea of the voice allocator's degradation is that we don't have to steal an entire voice at a time. We only need to make older voices cheaper to render, enough to free up enough DSP to sound the new layers.

The engine is (now) smart enough to skip DSP that wouldn't produce a result. That is, if your patch is wired up so that you can't produce any modulations that create frequencies above Nyquist, we won't 4x or 16x oversample it. If your modulator isn't assigned a spectral table we won't compute that waveform.

The allocator will (aspirationally) gain a "dsp pressure" mode, where it can poll a voice and say "make yourself X% cheaper please" and it could spread the degradation over as many voices as it needs to meet the polyphony goal. For example we could start by reducing the partial counts of additive oscillators, then by reducing the copy count of supersaw-type oscillators, then by dropping one or more individual layers, and then finally by dropping the entire voice. (This last paragraph is aspirational.)

The Engines

This section outlines each of the synthesis engines that a voice layer may use. The engine used by a layer is fixed at patch authoring time.

Common Elements

Each of these engines needs to do some tasks that are similar across them, and these are generally done in similar ways. Here are a couple of common elements. When implemented, these are per-engine: they can't reach outside the layer to a separate patch layer for modulations or anything, and not all engines have the same set of them.

Frame Timelines

A "frame timeline" is a set of lanes, each carrying its own value, and a series of snapshots of those lanes. Each snapshot is associated with a time and a curve. The curve (shared by all lanes) set the shape of the interpolation of the lane's value between frames, and the lanes all share the same set of timepoints. The lane values are in whatever units is appropriate for the lane, depending on what parameter the lane controls.

Timelines generally have a loop begin/end frame, and a loop mode (forward, alternating); the lane moves to the frame after the loop end when the note is gated off. There is no requirement that the release frame be the final frame.

The final frame of the layer's overall amplitude lane should be 0 by convention.

Most engine types have multiple timelines, and any given lane may be moved between them. The lanes are entirely independent (different frame times, curves, loops, release, and timeline speed). These are generally not as long as the main one (eg 512 steps vs 64 for the fm-pair engine).

In general, each engine will have a lane for every continous parameter; think of the lane value as the unmodulated value of each knob; you can then use the mod matrix to move from there.

Aspirationally, there are up to four "general purpose lanes", which don't control any specific parameter, but are available as automation lanes as both sources and destinations. (Think: a modulatable and automatable parameter that doesn't map to anything unless you route it there.)

The Mod Matrix

The mod matrix is a list of modulation sources, "via"s, and destinations. The modulation count is variable by engine from 16 to 64 routings.

The source is passed through a nonlinear function, then scaled by its "via", which is another source (without its own nonlinear function; there are other ways to achieve that). Then it can either scale its destination by its "amount", or it can add to it. (Nonlinear function is aspirational; issue #34.)

Mod destinations include any continous parameter in the engine (i.e., any timeframe lane), plus any controller/knob inputs.

Mod sources are the engine's own modulators and audio outputs — its envelopes, LFOs, oscillators, noise, note-on randoms — plus the voice-wide set (note, velocity, pressure, the eight macros, the voice's two LFOs and two randoms). The engine-local ones are published post-modulation, delayed by one sample.

Any source is usable either as a source (and is aspirationally passed through a nonlinear function on the way) or as a via (to scale a modulation).

Aspirationally, there are general purpose timeline lanes that can act as accumulators for complex modulation functions. The general purpose timeline lanes are covered by issue #36.

All modulation occurs at audio rates. If this modulation produces harmonics above Nyquist it may alias (depending on the engine). (Note that on the FM engine, the modulators' output may be itself oversampled, and its depth may be enough to kick the whole layer into further oversampling.)

LFOs

As noted above, each channel (instrument) has two global LFOs, and each voice (note) has two LFOs. Each engine's layer may also have its own LFOs.

LFOs all generally work the same way; they have a frequency, a shape, a parameter for that shape (eg pulse width), and a slew rate.

Frequencies will generally go from very low (1 cycle per minute or more) to around 30-40Hz. Shapes are sine, triangle, pulse, saw, S&H, random walk (walk aspirational). All continous parameters of the layer's LFOs (not shape; for the moment) are mod destinations; parameters of the instrument and the voice LFOs are not.

FM "Pair"

The FM pair consists of, obviously, three FM operators. These can be arranged as either a carrier and two parallel modulators, or as a carrier with two series modulators. (Why pair? Because at first there was only one modulator, like the Synclavier.)

Each of the three operators is an additive oscillator, which can use one of a number of shared spectrum tables owned by the instrument/patch. (I.e. multiple oscillators or layers can use the same tables.) Each spectrum table can contain up to 512 partials, and up to 512 partial snapshot frames. But the product of those counts can't exceed 16384, so you can have 32 partials at 512 frames, or 512 partials at 32 frames, or at 2x steps in between. You don't have to use all the frames or partials; the shape of the box is what's fixed.

The two modulators each have a single band formant filter, whose frequency does not track the keyboard by default (though the modulation matrix could make it do that). The carrier has three formant filters.

The "fm-pair" engine has a primary timeline up to 512 steps, and three alternate timelines of up to 64 steps.

There is no oscillator feedback; you don't need it because a modulator can itself have its own spectrum. There are no algorithms beyond "parallel or series" because multiple carriers can be done with multiple layers, and the more complex algorithms are pretty esoteric and might be better served with a more traditional FM engine.

Lanes include spectrum table selection (per oscillator, not modulatable) and position (affects all 3 oscillators as 0-1 within their tables), carrier pitch offset and amplitude, modulator ratio and depth, formant parameters for the modulators and the 3 carrier bands.

There are two layer LFOs that can be modulated (in-engine; they have lanes), and two note-on random values for the layer, making six LFOs (two instrument, two voice, and two layer) and four note-on random values (two voice, two layer). (The per-instrument LFOs are not yet built.)

Unvoiced Formants

This is pretty much a noise source, a formant bank, a frame timeline, and a mod matrix. More documentation to come.

Virtual Analog

The VA layer has one master oscillator, two general-purpose oscillators, a sub-osc, and a noise source.

The master oscillator, osc A, differs from B and C only in that it is the master oscillator against which hard sync will occur. Oscillators B and C get a hard-sync switch; these are reset when osc A's phase resets. A soft-sync may be coming.

All three general-purpose oscillators get a waveshape (saw, triangle, pulse, sine; more to come) and a detune control. They also get a "copies" control ("supersaw"-style), with pan and detune spread on the copies. The oscillator's output is stereo when "copies" and "spread" are non-zero.

The sub-osc is a sine wave frequency-locked to osc A, sounding one or two octaves down.

Aspirationally, each oscillator may track the note pitch, or not (and the mod matrix can affect the amount). (Except the sub always goes with A.)

Each oscillator has Roland-style cross-mod (FM/PM) with another oscillator that can be exp or linear, and the pairwise ring mod products are available at the source mixer.

The final input to the source mixer is a noise input, which has a one-pole tilt color control on it (note: pink can't be achieved with a one-pole tilt). (This makes 8: osc A, B, C, A×B, B×C, C×A, sub, noise.)

The audio then passes through two filters and one waveshaper, in one of three different topologies: the shaper may be before the filters, after both filters, or on the output of filter 1. There is a par/ser fader, which morphs between parallel and serial arrangement. Because the middle waveshaper position is between filter 1 and filter 2, it is inert when the routing is fully parallel. The shaper has a type (soft clip, etc.) and a drive/amount parameter.

Each of the two filters can be a 1- or 2-pole SVF, or a 1- to 4-pole ladder, or be bypassed. The SVF can be a linear or a driven variant. Choosing the ladder or the driven SVF forces the layer into 4x oversampling.

The SVFs are continuously variable between LP/Notch/HP; this morph, along with the frequency and Q, are lanes/modulation destinations. The driven SVF also has a "drive" parameter. This filter does not self-resonate.

The ladder is continously variable between LP/HP; this morph, along with the frequency and Q, are lanes/modulation destinations. There is also a "drive" parameter which applies drive between the ladder rungs. This filter self-resonates. There's an alternate mode with allpass and notch.

There are three layer LFOs (along with the two voice and the two aspirational whole-instrument/channel LFOs) and five envelopes. The envelopes are each an independent segment timeline. Envelope 1 is hardwired as unipolar and connected to layer volume; its last frame should probably be 0.

Envelopes 2-5 may be unipolar or bipolar and routed at will (for example, to filter cutoff or pitch or filter drive). Each of the five envelopes is a full segment timeline with a modulatable speed, segment count, values, curves, loop points and a release stage to which the envelope moves when a note is released. These are not the same as frame timelines, but are similar to single-lane timelines.

Mod matrix sources include all of the usuals, including one-sample-delayed oscillator outputs and modulated envelope outputs.

This engine may gain more routing and waveshaping abilities in the future, and may also gain things like circuit-emulated waveforms and/or a continously-variable waveform like the CS80.

Partial Bank

This is an MQ-style additive oscillator. The oscillator has a base pitch, and a list of 32 partials to render. Each partial has independent pitch and amplitude, which can be modulated; and a start phase, a pan placement and key tracking, which are fixed per partial. These partials, unlike the additive partials in the fm pair, can have any frequency relationship they want, including crossing over each other. (This is the nature of MQ resynthesis.)

There are multiple independent timelines and as usual any lane may be assigned to any timeline, and the speeds may be independently modulated.

There's a mod matrix with 32 entries; every continuous lane is a dest, as is each timeline's speed. Sources are two modulatable layer LFOs and two note-on randoms, plus the usual voice note-on randoms and the voice and global LFOs. (Global LFOs remain aspirational, see issue #22.)

This layer may gain more controls (formants, tilt EQ) as we gain experience with writing patches for it.

Modal

To be designed.

Karplus-Strong, resonators, waveguides, LPGs, wavefolders

To be designed, and intertwined with some the above.

Tonewheels? LA? Granular?

To be considered.

Sounds

Ones I aim to be able to achieve

1. Jump is penny ante
2. DX7 EP is a given
3. 808 basses, gimme a break
4. A realistic violin and cello
5. Realistic helicopter, jackhammer, rattlesnake
6. Broken glass
7. Tableware tinkling
8. Poured liquid
9. A spoken sentence, a sung sentence (FS1R fseq)
10. Harpsichord
11. Clavinet
12. Toy piano
13. A laser harp
14. Fairy sparkle, and the sound you hear when you whack your head really hard
15. The Hum
16. An automotive spring coil
17. Whispered speech
18. A resynthesized audio snippet
19. Kargyraa throat singing (Tuvan), probably also Sygyt/Khoomei.
20. The "Beat It" sound
21. Robert Fripp's guitar drone
22. The "THX Deep Note"
23. Chewbacca
24. Gongs, bells, xylophones and metallophones, glass, wood, hammers/anvils, bowed/struck/picked
25. Drums and struck surfaces
26. Struck stiff strings
27. Resonators
28. Complex/tortuous waveguides (think: long-tail thunder)
29. A realistic synthesized (not sampled) piano
30. A ruler flapping against the desk, or a cartoon arrow
31. Race cars, bullet whizzes, arrow impacts, gunshots, explosions
32. Blade Runner titles
33. A PPG pad
34. CS80 brasses and pads
35. Realistic brasses, reed instruments, flutes
36. JP8000 supersaw, hoover leads
37. TB303 squeal
38. Berlin Moog basslines, Mini basses
39. Arp 2600 R2D2/burble
40. Jupiter pads
41. Wavestation-style wavesequencing
42. Scary monsters and nice sprites, or at least the scary monsters
43. Jon Lord's Hammond B3 with a knife stuck in the Leslie
44. The wub wub wub sounds the Transformers make while fighting
45. Industrial machinery
46. Insects

Sounds/Features I'm not worrying about

1. Actual samples and extensive multisampling
2. Direct on-fabric audio recording and playback
3. Full PPG-style wavetables (our spectral frames are a superset of this)
4. Operator feedback (see FM Pair section for why it's superfluous)