# INTEGRATION-audio — Multi-Device Audio / MIDI Production (2026-08-07 sweep)

> **Topic.** Cultivating the "inner resonant cavity" — multi-device, multi-channel,
> networked audio production across iPad + Mac mini + MacBook. Mic input + 1-channel
> DSP + 1-channel direct line-in, MIDI + OSC to influence acoustics, simultaneous
> multichannel recording to desktop AND server, multi-MIDI + multi-audio channel
> sync across devices.
>
> **Why this is its own document.** The project has two parallel research threads:
> AMT (cycle-2b / cycle-2c / cycle-3 covered the music-transcription angle) and
> audio (this sweep). They share the same Mac hardware but are otherwise independent
> research domains. The AMT resources and integration are in `RESOURCES-cycle-3.md`
> and `INTEGRATION-cycle-3.md`; the audio side is here.

## 1. TL;DR — recommended stack

| Concern | Pick | Why |
|---|---|---|
| Tempo / phase / start-stop sync | **Ableton Link** | Free, MIT SDK, built into every DAW worth using and 200+ iOS apps. UDP 20808 on the LAN. Pairs cleanly with RTP-MIDI for the full "multi-device DAW" story. |
| MIDI over network | **macOS/iOS built-in Network MIDI** (AppleMIDI) + **`davidmoreno/rtpmidid`** on Linux | Native on every Apple device. Sister on Linux is the `rtpmidid` daemon. Sister on Windows is Tobias Erichsen's rtpMIDI. |
| OSC control | **madskjeldgaard/Birdhouse** (GPL-3.0) for plugin↔OSC bridge, **drlight-code/osccontrol-light** (MIT) for plugin→OSC, **ToscA** (free, IRCAM) for DAW→OSC automation | Three tools, three OSC use cases. Birdhouse = the "sensor into my DAW" adapter. osccontrol-light = the "my automation drives a light installation" plugin. ToscA = the "I want my DAW faders to talk to a KLANG / d&b" bridge. |
| DSP offload | **apohl79/audiogridder** (GPL-3.0) | Run a heavy VST on the Mac mini, stream the audio to the MacBook. The killer app for "two Macs, one big session." |
| Networked audio streaming | **SonoBus** (GPL-3.0) | Peer-to-peer, low-latency, multi-channel. The right tool for "record the same room from two devices" and "remote jam session." |
| macOS loopback / routing | **existentialaudio/BlackHole** (MIT) + built-in Aggregate + Multi-Output | The trio that turns a single laptop into a patchbay. Record system audio, combine interfaces, split outputs. |
| iPad hub | **AUM** (Kymatica, paid) + **Loopy Pro** (Athan, paid) + **Audiobus** (free) | De-facto iPad audio stack. AUM = the matrix. Loopy Pro = the timeline. Audiobus = inter-app routing. |
| Resonant cavity / room analysis | **REW** (free, Java) + **robotai/ResoScan** (MIT) | REW for the deep sweep. ResoScan for the 30-second "is this room OK?" browser check. |
| Multi-channel recording (24+ ch) | **`thejasvibr/fieldrecorder`** (MIT, Python, ASIO) + **FFmpeg** on the server | The Python harness for 24-channel dual-interface capture. FFmpeg for the "simultaneous local + server" push. |
| iPad-as-DSP-into-DAW | **pi-audio-duplex** (Raspberry Pi 4 USB UAC2 + Ethernet gadget) | Turns the iPad into a VST insert in a desktop DAW, low latency. The "free iConnectivity Audio4C" hack. |

## 2. The "inner resonant cavity" — what it actually means in this stack

The phrase "inner resonant cavity" can mean three things in audio:

1. **Vocal resonance** — the throat / mouth / nose as a physical resonator that shapes vocal tone. Mapped to: mic → interface → DAW → speaker, with EQ + compression to "tune" the perceived resonance.
2. **Speaker / cabinet resonance** — the physical enclosure of a monitor or instrument. Mapped to: monitor placement + room treatment + speaker calibration (REW / Sonarworks).
3. **Room modes** — the resonant frequencies of the room itself, which color every recording made in it. Mapped to: room measurement (REW / ResoScan) + acoustic treatment.

For the *stack* in this project, all three collapse to the same physical setup:
- **Capture chain** (mic → interface → DAW) — tuned by mic choice, interface DSP, DAW input gain
- **Monitor chain** (DAW → interface → speakers) — tuned by room analysis (REW), speaker placement, calibration
- **Software tuning** — the DAW + plugin chain on each input

The Session Configurator (Part B of the demo) produces the setup for all three.

## 3. The recommended "Mac + iPad + Mac mini" wiring

The classic setup the user's question describes:

```
┌────────────┐   USB   ┌────────────┐  LAN (Link + RTP-MIDI)  ┌────────────┐
│  iPad      │ ◀──────▶│  Mac mini  │ ◀───────────────────────▶│  MacBook   │
│  (AUM +    │  Audio  │  (the      │  Tempo + MIDI + Audio    │  (the host │
│  Loopy Pro)│  via    │  engine)   │  Gridder stream          │  DAW)      │
│            │  RPi4   │            │                          │            │
└────────────┘  UAC2   └────────────┘                          └────────────┘
                                   │                              │
                                   ▼                              ▼
                            ┌────────────┐                  ┌────────────┐
                            │  Mac mini  │                  │  Server    │
                            │  local     │   FFmpeg pipe    │  (backup + │
                            │  record    │ ────────────────▶│  process)  │
                            └────────────┘                  └────────────┘
```

- **iPad** runs AUM + Loopy Pro. Audio I/O via the iPad's class-compliant USB input OR via a Pi 4 in USB-gadget mode (the pi-audio-duplex hack).
- **Mac mini** runs the heavy stuff — the orchestrator, the multi-channel recording, the audio interface's DSP. Streams to the MacBook via AudioGridder.
- **MacBook** runs the live DAW (Live / Logic / Reaper). Receives audio from the Mac mini via AudioGridder. Sends MIDI/OSC back over RTP-MIDI.
- **Server** receives the FFmpeg-piped stream for backup + offline processing.

## 4. The new `demo-audio/` directory

This sweep ships a new `dashboard/demo-audio/` directory — a static browser app
with two parts.

### Part A — Audio / MIDI landscape (resource cards)

Six cards: Ableton Link, RTP-MIDI, BlackHole+Aggregate, AudioGridder, AUM,
REW+ResoScan. Hand-edited HTML, no JS.

### Part B — Session Configurator

A static config-templating engine. Reads the form state (devices, DAW, iOS apps,
audio interface, network, goal, sample rate, buffer size, mic, channel count) and
renders six copy-pasteable setup blocks:

1. **Sync config** — Link / RTP-MIDI / MTC / word clock, per-app setup steps
2. **Audio routing** — input map, output map, Aggregate/Multi-Output device, ASCII signal-flow diagram
3. **MIDI routing** — Network MIDI session, recommended channel map, Live↔AUM example, Linux bridge
4. **OSC mapping** — address scheme, default ports, Birdhouse/osccontrol-light wiring, sensor/Eurorack bridge
5. **Latency budget** — per-segment ms breakdown, total round-trip, live-tracking notes
6. **Recording setup** — desktop bounce, FFmpeg → server piping, SonoBus / AudioGridder variants, FLAC/Opus compression, timecode sync

Each card has a Copy button. "Copy all 6 blocks" bundles them into one clipboard
paste. The defaults are auto-generated on first load so the user sees the shape
immediately.

### How to extend

| If you want to… | Edit |
|---|---|
| Add a 7th output format | `demo-audio/app.js` — add a `genXxx(d)` function and register it in `TEMPLATES`. Add the matching card markup to `demo-audio/index.html` (copy an existing `<article class="cfg-output">` block). |
| Change the device / DAW / iOS app lists | `demo-audio/index.html` — add a `<label class="chip">` to the relevant `<div data-group="...">`. |
| Change a config template | `demo-audio/app.js` — the generator functions are pure functions of `d`. Edit and reload. |
| Change the recommended stack | `RESOURCES-audio.md` + the cards in `demo-audio/index.html`. |

### Smoke test

`demo-audio/` is smoke-tested with Puppeteer. The smoke verifies: page loads, all
6 cards populate, copy buttons render, copy-all bundles correctly, toggling a
device re-generates correctly, empty state (no DAW + no iOS app) doesn't crash,
iConnectivity + live goal correctly produces a "dual-host" mention, FFmpeg
appears in the recording block, latency budget changes when buffer size changes,
theme toggle works, no console errors.

## 5. What to update in the hub

The `dashboard/index.html` hub already has 7 cards (cycle-3 added the AMT demo
card). **This sweep adds 1 more card** pointing at `demo-audio/`. No other hub
changes.

The `dashboard/arxiv.html`, `github.html`, `assets.html`, `links.html`,
`tester.html` are **not touched** — they are generated by `build_dashboard.py`
from the MD reports.

The `dashboard/RESOURCES.md` and `INTEGRATION.md` (the main files) are **not
touched**. This sweep ships as `RESOURCES-audio.md` + `INTEGRATION-audio.md`
exactly the same way cycle-3 did.

## 6. The new hub card (to be added to `dashboard/index.html`)

Place it after the cycle-3 "AMT Demo + Long Horizon Prompt" card. One card,
one line of HTML:

```html
<a class="card" href="demo-audio/" style="text-decoration: none;">
  <div class="card__head">
    <div>
      <p class="card__step">Tool · audio sweep</p>
      <h3 class="card__title">Multi-Device Audio Configurator</h3>
      <p class="card__sub">iPad + Mac + Mac mini, wired in 60 s</p>
    </div>
    <span class="pill pill--ok">No server</span>
  </div>
  <p class="card__desc">Two tools in one: the audio-production resource cards (Ableton Link, RTP-MIDI, BlackHole, AudioGridder, AUM, REW + ResoScan) and a Session Configurator that turns your devices, DAW, iOS apps, and audio interface into six copy-pasteable setup blocks — sync config, audio routing, MIDI routing, OSC mapping, latency budget, and dual-destination recording. Vanilla JS, copy buttons.</p>
  <div class="card__row">
    <span class="card__label">Outputs</span>
    <span class="card__value">6 setup blocks</span>
  </div>
  <span class="card__link">Open audio demo</span>
</a>
```

## 7. Open questions for the next cycle

1. **Should the audio sweep ship standalone, or merge into the main `RESOURCES.md`?**
   The two threads (AMT + audio) share the same dashboard and the same hardware
   but are otherwise independent. Merging them might dilute the focus; keeping
   them apart might fragment the project. Worth a user decision.
2. **Should the Session Configurator grow into a real-time wiring tool?**
   Right now it's a static templating engine. The natural next step is a small
   Python/Node script that reads the same input and actually runs the steps
   (e.g. starts `rtpmidid`, opens Audio MIDI Setup, launches BlackHole installer).
   That's a real engineering project, not a "sweep deliverable."
3. **Is the iConnectivity Audio4C ($300+) worth recommending?**
   The Pi 4 / pi-audio-duplex hack is "free" but fragile. The Audio4C is
   rock-solid. The user has to decide which trade-off fits.
4. **The "MIDI + OSC to influence audio acoustics" angle is underdeveloped.**
   The OSC mapping in the configurator covers the basics (sensors → plugins,
   plugins → show-control), but the "use OSC to actively shape the resonant
   cavity" path (parametric EQ driven by a sensor, automated room correction)
   is a deeper topic. Worth a future sweep.
5. **What about Dante / AVB for the pro audio side?**
   Dante is the pro-audio industry standard for networked audio. Not on the
   watchlist here because it's commercial (Audinate) and the user is on Apple
   consumer hardware. But if the project ever scales to a venue or a studio,
   Dante is the right next step.