# Δ 2026-08-07 cycle-3 · AMT "what's NEW + MuScriptor verification" sweep

> **Status:** This is the **persisted delta** for the 2026-08-07 cycle-3 sweep.
> Cycles 2b (downstream + LALM) and 2c (separation + tempo + drum + lyrics) live in
> `RESOURCES-delta-cycle-2c.md` and the inlined section of the main `RESOURCES.md`.
> This cycle-3 covers the **hot-take question** (is MuScriptor really on HF+GH, and
> should it replace Basic Pitch in the monthly pipeline?) plus the **3-12-month new
> releases** the previous cycles missed.

## 0. Hot-take verdict — MuScriptor ✅ confirmed

**The previous cycle's "MuScriptor → replace Basic Pitch" recommendation is now
verified against primary sources.** Cycle-2c had the right conclusion; cycle-3
adds the receipts.

| Property | Value | Source |
|---|---|---|
| GitHub repo | [`muscriptor/muscriptor`](https://github.com/muscriptor/muscriptor) | fetched 2026-08-07 |
| Stars / forks | 999 / 128 | GH sidebar, fetched 2026-08-07 |
| Last commit | `c3b82e7` — "Scope transcribe preemption to the same client" | Jul 21 2026 |
| Latest release | v0.2.2 | Jul 21 2026 |
| Code license | **MIT** | [`LICENSE`](https://github.com/muscriptor/muscriptor/blob/main/LICENSE) |
| Weights license | **CC BY-NC 4.0** (gated) | README "License" section |
| Authors | Simon Rouard, Michael Krause, Axel Roebel, Carl-Johann Simon-Gabriel, Alexandre Défossez (Kyutai × Mirelo × IRCAM) | README "Citation" |
| Paper | [arXiv:2607.08168](https://arxiv.org/abs/2607.08168) (Jul 9 2026) | HF papers page |
| Variants on HF | `MuScriptor/muscriptor-{small,medium,large}` (100M / 300M / 1.3B params) | HF model cards |
| Live web demo | https://muscriptor.kyutai.org (no account needed) | kyutai blog post 2026-07-10 |
| Self-hostable UI | `uvx muscriptor serve` → http://127.0.0.1:8222/ (FastAPI + browser piano roll) | GH README "Web UI" |
| Reported Multi-F1 | **48.2** vs YourMT3+ baseline 21.9 (own test set, includes RL post-training) | arXiv:2607.08168 Table |
| Reported Frame-F1 | 73.3 vs 45.5 | arXiv:2607.08168 |
| Training data | 1.45M synthetic scores → 170k real songs (11k hours) → GRPO-style RL on 300 tracks | README "Web UI" section |
| Docker | yes (`Dockerfile` in repo) | opentrain.ai page |

**The honest read** (from a public-data evaluation, not the authors' set): on held-out
public sets MuScriptor **wins on pitch content** but **often loses on exact timing**
(e.g. Bach10 onset 43.1 vs YourMT3+ 59.8). Multi-F1 of 48.2 still means under half
the notes land at the strict ±50 ms window when notes overlap (drops to 42.0). Source:
[YouTube breakdown by Nölte / "n1ZREeKj8ZQ"](https://www.youtube.com/watch?v=n1ZREeKj8ZQ).

**Replacement verdict for the monthly pipeline:**
- **YES** for the "default" multi-instrument full-mix case (this is where Basic Pitch
  struggles most and where MuScriptor is the SOTA).
- **NO** for: real-time / low-latency (use Basic Pitch C++ or NeuralNote) and
  commercial redistribution of weights (gated CC BY-NC 4.0). For commercial-only
  stacks, Aria-AMT (piano) and StemSplitio/htdemucs-ft-onnx remain the picks from
  cycle-2c.
- The recommended drop-in for `setup/pipeline_starter.py` stage 2 is therefore:
  - `muscriptor-medium` as the **default** (was: Basic Pitch)
  - `basic-pitch` retained as the **per-stem** fallback (after Demucs/BS-RoFormer
    separation) — Basic Pitch is better on a clean isolated stem than on a full mix
  - MIROS as the **eval-time oracle** (cycle-3 new entry, see §2 below)

Add `muscriptor` to `trusted_packages.txt` (orchestrator sign-off still required
per AGENTS.md §"Trusted packages gate") with the justification:
`# muscriptor: MIT-code, CC BY-NC-4.0-weights; default multi-instrument AMT, gated download; 2026-07`.

---

## 1. New since cycle-2c — GitHub repos

> Cycle-2c already covered the upstream (separation, tempo, drum, lyrics) side.
> This cycle-3 covers the **mid-2025 → mid-2026 window** of releases and
> the **evaluation + agent-brain** angle that the previous cycle missed.

### 1.1 Multi-instrument full-mix AMT (the SOTA race)

- **[muscriptor/muscriptor](https://github.com/muscriptor/muscriptor)** — 999 ⭐ · 128 forks · v0.2.2 (Jul 21 2026) · MIT (code) / CC BY-NC 4.0 (weights)
  *The "MuScriptor" hot-take. See §0 above. Decoder-only Transformer trained on 170k real songs; beats YourMT3+ 2.2× on Multi-F1 (own test set). Self-hostable web UI + FastAPI server. The first open-weight model that handles a real full mix end-to-end.*

- **[mimbres/YourMT3](https://github.com/mimbres/YourMT3)** — 200+ ⭐ · MLSP 2024 · Apache-2.0
  *The pre-MuScriptor SOTA. The "YourMT3+" variant (Aug 2024) was the previous bar. Last commit Apr 2024 — the repo is now in maintenance mode because MuScriptor (Jul 2026) surpassed it. Still worth keeping as the F1≈0.59 reference baseline. Sister: [`mimbres/YourMT3+`](https://huggingface.co/spaces/mimbres/YourMT3) HF Space demo.*

- **[mason369/music-to-midi](https://github.com/mason369/music-to-midi)** — last commit May 6 2026 · v1.0.14 (May 4 2026) · MIT
  *An end-to-end pipeline tool that already integrates YourMT3+, MIROS, Transkun, Aria-AMT, ByteDance Piano, and BS-RoFormer (VOCAL_SPLIT). Has a Gradio Web UI + PyQt6 desktop + Google Colab. Six processing modes including "MIROS as experimental backend." The closest thing to a turnkey AMT pipeline right now. Note: not the canonical source for any of the models it wraps — wrapper only.*

### 1.2 Music foundation models (Apache-2.0-friendly alternatives to research-only)

- **[HeartMuLa/heartlib](https://github.com/HeartMuLa/heartlib)** — last commit 2026-Q1 · **Apache-2.0** (this is the **new** license — the project moved from research-only to Apache 2.0 between Jan 2026 and the current cycle)
  *A family of open music foundation models: `HeartMuLa-oss-3B` (LLM-based song generator with controllable attributes), `HeartCLAP` (audio-text alignment), `HeartCodec-oss` (12.5 Hz music codec tokenizer), `HeartTranscriptor-oss` (Whisper-based lyrics transcription). Together with MuScriptor this is one of the **only two Apache-2.0-friendly multi-modal music stacks in 2026**. The lyrics-transcription component (`HeartTranscriptor-oss`) is the **Apache-2.0 alternative to VocalParse / Qwen3-ASR** that the cycle-2c open question was hunting for.*

- **[multimodal-art-projection/YuE](https://github.com/multimodal-art-projection/YuE)** — ⭐ 3k+ · released Jan 28 2025 · Apache-2.0 (code), OpenRAIL-S (weights)
  *YuE (乐) — the first open-source full-song music generation foundation model. Lyrics-to-song, comparable to Suno. Uses `m-a-p/xcodec_mini_infer` tokenizer. Note: this is **generation** (text+lyrics → audio), not AMT. But it shares the same LALM-shaped architecture and the same family of "music codec tokens" that AMT decoders are now starting to use (cf. MIROS's MusicFM encoder). Watch as the upstream enabler for the next generation of AMT decoders.*

- **[hkchengrex/MMAudio](https://github.com/hkchengrex/MMAudio)** — CVPR 2025 · research-only
  *Video-to-audio synthesis. Generates synchronized audio from video and/or text. Adjacent to AMT in the multimodal-audio family; not directly an AMT tool but demonstrates the multimodal joint training that audio-LALMs now share.*

### 1.3 Real-time / live music models (2026 wave)

- **[google/magenta-realtime](https://github.com/magenta/magenta-realtime)** — released June 4 2026 · **Apache-2.0** (open-weights)
  *Magenta RealTime 2 (MRT2) — Google's open-weights live music model. The first real-time music generation foundation model from a major lab. The companion to "The Infinite Crate" DAW plugin (also open-sourced). Sister to the `Lyria RealTime` API. Not an AMT tool, but the **same VST-plugin shape** the AMT community needs for live transcription. The "Real-Time network adaptations for minimum latency piano transcription" paper (ISMIR 2025, pp. 83-90) is the AMT side of this story.*

- **[DamRsn/NeuralNote](https://github.com/DamRsn/NeuralNote)** — last commit 2026 · GPL-3.0
  *The audio plugin (VST/AU) that brings SOTA audio-to-MIDI into a DAW. Uses Basic Pitch under the hood. The plugin for "drop it into Logic and get a MIDI region" workflow.*

- **[JatinChowdhury18/RTNeural](https://github.com/jatinChowdhury18/RTNeural)** — last commit 2025 · MIT
  *Real-time neural network inference engine in C++. Designed for real-time audio processing. Pairs with NeuralNote and the ISMIR 2025 latency paper for live-DAW integration.*

### 1.4 Diffusion / score-generation / OMR

- **[hunnieBEE/Polyffusion](https://arxiv.org/abs/2210.05148)** — ISMIR 2023 paper · code at the linked paper
  *Polyffusion — diffusion model for polyphonic music score generation by treating music as image-like piano rolls. Internally controllable. The conceptual cousin of D3RM (next entry).*

- **[D3RM (KAIST)](https://mac.kaist.ac.kr/pubs.html)** — ICASSP 2025 · code linked from KAIST pubs
  *D3RM: Discrete Denoising Diffusion Refinement Model for piano transcription. Beats previous diffusion-based piano transcription on MAESTRO F1. KAIST 2026 also released D3PIA (piano accompaniment generation from lead sheet, ICASSP 2026) and the Multi-track Piano+Violin Duet Dataset (TISMIR 2026). The full KAIST stack is now: D3RM (transcription) + D3PIA (accompaniment) + PianoVAM (multimodal performance dataset, ISMIR 2025).*

- **[PianoVAM](https://mac.kaist.ac.kr/pubs.html)** — ISMIR 2025
  *A multimodal piano performance dataset. Not a model — the eval data D3RM and downstream KAIST work is built on. CC license — verify before commercial use.*

- **[Matchmaker](https://mac.kaist.ac.kr/pubs.html)** — ISMIR 2025
  *Open-source library for real-time piano score following and systematic evaluation. The right answer for the "play a piece, follow the score in real-time" use case. Sister to the onsets-and-frames family but live.*

### 1.5 End-to-end AMT-as-a-product (productionized)

- **[gabe-serna/songscribe](https://github.com/gabe-serna/songscribe)** — actively maintained · unknown license (verify before fork)
  *"The fastest way to turn any song into sheet music." A production web app: upload audio or YouTube URL → instrument splitter → MIDI per stem → sheet music PDF. The closest public competitor to a commercial transcription product. Worth reverse-engineering the UX even if the code license is restrictive. Live at [songscribe.xyz](https://songscribe.xyz/).*

- **[sevagh/basicpitch.cpp](https://github.com/sevagh/basicpitch.cpp)** — last commit 2024 · MIT
  *C++20 inference for Spotify's basic-pitch. ONNX Runtime + Eigen + libremidi. Ships a WebAssembly/Emscripten build. The path to "Basic Pitch in the browser" without PyTorch. Note: this is **per-stem** Basic Pitch — for full-mix, you still need Demucs upstream.*

### 1.6 Watchlist (gone / dormant / re-emerged)

- **omnizart** — re-emerged at [`Music-and-Culture-Technology-Lab/omnizart`](https://github.com/Music-and-Culture-Technology-Lab/omnizart) (a *different* org than the previously 404'd one). Not yet verified to be actively maintained. Keep on watch — if it has real activity in the next cycle, the dashboard's GitHub-pulse page should re-list it.
- **basic-pitch** — the Spotify repo is in slow-maintenance mode (last release older than 6 months). The community is moving to **MuScriptor** for full-mix and to **NeuralNote / basicpitch.cpp** for DAW integration. Basic Pitch is not "dead" but is no longer the "rising star" — flag as **dormant** rather than dropping.
- **mason369/music-to-midi** — NOT a research library, but the easiest end-to-end harness for ALL the AMT models above. Keep watching for MuScriptor + MIROS integration in the next minor release.

---

## 2. New since cycle-2c — HuggingFace

### 2.1 Models (multi-instrument / full-mix AMT)

- **[MuScriptor/muscriptor-small](https://huggingface.co/MuScriptor/muscriptor-small)** · **[MuScriptor/muscriptor-medium](https://huggingface.co/MuScriptor/muscriptor-medium)** · **[MuScriptor/muscriptor-large](https://huggingface.co/MuScriptor/muscriptor-large)** — 100M / 300M / 1.3B params · CC BY-NC 4.0 (gated)
  *The three MuScriptor checkpoints. Sister mirror at `cocktailpeanut/muscriptor-medium` for direct download without HF account. All three are "open-weight" in the sense that the code is MIT and the weights are downloadable after free license acceptance; not "open" in the Apache-2.0 sense (weights are CC BY-NC 4.0).*

- **[minzwon/MusicFM](https://huggingface.co/minzwon/MusicFM)** — ICASSP 2024 · `pretrained_fma.pt` + `pretrained_msd.pt` checkpoints · research-only, non-commercial
  *The conformer + BEST-RQ encoder that MIROS uses as its backbone. Two pretrained checkpoints: FMA-large and Million Song Dataset. **Research use only**. The only piece of MIROS that is public today.*

### 2.2 Music foundation model families (Apache-2.0-friendly)

- **[HeartMuLa/HeartMuLa-oss-3B](https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B)** — 3B params · **Apache 2.0** (Jan 2026 onwards)
  *The LLM-based song generator component. Also released: `HeartMuLa-RL-oss-3B-20260123` (RL-refined) and `HeartCLAP` / `HeartCodec-oss` / `HeartTranscriptor-oss` siblings. **The Apache-2.0 music-foundation-model answer to the previous cycle's "music-LALM" gap.***

- **[HeartMuLa/HeartTranscriptor-oss](https://huggingface.co/HeartMuLa/HeartTranscriptor-oss)** — Whisper-tuned for lyrics · **Apache 2.0**
  *The Apache-2.0 lyrics-transcription model. Closes the cycle-2c "VocalParse needs Qwen3-ASR" gap. Sister to the non-commercial `cohere-transcribe-03-2026` (research-only).*

- **[syvai/cohere-transcribe-diarize](https://huggingface.co/syvai/cohere-transcribe-diarize)** — 2B params · Apache 2.0 (inherited from base)
  *Not an AMT model but the same "speech-to-symbolic-output" family. Drop-in for short-form English audio transcription with speaker labels and word-level timestamps. Useful for the lyrics-Transcription side of the project when English-language vocal tracks are involved.*

### 2.3 Datasets (NEW since cycle-2c)

- **[projectlosangeles/maestro-v3.0.0](https://huggingface.co/datasets/projectlosangeles/maestro-v3.0.0)** — ~200 hours · **CC BY-NC-SA 4.0**
  *HF mirror of MAESTRO v3.0.0 (the 2026 patch that removes 6 mis-included string-quartet-accompaniment piano pieces). v3 is **not compatible with v2** — any existing checkpoint trained on v2 needs re-training or re-evaluation. The original 101 GB zip and 56 MB MIDI-only archive are on the magenta CDN; the HF mirror is the lighter mirror.*

### 2.4 Spaces (already-running demos)

- **[huggingface.co/spaces/mimbres/YourMT3](https://huggingface.co/spaces/mimbres/YourMT3)** — yourmt3+ live demo (free GPU)
  *The HF Space for YourMT3+ — paste a YouTube URL or upload audio, get the multi-track MIDI. Still useful as a free GPU playground even with MuScriptor out.*

- **[huggingface.co/spaces/nvidia/music-flamingo](https://huggingface.co/spaces/nvidia/music-flamingo)** — Music Flamingo 7B live demo
  *The Qwen-2.5 + AF-Whisper music LALM. Note: research-only, non-commercial. The Apache-2.0 sister (`nvidia/audio-flamingo-3-hf`) is the commercial-friendly alternative from cycle-2c.*

---

## 3. The recommended stack for the demo

| Stage | Pick | Why |
|---|---|---|
| AMT (multi-instrument full mix) | `MuScriptor/muscriptor-medium` (HF) + `muscriptor/muscriptor` (GH, MIT code) | The SOTA on the authors' own test set; self-hostable web UI; no torch dep at runtime once pip-installed. Use `large` for the eval, `medium` for the monthly default. |
| AMT (per-stem after separation) | `spotify/basic-pitch` (Apache-2.0) | Still best on a clean isolated stem; the post-Demucs pipeline keeps it. |
| Source separation | `StemSplitio/htdemucs-ft-onnx` (Apache-2.0) — cycle-2c | Drop-in for Demucs without PyTorch; ONNX for browser/mobile. |
| Tempo / beat | `CPJKU/beat_this` (MIT) — cycle-2c | Only commercial-friendly SOTA beat tracker (no DBN patent). |
| Drum | `DrumScript/DrumScript` (MIT) — cycle-2c | MIT deterministic, non-ML. |
| Lyrics | `HeartMuLa/HeartTranscriptor-oss` (Apache 2.0) | The new Apache-2.0 alternative to VocalParse / Qwen3-ASR. |
| LALM (agent brain) | `nvidia/audio-flamingo-3-hf` (Apache 2.0) — cycle-2c | Commercial-friendly. |
| OMR | `liebharc/homr` (MIT) — cycle-2c | Python OMR, MIT. Beats GPL-3.0 Audiveris on Python-vendoring. |

**Test data:** `MusicNet` (CC-BY 4.0) for the eval, MAESTRO (CC BY-NC-SA — flag for
non-commercial only) for the train. Both already listed in the existing `assets.html`.

**Why this combo (2-3 lines):** the new MuScriptor is the only 2026 open-weight
multi-instrument full-mix AMT. The combination of MIT code (you can ship it) + CC
BY-NC 4.0 weights (gated download) matches Basic Pitch's license posture (Apache
for code, CC for some components), so the migration is a license-for-license
swap, not a license expansion. The Apache-2.0 music foundation models from
HeartMuLa close the LALM / lyrics / generation side of the cycle-2c open
question. The remaining open question — multi-instrument full-mix AMT under
Apache-2.0 — still has no answer; that's the gap to keep watching.

---

## 4. License summary

- **Permissive (MIT / Apache 2.0):** MuScriptor **code** (MIT), heartlib + HeartMuLa models (Apache 2.0), BS-RoFormer (MIT), beat_this (MIT), DrumScript (MIT), HOMR (MIT), StemSplitio/htdemucs-ft-onnx (Apache 2.0), cohere-transcribe-diarize (Apache 2.0, inherited), basicpitch.cpp (MIT), Magenta RealTime 2 (Apache 2.0).
- **CC BY 4.0 (commercial OK with attribution):** MAESTRO is **not** in this bucket — it's CC BY-NC-SA 4.0. OaF Drums uses CC BY 4.0 data. MIROS uses MusicFM (research-only).
- **CC BY-NC 4.0 (non-commercial only — gated):** MuScriptor **weights** (the model), SAM-Audio weights (cycle-2c), MusicFM (research-only), Music Flamingo (research-only).
- **GPL-3.0:** NeuralNote (DAW plugin), Audiveris (cycle-2c OMR — Python-vendoring risk).
- **Unknown / verify before shipping:** songscribe (license not declared on GH page — flag for review before any fork).
- **Watchlist (gone / dormant):** basic-pitch (slow maintenance but not dead), omnizart (re-emerged under a new org — verify).

**Commercial stack assemble:** a fully commercial stack is now possible
(MuScriptor **code** is MIT and can be shipped; the **weights** are CC BY-NC
and require a license for commercial use — for true commercial use, contact
Mirelo AI). For pure Apache-2.0 multi-instrument, the gap remains; the
workaround is per-stem Demucs → Basic Pitch (both Apache 2.0). That is the
stack the project should ship unless we have explicit Mirelo licensing.

---

## 5. Integration with existing pipeline (see `INTEGRATION-cycle-3.md`)

The new pieces fit into the existing four-stage pipeline as follows:

| Stage | Cycle-3 change | Effect |
|---|---|---|
| Stage 1 (separation) | no change | StemSplit ONNX from cycle-2c remains default |
| Stage 2 (AMT) | **MuScriptor-medium → default for full-mix; Basic Pitch retained for per-stem** | The hot-take replacement is now wired in |
| Stage 2.5 (lyrics) | **HeartTranscriptor-oss → Apache-2.0 alternative to Qwen3-ASR** | Closes the cycle-2c open question |
| Stage 2.6 (tempo) | no change | beat_this from cycle-2c |
| Stage 3 (cleanup) | no change | music21 |
| Stage 4 (engraving) | no change | MuseScore / Verovio |
| Eval / oracle | **MIROS (research-only) as the F1 oracle for quarterly eval** | The 2025 AMT-Challenge winner; use it as the gold standard |
| Agent brain (LALM) | no change | Audio Flamingo 3 from cycle-2c |
| Real-time / live stage | **NeuralNote (GPL-3.0) + RTNeural (MIT)** for the "DAW plugin" use case | Not in the default pipeline; in the watchlist for v0.3 of the standalone Transcriber app |
| Demo | **new `demo/` directory** with AMT resources showcase + the Long Horizon Prompt tool | See INTEGRATION-cycle-3.md §3 |