# Δ 2026-08-08 cycle-5 · From AMT to "personal publishing company" sweep > **Status:** This is the **persisted delta** for the 2026-08-08 cycle-5 sweep. > Cycle-1 → cycle-4 built the AMT × LLM/agent/foundation-model monitor (Demucs → Basic Pitch → music21 → Verovio cascade, plus the weekly arXiv/GitHub/assets reports, the LH-prompt demo, the audio session-configurator demo). > **Cycle-5 is the scope expansion** — the user is no longer just watching AMT. They want a real workflow for *publishing music*: creating **lead sheets**, preparing **MEI/MusicXML files**, and **standardising engraving** with the feel of a personal publishing house. This delta surfaces the open-source tools that make that workflow free, license-clean, and offline-runnable. > > **Carried forward unchanged:** Demucs, Basic Pitch, MT3, music21, MuScriptor, MIROS, HeartMuLa, ACE-Step, Libretto, Rubato, Stable Audio 3.0, V2N, the 2025 AMT Challenge results, the audio sweep (Ableton Link / RTP-MIDI / BlackHole / AudioGridder), the LH-prompt spec. Those live in `RESOURCES-cycle-4.md`, `RESOURCES-cycle-3.md`, `RESOURCES-audio.md`, and the carried-forward lines below. --- ## 0. What's actually new since cycle-4 (delta table) | # | Item | Why it matters | Where it lands | |---|---|---|---| | 1 | **Verovio 6.0.0 (Jan 28 2026) + 6.1 + Rust bindings (lib.rs, Jun 21 2026)** | First major Verovio release of 2026. **MEI 5.1 schema**, ossia staves, expansion generation, improved MusicXML importer, tablature. The Rust path is fresh — drop-in for any pipeline that needs MusicXML → SVG / PDF / PNG / MIDI without the C++ build. | **Engraving** leg of the personal-publishing pipeline. Default Stage-4 alternative to MuseScore headless. | | 2 | **chrisdonahue/sheetsage** | The only open-source tool that takes audio → **lead sheet (melody + chords)** in one shot. Outputs LilyPond + PDF + audio-aligned MIDI. Trained on 50 h of Hooktheory TheoryTab (CC BY-NC-SA 3.0). Linux + Docker. | **Lead sheet** leg. Use when you have a recording and want a playable lead sheet, not a piano reduction. | | 3 | **RedFerret61/VeeHarmGen** | MusicXML in → MusicXML out, with chord symbols added to a melody. Vertical-harmony generation in pop / jazz / classical styles. | **Lead sheet** leg. Use after melody extraction to get the chord chart. | | 4 | **BACHI (ICASSP 2026, arXiv:2510.06528)** | SOTA symbolic chord recognition on **POP909-CL** (pop) and **When-in-Rome + DCML** (classical). Boundary-aware masked iterative decoding. Root acc 0.6991 / chord acc 0.5718 on classical. | **Chord extraction** leg. Beats autochord and earlier CRNN baselines. | | 5 | **Impro-Visor (GPLv2)** | Monophonic lead sheet tool with automatic backing-track generation from chord symbols. MusicXML in/out, MIDI export, "roadmap" generator. Mature, used in jazz education. | **Lead sheet** leg. The jazz/fake-book reference. | | 6 | **opensheetmusicdisplay (OSMD) — PhonicScore, Vienna** | The de-facto browser MusicXML renderer (TypeScript on top of VexFlow). v0.8.x maintained, React + WordPress bindings exist. | **Browser preview** of any MEI/MusicXML you produce. | | 7 | **musicmetadata/cwr2** + **matijakolaric-com/django-music-publisher (DMP)** | The two open-source pieces of the **music publishing admin** stack: CWR 2.1/2.2 parser/validator/generator (CISAC spec), and a full Django admin for metadata + CMO registration + royalty processing. Apache / MIT style. | **Publishing admin** leg. This is what makes "personal publishing company" actually possible. | | 8 | **leadsheetarrangement (liuhaomin)** | Academic, but the only public **lead sheet → multi-instrument arrangement** pipeline. The bridge between your lead sheet and a full DAW session. | **Arrangement** leg. Long-term watch — not production-ready yet. | | 9 | **chord-mark** | Reference implementation of the ChordMark text format for lyrics + chords + rhythm. Round-trip with Nashville numbers, Roman numerals, solfège. | **Lyric+chord chart** leg. Pairs with the lead sheet output. | | 10 | **LilyPond 2.27.1 (Jun 6 2026) / 2.26.0 (Apr 21 2026)** | Active mainline. Cairo-output option, spacing improvements, **the** reference engraver when quality matters more than speed. GPL. | **Engraving** leg. The "personal publishing" feel comes from here. | | 11 | **DiffRhythm 2 (ASLP-lab, Oct 30 2025)** | Apache 2.0 full-length song gen with lyric alignment. Semi-autoregressive diffusion. | **Music generation** leg (added to cycle-4's ACE-Step / YuE / SongGeneration set). | | 12 | **tencent/SongGeneration (LeVo) v2** | Multi-language (zh, en, es, ja, ...) song gen, up to 4:30. Apache-style license. | **Music generation** leg. The "lyrics in 5 languages" watch. | | 13 | **Qwen-Music (Alibaba, arXiv:2607.11699, Jul 2026)** | Text-to-music + cover-song mode. Vocal-singing capable. | **Music generation** leg. The "cover your old song" watch. | | 14 | **magenta/midi-ddsp (Apache 2.0, Feb 2024 archive)** | Hierarchical MIDI-to-audio synthesis (Notes → Performance → Synthesis). 4 pretrained instruments: violin, flute, saxophone, trumpet. 6 expression controls (volume, vibrato, brightness, attack, vol_fluc, vol_peak_pos). Runs locally with TensorFlow — no cloud. | **Stage 2.5** of the pipeline: post-AMT timbre transfer. MIDI in → re-synthesised audio in a chosen instrument timbre. The "round-trip" stage that completes the audio→score→re-timbred-audio cycle. | **Nothing removed from the existing pipeline.** The cycle-3 / cycle-4 AMT cascade is untouched. Cycle-5 is **additive** — it adds a publishing workflow on top of the transcription workflow. --- ## 1. Lead sheet generation — what's out there ### 1.1 Sheet Sage — audio → lead sheet (the centerpiece) - **[chrisdonahue/sheetsage](https://github.com/chrisdonahue/sheetsage)** — last commit 2024, paper Dec 2022 · **CC BY-NC-SA 3.0** (code + dataset) *The only open-source tool that takes audio (YouTube URL or local file) and outputs a complete lead sheet (melody + chords) as both a LilyPond file and a PDF. 50 hours of aligned melody + harmony data derived from Hooktheory TheoryTab, released alongside under CC BY-NC-SA 3.0. Jukebox-based melody transcription (77% stronger than the strongest prior baseline) + Jukebox-based chord recognition + LilyPond engraving. Linux + Docker. The right answer to "I have a recording, give me a lead sheet for my band."* ### 1.2 VeeHarmGen — melody + placeholder chords → filled chord chart - **[RedFerret61/VeeHarmGen](https://github.com/RedFerret61/VeeHarmGen)** — last commit 2023 · **CC-BY-SA 4.0** (per the README) *Vertical Harmony Generation. Input: MusicXML with a melody and placeholder chord symbols. Output: MusicXML with full chord symbols in pop / jazz / classical styles. The "I have a melody line, fill in the harmony" tool. Pairs perfectly with the basic-pitch / music21 melody extraction path.* ### 1.3 Impro-Visor — the jazz lead sheet reference - **[Impro-Visor/Impro-Visor](https://github.com/Impro-Visor/Impro-Visor)** — last commit ~2024 (active-ish) · **GPLv2** (from SourceForge) *Monophonic lead sheet tool aimed at jazz improvisers. Automatic backing-track generation from chord symbols. MusicXML import via MuseScore, MusicXML + MIDI export, "roadmap" generator for keys and idiomatic chord progressions, chord-voicing keyboard. Mature, used in jazz education for over a decade. The right answer to "I want to learn how to improvise over a real-book chart." License is GPLv2 — fine for personal use, problematic for closed-source commercial.* ### 1.4 BACHI — SOTA chord recognition (pop + classical) - **[Itsuki-music/BACHI_Chord_Recognition](https://huggingface.co/datasets/Itsuki-music/BACHI_Chord_Recognition)** — ICASSP 2026 · [arXiv:2510.06528](https://arxiv.org/abs/2510.06528) *Boundary-Aware Symbolic Chord Recognition. Decomposes chord recognition into boundary detection + iterative ranking of root, quality, bass (inversion). Trained on **POP909-CL** (the corrected pop dataset) and **When-in-Rome + DCML** (classical). Classical: root acc 0.6991, chord acc 0.5718. Apache 2.0. Use when you already have a MusicXML melody line and want the chord chart filled in by a model, not by hand.* ### 1.5 LeadsheetArrangement — the academic lead-sheet → arrangement bridge - **[liuhaomin/LeadsheetArrangement](https://liuhaomin.github.io/LeadsheetArrangement/)** — academic project, last update 2021 *Lead sheet → multi-instrument arrangement. The only public project that bridges the lead-sheet format (melody + chord symbols) and the multi-instrument MIDI format. Long-term watch — the academic seed of a "write a lead sheet, get a full arrangement" pipeline. License: not specified on the project page; treat as research-only.* ### 1.6 ChordMiniApp — the chord-recognition app that already ships Sheet Sage - **[ptnghia-j/ChordMiniApp](https://github.com/ptnghia-j/ChordMiniApp)** — active 2025-2026 *Open-source music analysis: chord recognition, beat tracking, piano visualizer, guitar diagrams, lyric sync, and an experimental Sheet Sage melody backend. Production-grade Next.js + Python. The most polished UI you can copy patterns from for your own lead-sheet generator.* --- ## 2. Engraving — Verovio, LilyPond, OSMD, music21 ### 2.1 Verovio 6.0+ — the MEI engraver - **[rism-digital/verovio](https://github.com/rism-digital/verovio)** — v6.0.0 (Jan 28 2026) · **LGPL** (or commercial license) *Music notation engraving library for MEI with MusicXML, Humdrum, ABC, PAE, EsAC, MuseData input. Outputs SVG, PNG, PDF, MIDI, timemap. v6.0 (Jan 28 2026) added ossia staves, expansion generation in MusicXML, default expansion in MIDI/timemap, improved clef-change/tempo spacing, `note@head.fill` support, improved SVG styling structure. v6.1 (later in 2026) is the first release to use the development version of MEI 5.1 — tablature support, additional data.HEXNUM. Already in the project for the AMT pipeline (cycle-1). The reference for the engraving leg of the personal publishing company.* - **[verovio-data (Rust bindings)](https://lib.rs/crates/verovio-data)** — v0.3.5 (Jun 21 2026) · **MIT/Apache** *Safe Rust bindings to Verovio. Loads MusicXML / MEI / Humdrum / ABC / PAE; renders SVG / PNG / single- and multi-page PDF; produces SMF MIDI with full multi-track playback; synthesises offline WAV via SoundFont; exposes timemap, tempo map, measure timeline, bbox map, score metadata. v0.3.4 (Jun 1 2026) was the first crates.io release. Feature-complete for read+play workflows. The right answer for a Rust-side service that needs to render scores server-side without bundling the C++.* ### 2.2 LilyPond 2.27.1 — the artisanal engraver - **[lilypond/lilypond](https://gitlab.com/lilypond/lilypond)** — v2.27.1 (Jun 6 2026), v2.26.0 stable (Apr 21 2026) · **GPL** *The "personal publishing company" feel comes from here. LilyPond's output is closer to hand-engraved music than any other free tool. v2.26 (Apr 21 2026) added Cairo-library output, improved clef-spacing, time-signature spacing. v2.27.1 is the dev cycle toward 2.28. Use when quality > speed. The "Slow but gorgeous" path. GPL — fine for personal publishing.* - **[openlilylib](https://openlilylib.org/)** — active · **GPL** family *Family of resources for engraving beautiful music with LilyPond + typesetting beautiful texts about music with LaTeX. Snippet library, templates, extensions. The "LilyPond cookbook" community.* ### 2.3 OpenSheetMusicDisplay — the browser renderer - **[opensheetmusicdisplay/opensheetmusicdisplay](https://github.com/opensheetmusicdisplay/opensheetmusicdisplay)** — v0.8.x maintained · **BSD-3-Clause** (per the LICENSE) *TypeScript library on top of VexFlow that renders MusicXML in the browser. v0.8.x is the current series. React component, WordPress block, Node.js server-side rendering all exist. The de-facto "open the score in a browser" tool. Already in the project for the tester page (cycle-1). The right answer for "I have a MusicXML, show it in the page."* ### 2.4 music21 — the Python symbolic toolkit (carried forward) - **[cuthbertLab/music21](https://github.com/cuthbertLab/music21)** — BSD · unchanged *Already in the project. For the cycle-5 publishing leg: music21 is the **MusicXML construction + validation + round-trip** tool. Use it to construct the lead-sheet MusicXML programmatically before passing to Verovio or LilyPond.* --- ## 3. Music publishing admin — metadata, CWR, royalties ### 3.1 Django-Music-Publisher (DMP) — the full publishing admin - **[matijakolaric-com/django-music-publisher](https://github.com/matijakolaric-com/django-music-publisher)** — active · **MIT** *Free, open-source Django app for **managing music metadata, registration/licensing of musical works, royalty management and music data distribution.** Implements **batch registration with CMOs and DSPs via CWR** (CISAC Common Works Registration). Royalty processing with splits, publisher fees, writer shares. MusicBrainz + AC lookup. The "personal publishing company" backbone. The only open-source tool that does the full publishing-admin workflow end-to-end.* ### 3.2 musicmetadata/cwr2 — the CWR parser - **[musicmetadata/cwr2](https://github.com/musicmetadata/cwr2)** — active · **MIT** *Python library for parsing, validation, and generation of **CWR 2.1 and 2.2 files** (CISAC spec, the current version as of 2026). File-, group-, field-, record-, and most transaction-level validation. Convertible to HTML (for visual validation), JSON, etc. The right answer to "I need to register a song with BMI/ASCAP/SESAC and produce a CWR file." Sister repos: `cwr3`, `edi`, `territories`.* ### 3.3 music-metadata.org — the reference architecture - **[music-metadata.org](https://www.music-metadata.org/)** — active engineering reference *Production-grade architecture, code, and operational playbooks for music royalty distribution and metadata reconciliation. The "how do the big publishers actually do this" reference.* ### 3.4 Other (commercial, mentioned for context) - **TuneCore Publishing** (commercial) — 9.0/10 on f6s; tracks publishing royalties across DSPs. - **Songtrust** (commercial) — 8.2/10; publishing administration + reporting. - **Kobalt Music** (commercial) — enterprise-scale publishing platform. - **APORIA** (open-source CWR generator, per the producertour.com reference) — open-source CWR authoring tool; verify license before adopting. --- ## 4. Music generation — what's new since cycle-4 ### 4.1 DiffRhythm 2 - **[ASLP-lab/DiffRhythm2](https://huggingface.co/ASLP-lab/DiffRhythm2)** — released Oct 30 2025 · **Apache 2.0** *Next-gen music-gen. Semi-autoregressive diffusion (vs. the original fully-autoregressive). Full-length songs, **precise lyric alignment, coherent musical structure**. Code + weights on HF. The right answer to "I have lyrics, give me a complete song with proper structure." Apache-2.0 — the most permissive new entry.* ### 4.2 Tencent SongGeneration (LeVo) v2 - **[tencent/SongGeneration](https://huggingface.co/tencent/SongGeneration)** — v2-large + v2-medium (4:30, multi-language) · Apache-style LICENSE file (verify) *Multi-language (zh, en, es, ja, ...) song generation. Up to 4:30 output. Multi-preference alignment. Model cards publish GPU memory + RTF on H20. The right answer to "I want a song in Spanish / Japanese / Chinese." Watchlist, not default — verify the LICENSE before shipping.* ### 4.3 Qwen-Music - **[Qwen-Music](https://huggingface.co/papers/2607.11699)** — paper Jul 2026 · Apache-style (per the Qwen family) *Alibaba's music generation model. Text-to-music + cover-song mode. Vocal-singing capable. Paper-only at the moment — watchlist for the next cycle.* ### 4.4 Carried forward from cycle-4 - **ACE-Step v1.5** (Apache 2.0 / MIT) — budget music-gen default. - **YuE** (Apache 2.0) — high-quality lyrics+full song. - **Stable Audio 3.0** (Stability Community License) — commercial-safe open music-gen. - **HeartMuLa / HeartTranscriptor-oss** (Apache 2.0) — lyrics extraction. --- ## 5. Recommended stack for the personal-publishing pipeline (cycle-5) The project now has **two parallel pipelines** that converge on the same output (a publishable score): ### Pipeline A — "AMT-first" (carried from cycle-1, unchanged) ``` Audio (any) → Demucs / MelBand-RoFormer (separation) → MuScriptor (full-mix AMT) OR Basic Pitch (per-stem) → music21 (cleanup, key/time/quantize) → Verovio / OSMD (engraving → SVG / PDF) → MusicXML / MIDI export ``` ### Pipeline B — "Lead-sheet-first" (NEW for cycle-5) ``` Audio (pop/rock/folk) → Sheet Sage (melody + chords) → VeeHarmGen (fills in / re-voices the chord chart) → music21 (validation, key inference, MusicXML round-trip) → LilyPond (engraving → PDF) or Verovio (SVG / web) → DMP / CWR (publishing admin → CMO registration) ``` ### Pipeline C — "Generation-first" (music from scratch, NEW for cycle-5) ``` Idea / lyrics → ACE-Step v1.5 (budget) / YuE (high quality) / DiffRhythm 2 (full-length) → Stable Audio 3.0 (commercial-safe) → Sheet Sage (extract lead sheet from generated audio) → Pipeline B (engrave + register) ``` ### Why the lead-sheet path is different from the AMT path | Aspect | AMT (Pipeline A) | Lead sheet (Pipeline B) | |---|---|---| | Output | Full piano reduction / multi-instrument score | Melody + chord symbols only | | Best for | Classical, jazz, film music | Pop, rock, folk, worship, lead-sheet fake books | | Engraver | Verovio (SVG) or MuseScore (XML) | **LilyPond** (typographically gorgeous) or Verovio | | Notation | Notes + rhythms + dynamics | Notes + chord symbols + lyrics | | Read by | Trained musicians | Singers + rhythm section + amateurs | | License posture | Most pieces Apache 2.0 / MIT | Sheet Sage is CC BY-NC-SA 3.0 (research only) — verify for commercial | ### "Personal publishing company" stack (the minimum viable publishing house) | Function | Tool | License | Why this one | |---|---|---|---| | Audio → melody | Basic Pitch OR Sheet Sage | Apache 2.0 / CC BY-NC-SA | Basic Pitch for clean stems; Sheet Sage for pop/rock full-mix | | Chord chart | VeeHarmGen OR BACHI | CC-BY-SA 4.0 / Apache 2.0 | VeeHarmGen from MusicXML; BACHI from audio | | Cleanup + validation | music21 | BSD | Round-trip MusicXML, key inference, voice leading | | **Engraving (quality)** | **LilyPond** | **GPL** | The "personal publishing company" feel. Slow but gorgeous. | | Engraving (speed) | Verovio | LGPL | SVG / PDF / MIDI, runs in browser, web-friendly | | Browser preview | OpenSheetMusicDisplay | BSD-3-Clause | Drop the score in a webpage | | Music metadata | Django-Music-Publisher (DMP) | MIT | The only open-source full publishing admin | | CWR registration | musicmetadata/cwr2 | MIT | CISAC CWR 2.1/2.2 generation + validation | | Music generation (idea → audio) | ACE-Step v1.5 | Apache 2.0 / MIT | Budget, fast, 8 GB VRAM | | Music generation (high-quality) | YuE | Apache 2.0 | 16 GB VRAM, 3-4 min output | **The minimum viable "personal publishing company" is a single laptop running:** - Sheet Sage (audio → lead sheet) for the chart - VeeHarmGen (chord re-voicing) for the harmony - music21 (cleanup) for the validation - LilyPond (engraving) for the PDF - DMP (metadata) for the catalog - cwr2 (CWR file) for the registration All open-source. No SaaS fees. No cloud lock-in. License posture: GPL (LilyPond, DMP dependencies) + BSD-3 / MIT / Apache 2.0 for the rest. --- ## 6. License summary (cycle-5 update) - **Permissive (MIT / Apache 2.0 / BSD):** - Demucs, Basic Pitch, music21, mido, librosa, torch, langgraph, crewai, verovio, ACE-Step, YuE, Magenta RealTime 2, StemSplitio/htdemucs-ft-onnx, Beat_This, DrumScript, HOMR, Aria-AMT, **NEW:** DiffRhythm 2 (Apache 2.0), **NEW:** musicmetadata/cwr2 (MIT), **NEW:** Django-Music-Publisher (MIT), **NEW:** OpenSheetMusicDisplay (BSD-3), **NEW:** Verovio Rust bindings (MIT/Apache), **NEW:** VeeHarmGen (CC-BY-SA 4.0) - **LGPL:** - **Verovio** (LGPL or commercial license) — unchanged - **GPL (personal use OK, commercial needs care):** - **Impro-Visor** (GPLv2) — NEW - **LilyPond** (GPL) — NEW - **OpenLilyLib** (GPL) — NEW - **Audiveris** (AGPL-3.0) — unchanged - **CC BY-NC 4.0 (non-commercial only — gated):** - **Sheet Sage** (CC BY-NC-SA 3.0) — NEW - MuScriptor weights, SAM-Audio weights, MusicFM, Music Flamingo — unchanged - **CC BY-NC-SA 4.0:** - **POP909-CL dataset** (BACHI) — NEW - MAESTRO v3.0.0 — unchanged - **Stability AI Community License:** - Stable Audio 3.0 — unchanged - **Research-only / unclear:** - Libretto, Rubato, V2N, LeadsheetArrangement — unchanged - **NEW:** Qwen-Music (paper only) — watchlist - **Watchlist (gone / dormant):** - basic-pitch (slow maintenance) — unchanged - omnizart (verify under new org) — unchanged - MIROS public weights (still gated) — unchanged - MuScriptor PyPI (still "forthcoming") — unchanged **The fully-commercial clean stack for the personal publishing pipeline:** ``` Sheet Sage (CC BY-NC-SA — non-commercial only) ← swap for Basic Pitch if you need commercial ↓ VeeHarmGen (CC-BY-SA 4.0) ← clean music21 (BSD) ← clean LilyPond (GPL — fine for personal publishing, careful if you sell the .ly source) Verovio (LGPL — fine for personal publishing, dual-license option for closed-source) OpenSheetMusicDisplay (BSD-3) ← clean Django-Music-Publisher (MIT) ← clean musicmetadata/cwr2 (MIT) ← clean ``` **For commercial use, replace Sheet Sage with a Basic-Pitch + custom chord-recognition pipeline** (BACHI is Apache 2.0 for the model + dataset is CC BY-NC-SA, so verify before commercial). --- ## 7. Integration with existing pipeline (see `INTEGRATION-cycle-5.md`) The new pieces fit into the existing 4-stage pipeline as follows: | Stage | Cycle-5 change | Effect | |---|---|---| | Stage 1 (separation) | no change | Demucs / MelBand-RoFormer still the default | | Stage 2 (AMT, full-mix) | **ADD Sheet Sage as the pop/rock default** | Sheet Sage produces a lead sheet (melody + chords) instead of a piano reduction. Best for the "personal publishing" use case where the output is for a singer + rhythm section, not a soloist. | | Stage 2 (AMT, per-stem) | no change | Basic Pitch retained for per-stem | | Stage 2.5 (chord recognition) | **ADD BACHI** (Apache 2.0 model) | Replaces autochord / earlier CRNN baselines. SOTA on both pop and classical. | | Stage 2.6 (lyrics) | no change | HeartMuLa remains default | | Stage 2.7 (chord re-voicing) | **NEW:** VeeHarmGen | When the auto chord chart is thin, re-voice with VeeHarmGen in pop / jazz / classical style. | | Stage 3 (cleanup) | no change | music21 | | Stage 4 (engraving) | **ADD LilyPond as the "quality" path** | LilyPond for the PDF. Verovio still default for SVG / web. The "personal publishing company" feel comes from the LilyPond path. | | Stage 4 (browser preview) | **PROMOTE OpenSheetMusicDisplay** | OSMD becomes the standard browser preview for any MusicXML you produce. Already used in the tester. | | Stage 5 (publishing admin) | **NEW:** Django-Music-Publisher | The first end-to-end open-source publishing admin. CMO registration via CWR, royalty processing, splits, music libraries. | | Stage 5 (CWR generation) | **NEW:** musicmetadata/cwr2 | CISAC CWR 2.1/2.2 generation + validation. Required for CMO registration. | | Music generation (lyrics→song) | **ADD DiffRhythm 2** as the long-form default | Apache 2.0, 4-min output, lyric-aligned. | | Music generation (multi-language) | **ADD Tencent SongGeneration v2** | Watchlist. Verify LICENSE before shipping. | | Music generation (cover songs) | **ADD Qwen-Music** | Watchlist. Paper-only as of Aug 8 2026. | | Eval / oracle | no change | MIROS (research-only) as the F1 oracle. 2025 AMT Challenge leaderboard unchanged. | | Agent brain (audio LALM) | no change | Audio Flamingo 3 from cycle-2c | | Agent brain (symbolic) | no change | Libretto (paper only) from cycle-4 | | **End-to-end audio→score** | no change | Rubato (paper only) from cycle-4 | --- ## 8. MIDI-DDSP — Stage 2.5 timbre transfer (cycle-5 post-publication addendum) **Added 2026-08-09 after the user approved the Stage 2.5 integration in the questionnaire.** ### 8.1 The library - **[magenta/midi-ddsp](https://github.com/magenta/midi-ddsp)** — last commit Jan 21 2022 · **archived Feb 1 2024 (read-only)** · **Apache 2.0** · ICLR 2022 · [arXiv:2112.09312](https://arxiv.org/abs/2112.09312) *Hierarchical audio generation model. Three levels: Notes (the MIDI you feed in) → Performance (note-level expression: volume, vibrato, brightness, attack, vol_fluc, vol_peak_pos) → Synthesis (DDSP neural synth, 4 pretrained instruments: violin, flute, saxophone, trumpet). Trained on URMP chamber-ensemble dataset. CLI similar to FluidSynth. **The "MIDI in, expressive audio out" tool the cycle-2c sweep kept asking for.** Archived but the code, weights, and inference script all work; nothing about the inference path has gone stale since 2022.* - **[MIDI-DDSP demo page](https://midi-ddsp.github.io/)** — the canonical audio examples (real recordings, MIDI-DDSP renders, and a comparison). Use this for the "what does it sound like" reference. - **[Magenta project page](https://magenta.withgoogle.com/midi-ddsp)** — Colab link, Spaces link (HF Space at @akhaliq/midi-ddsp is no longer functional, as of late 2023), paper PDF. - **[audio-development-tools (Yuan-ManX)](https://github.com/Yuan-ManX/audio-development-tools)** — community aggregator that lists MIDI-DDSP, DDSP, DDSP-VST, and related tools. Useful for the "what else exists" sweep. ### 8.2 Why MIDI-DDSP, not plain DDSP Plain `magenta/ddsp` (v3.5.1, Apr 2023) does **timbre transfer** — audio in, different-timbre audio out. Needs a source audio file with known pitch + loudness curves. Useful for "make this voice sound like a violin." **MIDI-DDSP** does **MIDI-to-audio synthesis** — MIDI in, expressive audio out. Doesn't need source audio. Useful for "I have the score (MIDI), now render it as a saxophone with my own note expression." For the personal publishing pipeline, MIDI-DDSP is the right call because: - We have MIDI (from MuScriptor or Basic Pitch at Stage 2) - We want to choose the timbre (cover-song workflow) - We want expressive control (volume + vibrato + brightness per note) - The output is audio that gets engraved (Verovio at Stage 4) or shipped (DMP / CWR at Stage 5) ### 8.3 Where it sits in the pipeline ``` ┌────────────────────────────────────────────────────────────────────────────┐ │ STAGE 2 — AMT (audio → MIDI) · unchanged from cycle-1 │ │ audio → Demucs → MuScriptor (full-mix) OR Basic Pitch (per-stem) │ │ ↓ │ │ MIDI ← clean note events, no expression │ │ ↓ │ │ STAGE 2.5 — MIDI-DDSP (MIDI → expressive audio in a chosen timbre) · NEW │ │ ↓ │ │ Pick an instrument (violin, flute, sax, trumpet) │ │ Optional: edit note expression (volume, vibrato, brightness, ...) │ │ ↓ │ │ audio (the re-timbred, expressive version) │ │ ↓ │ │ STAGE 3 — music21 cleanup (key inference, quantize, MusicXML round-trip) │ │ STAGE 4 — Verovio / LilyPond (engraving) │ │ STAGE 5 — DMP / cwr2 (publishing admin) │ └────────────────────────────────────────────────────────────────────────────┘ ``` **Why 2.5 instead of 2 or 3:** - Stage 2 (AMT) only needs MIDI — adding timbre transfer there couples two concerns - Stage 3 (music21 cleanup) is symbolic-only — wrong shape for audio synthesis - 2.5 = the natural MIDI-to-audio bridge, between the symbolic output of 2 and the symbolic input of 3 ### 8.4 The local demo — what runs in the browser A pure-Web-Audio-API approximation lives in `dashboard/demo-ddsp/`. It uses **oscillator + filter + envelope + vibrato** to approximate the 4 MIDI-DDSP instrument timbres: - **Violin**: sawtooth + low-pass filter with slow attack, gentle vibrato - **Flute**: sine + slight noise + minimal vibrato, breathy - **Saxophone**: sawtooth + band-pass filter, expressive vibrato - **Trumpet**: square + soft saturation, brassy attack This is the "concept demonstrator" — same UI shape as the real MIDI-DDSP, same MIDI input, same WAV output. The Python MIDI-DDSP path lives in `setup/ddsp_transfer.py` and produces the real neural-synthesis version when you want it. ### 8.5 License posture - MIDI-DDSP code: Apache 2.0 ✓ - MIDI-DDSP weights (URMP-trained): Apache 2.0 (per the Magenta project page) ✓ - The Python inference path: fully commercial-clean - The browser demo: no external dependencies, no fetch, no model download. Self-contained HTML/CSS/JS. **The MIDI-DDSP path is the cleanest commercial-friendly Stage 2.5 in the project.** Sheet Sage is gated to CC BY-NC-SA 3.0 (non-commercial), and BACHI is Apache 2.0 for the model but the POP909-CL dataset is CC BY-NC-SA. MIDI-DDSP has no such gates.