Automatic Music Transcription of Six Thai Musical Instruments

Automatic music transcription converts audio into note events and, for multi-instrument systems, source-instrument labels [1]. Target assignment additionally requires voice selection, timing, tuning, and register decisions. We study this downstream transformation into musical notation for a fixed target ensemble. The target ensemble comprises six Thai musical instruments drawn from the piphat ensemble [2], [3], including ranat ek, khong wong yai, pi nai, ching, taphon, and klong that (Table I). The generated notation is not claimed to follow traditional ensemble arranging. This boundary matters because Thai musical notation is context-dependent [2], while measured tunings vary in absolute pitch and step size [4], [5]. Prior work addresses melody identification and musical structure [6], [7], [8]. This paper makes three contributions. (1) A deterministic pipeline that converts audio or MIDI into validated musical notation for six Thai instruments, with every source note traced in an audit ledger. (2) Six sound engines that render the notation into one audio track per instrument. (3) A stage-separated evaluation that distinguishes transcription error, intentional omission during arrangement, end-to-end arrangement agreement, and renderer conformance. Fig. 1 separates processing from evaluation; Fig. 2 shows a real ensemble for context. We ask: RQ1, how much labeled melody and supporting material does the core retain on symbolic pop music? RQ2, how much does real-audio transcription alter the notation relative to a reference-MIDI path? RQ3, how does the same comparison behave for controlled synthetic audio? RQ4, do the sound engines obey notation constraints and rerender repeatably?

Automatic music transcription. Multi-instrument AMT models estimate note events together with source-instrument labels [1]; they output what was played, not how a different ensemble should play it. Melody and structure in symbolic music. Melody-bearing channels in MIDI can be identified with dynamic programming over measures [6] or with convolutional models on symbolic scores [7], and music structure analysis segments pieces into sections [8]. We adapt these ideas to select material for target parts. Thai music representation. Thai notation conventions are context-dependent [2], and measured Thai tunings vary in absolute pitch and step size [4], [5], so a fixed equal-step model cannot be assumed. Instrument sound synthesis. Excitation–resonator models are standard in sound synthesis [11], and reed instruments have dedicated digital models [12]. To our knowledge, these components have not been combined into a pipeline that assigns transcribed music to a Thai target ensemble.

MuScriptor Medium [1] estimates note onset, offset, pitch, and source instrument. Normalization then handles tempo and meter maps, PPQN or SMPTE timing, sustain, program changes, overlapping notes, and incomplete note-offs. A logical source is a track–channel–program span. Polyphonic sources are separated with a directed note graph whose edge costs include pitch distance, temporal gap, overlap, metrical position, contour, and continuity. Voice analysis distinguishes a source track from a melodic line inside that track [6], [7]. Exact duplicates are merged for analysis while their source identifiers remain in the audit ledger. Section boundaries first follow markers and tempo or meter changes. If these do not describe the form, bar-level novelty is computed from pitch-class content, contour, density, and instrumentation, with repetition and local homogeneity limiting spurious cuts. These adapt segmentation principles reviewed in [8]. The notation calls its metrical unit a room; this is a project term rather than a universal Thai-notation claim. Onsets and durations are quantized to four slots per room without changing event order.

Candidate voices are scored from note density, inter-onset intervals, pitch and duration prominence, velocity, monophony, continuity, repetition, and metrical salience. Prior work uses dynamic programming to identify melody-bearing MIDI channels across measures [6]. For each room, a segmental dynamic program chooses one candidate or an explicit NONE state. Its transition cost discourages arbitrary source switching but falls at rests and weak boundaries. NONE is favored when material is silent, chordal, or lacks a dominant melodic source; up to two dissimilar high-ranked paths are retained as secondary material. For each section, tonal analysis tests all 12 tonic classes against the seven degrees of the selected tuning profile. Sustained notes and cadential positions receive greater weight; unnecessary tonic changes incur a penalty. The MIDI key signature provides a prior. Selected notes are mapped to the nearest target degree and octave, with pitch and timing changes recorded. This permits modulation without assuming a universal equal-step Thai tuning model [4], [5].

Ranat plays the main grid, reinforced by khong at a lower level. Secondary paths become independent parts. Source-derived texture replaces absent main material; unused source notes may bridge short gaps. These are intentional arrangement choices rather than duplicate transcription. Pi uses only notes from a suitable monophonic source path. Depending on its rests, motion, and relation to the main line, it supplies a counterline or takes selected source phrases while ranat carries the complement. Register is chosen over the phrase rather than independently per note. Articulation follows source timing: connected changes are legato, repeated onsets are tongued, and long lines receive an explicit circular-breath cue instead of an invented rest. No decorative pi notes are composed. Percussion is a metrical accompaniment rather than a direct remapping of MIDI drum pitches. Ching/chap establishes the room cycle; taphon is enabled by aggregate energy or percussion evidence; klong that marks stronger source-drum evidence and percussion-only passages. Pattern length always covers its section and MIDI drums influence energy and accent placement. These rules are deterministic contemporary heuristics, not named traditional nathap or a claim of expert-approved Thai arranging. The audit ledger assigns every input note to represented or arrangement_omitted; dropped remains zero. Because the method deliberately selects and layers material, neither full source-note coverage nor a one-to-one source/target event relation is expected. The UTF-8 .piphat format records Thai solfege degrees ด ร ม ฟ ซ ล ท, octaves, note timing, playing instructions, and percussion patterns. A line contains eight rooms of four slots; - sustains, . rests, and parentheses subdivide a slot. This project notation is not a replacement for traditional performance notation [2]. Before output is saved, the notation is formatted as .piphat text, parsed back, validated against slot structure and instrument playability, and compiled into the event timeline used for rendering; any failure stops the run.

Each note event specifies instrument, pitch or stroke, timing, velocity, and playing instructions. Pi also specifies fingering, tongue contact, pressure, and breath, informed by documented playing techniques [10]. All sound engines read the same event timeline. Fig. 3 summarizes the runtime path from note events and stored response data to an instrument stem. Table I describes the six implemented engines. Ranat is the only audio-domain response model; the remaining engines store numeric acoustic parameters. The common excitation–resonator organization follows established sound-synthesis practice [11], but they are not physical models calibrated against recordings of the target instruments. A continuous pi phrase is rendered in one pass so oscillator phase, pressure, airflow, vibrato, and breath noise survive note boundaries. Each event or phrase is synthesized once into an instrument stem; one normalization gain is shared by all stems and their master.

The evaluation follows the four checks in the lower box of Fig. 1: transcription, conversion, end-to-end arrangement, and sound rendering. Table III lists the corpus used for each check.

POP909 provides all 909 symbolic pop arrangements with labeled MELODY, BRIDGE, and PIANO tracks [13]. It tests the core arrangement without MuScriptor. URMP supplies 44 duet-to-quintet performances with real mixture audio and aligned note annotations [14]. BabySlakh supplies 20 aligned synthetic mixtures as a controlled diagnostic [15], [16]. URMP and BabySlakh use 30 s from the first reference onset. Their reference-MIDI and audio-derived paths are placed on the same 120-BPM absolute-time grid before target arrangement. Every failure remains in the completion denominator. Table II summarizes the role of each test set. For matched events m, precision is P=m/n_pred, recall is R=m/n_ref, and F_1=2PR/(P+R), where n_pred and n_ref are the predicted and reference event counts. Corpus results are macro means of item-level F1 values. An AMT note match requires onset agreement within 50 ms and pitch agreement within 50 cents; onset–offset matching also requires offset agreement within the greater of 20% of reference duration and 50 ms, using mir_eval [17]. The arrangement F1 in Table IV compares multisets of section, room, slot, and mapped degree between the reference-MIDI and audio-derived arrangements. This coarse metric summarizes end-to-end structural agreement at the target-notation stage. We also report stricter target onset–pitch, onset–offset–pitch, and target-instrument-conditioned F1. Target events use maximum-cardinality one-to-one matching with a 50-ms onset tolerance and exact target degree plus octave. These are structural diagnostics: they may penalize a musically valid change in melody selection, section-dependent tonal mapping, or intentional ranat–khong layering. They are not listening-quality measures. POP909 also reports labeled-role coverage, omission, pitch/timing displacement, playability, and source-free pitched events. To separate the stages, AMT compares transcribed notes with annotated reference notes; conversion retention compares represented source-note IDs with the MIDI notes supplied to the arranger. Retention is 100n_retained/n_input; omission is its complement when every input note is accounted for. Thus, omitting 2 of 100 notes is 2%, whereas omitting 2 of 50 is 4%. These are illustrative counts, not experimental results. Layered target parts do not increase the retained source-note count. Intentional omission is distinct from processing failure. Arrangement F1 compares two completed arrangements and is an end-to-end diagnostic, not an isolated conversion-loss percentage or a quantity to subtract from AMT F1. Results describe the frozen evaluation artifacts supplied with this paper, not subsequent interactive development trials. The two audio paths are independent. The reference-MIDI path is completed even when transcription fails, and the audio-derived path reuses a transcription only when the audio checksum, MuScriptor Medium revision, and decoding settings match. Instrument-aware AMT uses a declared mapping from source labels to GM families; target-instrument-conditioned F1 instead asks whether matched target events use the same Thai instrument. The latter therefore measures target-part agreement in addition to onset and pitch agreement. Canonical test passages cover six instruments, pitch, duration, velocity, articulation, pi continuity, deterministic rerendering, and stem summation. Each corpus is summarized separately by macro mean, median, and a 10,000-resample bootstrap 95% confidence interval. Tables III and IV separate symbolic retention from audio-path fidelity so that each numeric column has one meaning. Completed items over total items make each denominator explicit. Undefined values remain missing.

AMT unmatched-reference rates (100(1-R); onset–pitch matching) average 39.6% on 43 successful URMP items and 56.0% on 20 BabySlakh items. Separately, conversion intentionally omits 20.4% of labeled melody notes on average over 909 POP909 MIDI inputs. These percentages use different input populations and cannot be added. Table III isolates symbolic-MIDI retention. On POP909, melody-transfer F1 is 86.2% (95% CI 84.9–87.5), and melody coverage is 79.6% (78.1–81.1). The two measures answer different questions: F1 checks whether source melody notes occupy matching room–slot–degree positions, whereas coverage counts how many labeled melody notes are represented. Bridge coverage is 57.7% (95% CI 56.3–59.0), and piano coverage is 14.3% (13.8–14.8). Source-free pitched events and playability violations both have a mean of 0.0. RQ1: On symbolic pop MIDI, the core retains most labeled melody (79.6% coverage, 86.2% F1) but keeps only 57.7% of bridge and 14.3% of piano notes, so supporting material is selectively reduced. Table IV compares the two audio corpora using the same two stage-specific columns. For URMP, AMT note F1 is 60.4% and arrangement F1 is 35.6%; for BabySlakh the values are 49.8% and 27.4%. These F1 values should not be subtracted because their matching rules differ. The notation results instead show how often the complete audio path reproduces the coarser room–slot–degree decisions made from reference MIDI. Mean F1 for target onset–pitch, onset–offset–pitch, and instrument-conditioned matching is 13.8%, 9.6%, and 9.8% on URMP, and 8.4%, 3.6%, and 5.8% on BabySlakh. Agreement decreases when timing, octave, and target instrument must also match. All 44 URMP reference-MIDI paths produced valid musical notation. The audio path completed for 43; one four-player excerpt produced no note events and is retained as a transcription failure. All 20 BabySlakh paths completed. RQ2: Real-audio transcription substantially alters the notation: on URMP, arrangement F1 between the audio-derived and reference-MIDI paths is 35.6%, falling to 13.8% when exact target onset and pitch must match. RQ3: Controlled synthetic audio does not reduce this gap: on BabySlakh, arrangement F1 is 27.4%. Neither result establishes that one arrangement is musically preferable. Across 61 pitched target keys, median and 95th-percentile pitch error is 1.23 and 7.12 cents. Median pi legato-boundary RMS continuity is 0.966, with 0 detected clicks. All probe waveform samples are finite, no clipping is detected, repeated renders are identical, and stems sum to the master. RQ4: Yes. The engines respect notation constraints (median pitch error 1.23 cents, no clipping, no clicks) and rerender identically. This is an engineering conformance result, not a quality judgment.

POP909 is symbolic pop, URMP is Western chamber music without percussion, and BabySlakh is synthetic. These corpora do not include matched recordings of the six Thai target instruments, and the study does not include a listening test or assessment by Thai music experts. Objective note and renderer checks therefore cannot establish naturalness, cultural authenticity, or listener preference. Future work should collect licensed target recordings, evaluate Thai and non-Thai repertoire separately, and use Thai musicians and listeners to assess part assignment and sound quality.

The proposed system converts audio-derived notes into musical notation and simulated performances for six Thai instruments. For RQ1, symbolic input preserves most labeled melody while selectively reducing supporting parts. For RQ2 and RQ3, audio transcription is the main source of disagreement: arrangement agreement with the reference-MIDI path is 35.6% on real audio and 27.4% on synthetic audio. For RQ4, the renderer is accurate in pitch and fully repeatable. These findings do not establish perceived quality or traditional arranging validity; these require evaluation by Thai musicians.