JamSketch Deep α: A CNN-based Improvisation System in Accordance with User's Melodic Outline Drawing

JamSketch Deep α: A CNN-based Improvisation System in Accordance with User's Melodic Outline Drawing Tetsuro Kitahara and Akio Yonamine (Nihon Univ., Japan) Japan Goal Related works To enable everyone to enjoy musical improvisation including non-musicians ism [Ishida et al. 2004] MIDI Tone Generator Why improvisation is difficult If the N-gram probability for this note is lower than a threshold, the note is replaced to another one Cmaj7 Am7 Chord progression Harmony theory Create a melody Our proposal System makes keys for musically unavailable notes hot to avoid playing such notes long coJIVE [Buchholz et al. 2007] Music knowledge System controls key width in the virtual keyboard according to musical availability Do-Re-Mi Learned musical phrases at the same time User draws: System corrects musically unavailable notes Musical scale Play the melody Thermoscore [Miyashita et al. 2004] JamSketch Deep α Melodic outline System generates: Melody Intuitive, easy to draw Technical details Data representation time ... Rest in outline (1-dim) ... (B) x73 ... y60 (B6) y61 (C2) note continued y62 (C#2) from the previous time y63 (D2) Pitch (cont.) in melody (60-dim) ● Rest in melody (1-dim) Output: 96 Blues melodies Input: Melodic outlines made by smoothing the melodies deconv. deconv. ch ch time from Weimar Jazz Database. ● y120(B6) y121 conv. Data for training ... Chroma in chord tones (12-dim) (B6) x60 x61 (C) x62 (C#) x63 Melodic outline conv. pitch ... ... Pitch in outline (60-dim) Chord y1 (C2) note y2 (C#2) starting at that time y3 (D2) Pitch (new) in melody (60-dim) (C2) x1 (C#2)x2 (D2) x3 inv. filter size filter size ch Example Output matrix Example 12 frames pitch Input matrix 12 frames Model time Pitch (cont) Pitch time