Earlier this year Avid filed a patent application for what they describe as “Metrical grid inference for free rhythm musical input.”
The Problem Of Being Constrained By The Metronome
The basis of the patent revolves around the restrictions often placed the user when recording, in the application for the patent Avid state;
Systems that enable a user to record music for playback or for producing musical notation require that a tempo and a time signature for the music be supplied before the user can start recording. Once the user starts recording, he is constrained to play in time to a metronome click. This is an impediment to recording new ideas, as decisions about changes in tempo and time signature have to be made in advance and the user is unable to introduce timing variations that are a natural part of a performance. If the user opts to turn off the metronome click, he can record in an unconstrained manner, but the recording system is unable to interpret the musical data for quantization or displaying notation because the input is not aligned with an internal tempo map. The requirement to pre-select a time signature and tempo is particularly difficult for less experienced users, who may not be familiar with music theory or even with musical notation. There is therefore a need for musical input systems that free users of this constraint.”
What The Technology Aims To Do
The patent goes on to suggest how the algorithm would work;
A free rhythm musical input system enables a user to record in a natural manner by not requiring the user to make decisions about tempo or time signatures. It also removes the constraint of playing at a constant tempo that follows a metronome click. In such a system, the user is able to indicate to the system that he is ready to start recording, e.g., by hitting a record button, and take his time to prepare this thoughts, at which time he can start playing in an expressive manner The system then interprets what the user plays to infer the locations of the measures and beats, and the corresponding tempos and time signatures that best represent the musical intentions of the user.
Audio Or MIDI?
As to the kind of input types the patent suggest both MIDI and Audio;
In the system described above, the free rhythm music is received in the form of a sequence of MIDI events, in which the temporal locations of the notes and their duration are provided explicitly. It may also be possible to receive the music in audio form, and use audio analysis tools to determine the temporal locations of the note onsets and, in some cases, also the durations of the notes within the audio, thus broadening the applicability of the described techniques to acoustic performances or recordings.
Other Similar Software
Other similar software is made such as UJAM technology which allows the user to sing/play into a microphone and then places the music around the free form performance. AudioScore is a similar technology that already ships with Sibelius.
Where Will We See This Technology?
The patent is unclear as to what Avid products would implement this technology, although Avid have been working to create core solutions that can be used in various products such as Pro Tools and Sibelius.
A Point To Mention
The inventor of this technology is Paul Walmsley, who was part of the disbanded Avid Sibelius team and now works with the new team at Steinberg.