madmom.evaluation.beats¶
This module contains beat evaluation functionality.
The measures are described in [R84778d1bb8cd-1], a Matlab implementation exists here: http://code.soundsoftware.ac.uk/projects/beat-evaluation/repository
Notes¶
Please note that this is a complete re-implementation, which took some other design decisions. For example, the beat detections and annotations are not quantised before being evaluated with F-measure, P-score and other metrics. Hence these evaluation functions DO NOT report the exact same results/scores. This approach was chosen, because it is simpler and produces more accurate results.
References¶
[R84778d1bb8cd-1] | Matthew E. P. Davies, Norberto Degara, and Mark D. Plumbley, “Evaluation Methods for Musical Audio Beat Tracking Algorithms”, Technical Report C4DM-TR-09-06, Centre for Digital Music, Queen Mary University of London, 2009. |
-
exception
madmom.evaluation.beats.
BeatIntervalError
(value=None)[source]¶ Exception to be raised whenever an interval cannot be computed.
-
madmom.evaluation.beats.
array
(metric)[source]¶ Decorate metric to convert annotations and detections to numpy arrays.
-
madmom.evaluation.beats.
score_10
(metric)¶ Metric to decorate
-
madmom.evaluation.beats.
score_1100
(metric)¶ Metric to decorate
-
madmom.evaluation.beats.
variations
(sequence, offbeat=False, double=False, half=False, triple=False, third=False)[source]¶ Create variations of the given beat sequence.
Parameters: - sequence : numpy array
Beat sequence.
- offbeat : bool, optional
Create an offbeat sequence.
- double : bool, optional
Create a double tempo sequence.
- half : bool, optional
Create half tempo sequences (includes offbeat version).
- triple : bool, optional
Create triple tempo sequence.
- third : bool, optional
Create third tempo sequences (includes offbeat versions).
Returns: - list
Beat sequence variations.
-
madmom.evaluation.beats.
calc_intervals
(events, fwd=False)[source]¶ Calculate the intervals of all events to the previous/next event.
Parameters: - events : numpy array
Beat sequence.
- fwd : bool, optional
Calculate the intervals towards the next event (instead of previous).
Returns: - numpy array
Beat intervals.
Notes
The sequence must be ordered. The first (last) interval will be set to the same value as the second (second to last) interval (when used in fwd mode).
-
madmom.evaluation.beats.
find_closest_intervals
(detections, annotations, matches=None)[source]¶ Find the closest annotated interval to each beat detection.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- matches : list or numpy array
Indices of the closest beats.
Returns: - numpy array
Closest annotated beat intervals.
Notes
The sequences must be ordered. To speed up the calculation, a list of pre-computed indices of the closest matches can be used.
The function does NOT test if each detection has a surrounding interval, it always returns the closest interval.
-
madmom.evaluation.beats.
find_longest_continuous_segment
(sequence_indices)[source]¶ ind the longest consecutive segment in the given sequence.
Parameters: - sequence_indices : numpy array
Indices of the beats
Returns: - length : int
Length of the longest consecutive segment.
- start : int
Start position of the longest continuous segment.
-
madmom.evaluation.beats.
calc_relative_errors
(detections, annotations, *args, **kwargs)[source]¶ Errors of the detections relative to the closest annotated interval.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- matches : list or numpy array
Indices of the closest beats.
Returns: - numpy array
Errors relative to the closest annotated beat interval.
Notes
The sequences must be ordered! To speed up the calculation, a list of pre-computed indices of the closest matches can be used.
-
madmom.evaluation.beats.
pscore
(detections, annotations, *args, **kwargs)[source]¶ Calculate the P-score accuracy for the given detections and annotations.
The P-score is determined by taking the sum of the cross-correlation between two impulse trains, representing the detections and annotations allowing for a tolerance of 20% of the median annotated interval [1].
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- tolerance : float, optional
Evaluation tolerance (fraction of the median beat interval).
Returns: - pscore : float
P-Score.
Notes
Contrary to the original implementation which samples the two impulse trains with 100Hz, we do not quantise the annotations and detections but rather count all detections falling withing the defined tolerance window.
References
[1] (1, 2) M. McKinney, D. Moelants, M. Davies and A. Klapuri, “Evaluation of audio beat tracking and music tempo extraction algorithms”, Journal of New Music Research, vol. 36, no. 1, 2007.
-
madmom.evaluation.beats.
cemgil
(detections, annotations, *args, **kwargs)[source]¶ Calculate the Cemgil accuracy for the given detections and annotations.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- sigma : float, optional
Sigma for Gaussian error function.
Returns: - cemgil : float
Cemgil beat tracking accuracy.
References
[1] A.T. Cemgil, B. Kappen, P. Desain, and H. Honing, “On tempo tracking: Tempogram representation and Kalman filtering”, Journal Of New Music Research, vol. 28, no. 4, 2001.
-
madmom.evaluation.beats.
goto
(detections, annotations, *args, **kwargs)[source]¶ Calculate the Goto and Muraoka accuracy for the given detections and annotations.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- threshold : float, optional
Threshold.
- sigma : float, optional
Allowed std. dev. of the errors in the longest segment.
- mu : float, optional
Allowed mean. of the errors in the longest segment.
Returns: - goto : float
Goto beat tracking accuracy.
Notes
[1] requires that the first correct beat detection must occur within the first 3/4 of the excerpt. In order to be able to deal with audio with varying tempo, this was altered that the length of the longest continuously tracked segment must be at least 1/4 of the total length [2].
References
[1] (1, 2) M. Goto and Y. Muraoka, “Issues in evaluating beat tracking systems”, Working Notes of the IJCAI-97 Workshop on Issues in AI and Music - Evaluation and Assessment, 1997. [2] (1, 2) Matthew E. P. Davies, Norberto Degara, and Mark D. Plumbley, “Evaluation Methods for Musical Audio Beat Tracking Algorithms”, Technical Report C4DM-TR-09-06, Centre for Digital Music, Queen Mary University of London, 2009.
-
madmom.evaluation.beats.
cml
(detections, annotations, *args, **kwargs)[source]¶ Calculate the cmlc and cmlt scores for the given detections and annotations.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- phase_tolerance : float, optional
Allowed phase tolerance.
- tempo_tolerance : float, optional
Allowed tempo tolerance.
Returns: - cmlc : float
Longest continuous segment of correct detections normalized by the maximum length of both sequences (detection and annotations).
- cmlt : float
Same as cmlc, but no continuity required.
References
[1] S. Hainsworth, “Techniques for the automated analysis of musical audio”, PhD. dissertation, Department of Engineering, Cambridge University, 2004. [2] A.P. Klapuri, A. Eronen, and J. Astola, “Analysis of the meter of acoustic musical signals”, IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 1, 2006.
-
madmom.evaluation.beats.
continuity
(detections, annotations, *args, **kwargs)[source]¶ Calculate the cmlc, cmlt, amlc and amlt scores for the given detections and annotations.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- phase_tolerance : float, optional
Allowed phase tolerance.
- tempo_tolerance : float, optional
Allowed tempo tolerance.
- offbeat : bool, optional
Include offbeat variation.
- double : bool, optional
Include double and half tempo variations (and offbeat thereof).
- triple : bool, optional
Include triple and third tempo variations (and offbeats thereof).
Returns: - cmlc : float
Tracking accuracy, continuity at the correct metrical level required.
- cmlt : float
Same as cmlc, continuity at the correct metrical level not required.
- amlc : float
Same as cmlc, alternate metrical levels allowed.
- amlt : float
Same as cmlt, alternate metrical levels allowed.
See also
-
madmom.evaluation.beats.
information_gain
(detections, annotations, *args, **kwargs)[source]¶ Calculate information gain for the given detections and annotations.
Parameters: - detections : list or numpy array
Detected beats.
- annotations : list or numpy array
Annotated beats.
- num_bins : int, optional
Number of bins for the beat error histogram.
Returns: - information_gain : float
Information gain.
- error_histogram : numpy array
Error histogram.
References
[1] M. E.P. Davies, N. Degara and M. D. Plumbley, “Measuring the performance of beat tracking algorithms algorithms using a beat error histogram”, IEEE Signal Processing Letters, vol. 18, vo. 3, 2011.
-
madmom.evaluation.beats.
tostring
(obj)[source]¶ Format the evaluation metrics as a human readable string.
Returns: - str
Evaluation metrics formatted as a human readable string.
-
class
madmom.evaluation.beats.
BeatEvaluation
(detections, annotations, fmeasure_window=0.07, pscore_tolerance=0.2, cemgil_sigma=0.04, goto_threshold=0.175, goto_sigma=0.1, goto_mu=0.1, continuity_phase_tolerance=0.175, continuity_tempo_tolerance=0.175, information_gain_bins=40, offbeat=True, double=True, triple=True, skip=0, downbeats=False, **kwargs)[source]¶ Beat evaluation class.
Parameters: - detections : str, list or numpy array
Detected beats.
- annotations : str, list or numpy array
Annotated ground truth beats.
- fmeasure_window : float, optional
F-measure evaluation window [seconds]
- pscore_tolerance : float, optional
P-Score tolerance [fraction of the median beat interval].
- cemgil_sigma : float, optional
Sigma of Gaussian window for Cemgil accuracy.
- goto_threshold : float, optional
Threshold for Goto error.
- goto_sigma : float, optional
Sigma for Goto error.
- goto_mu : float, optional
Mu for Goto error.
- continuity_phase_tolerance : float, optional
Continuity phase tolerance.
- continuity_tempo_tolerance : float, optional
Ccontinuity tempo tolerance.
- information_gain_bins : int, optional
Number of bins for for the information gain beat error histogram.
- offbeat : bool, optional
Include offbeat variation.
- double : bool, optional
Include double and half tempo variations (and offbeat thereof).
- triple : bool, optional
Include triple and third tempo variations (and offbeats thereof).
- skip : float, optional
Skip the first skip seconds for evaluation.
- downbeats : bool, optional
Evaluate downbeats instead of beats.
Notes
The offbeat, double, and triple variations of the beat sequences are used only for AMLc/AMLt.
-
global_information_gain
¶ Global information gain.
-
class
madmom.evaluation.beats.
BeatMeanEvaluation
(eval_objects, name=None, **kwargs)[source]¶ Class for averaging beat evaluation scores.
-
fmeasure
¶ F-measure.
-
pscore
¶ P-score.
-
cemgil
¶ Cemgil accuracy.
-
goto
¶ Goto accuracy.
-
cmlc
¶ CMLc.
-
cmlt
¶ CMLt.
-
amlc
¶ AMLc.
-
amlt
¶ AMLt.
-
information_gain
¶ Information gain.
-
error_histogram
¶ Error histogram.
-
global_information_gain
¶ Global information gain.
-
-
madmom.evaluation.beats.
add_parser
(parser)[source]¶ Add a beat evaluation sub-parser to an existing parser.
Parameters: - parser : argparse parser instance
Existing argparse parser object.
Returns: - sub_parser : argparse sub-parser instance
Beat evaluation sub-parser.
- parser_group : argparse argument group
Beat evaluation argument group.