madmom.evaluation.beats

This module contains beat evaluation functionality.

The measures are described in [R3], a Matlab implementation exists here: http://code.soundsoftware.ac.uk/projects/beat-evaluation/repository

Notes

Please note that this is a complete re-implementation, which took some other design decisions. For example, the beat detections and annotations are not quantised before being evaluated with F-measure, P-score and other metrics. Hence these evaluation functions DO NOT report the exact same results/scores. This approach was chosen, because it is simpler and produces more accurate results.

References

[R3]	Matthew E. P. Davies, Norberto Degara, and Mark D. Plumbley, “Evaluation Methods for Musical Audio Beat Tracking Algorithms”, Technical Report C4DM-TR-09-06, Centre for Digital Music, Queen Mary University of London, 2009.

exception madmom.evaluation.beats.BeatIntervalError(value=None)

Exception to be raised whenever an interval cannot be computed.

madmom.evaluation.beats.load_beats(*args, **kwargs)

Load the beats from the given values or file.

To make this function more universal, it also accepts lists or arrays.

Parameters:

values : str, file handle, list or numpy array

Name / values to be loaded.

downbeats : bool, optional

Load downbeats instead of beats.

Returns:

numpy array

Beats.

Notes

Expected format:

‘beat_time’ [additional information will be ignored]

madmom.evaluation.beats.variations(sequence, offbeat=False, double=False, half=False, triple=False, third=False)

Create variations of the given beat sequence.

Parameters:

sequence : numpy array

Beat sequence.

offbeat : bool, optional

Create an offbeat sequence.

double : bool, optional

Create a double tempo sequence.

half : bool, optional

Create half tempo sequences (includes offbeat version).

triple : bool, optional

Create triple tempo sequence.

third : bool, optional

Create third tempo sequences (includes offbeat versions).

Returns:

list

Beat sequence variations.

madmom.evaluation.beats.calc_intervals(events, fwd=False)

Calculate the intervals of all events to the previous/next event.

Parameters:

events : numpy array

Beat sequence.

fwd : bool, optional

Calculate the intervals towards the next event (instead of previous).

Returns:

numpy array

Beat intervals.

Notes

The sequence must be ordered. The first (last) interval will be set to the same value as the second (second to last) interval (when used in fwd mode).

madmom.evaluation.beats.find_closest_intervals(detections, annotations, matches=None)

Find the closest annotated interval to each beat detection.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

matches : list or numpy array

Indices of the closest beats.

Returns:

numpy array

Closest annotated beat intervals.

Notes

The sequences must be ordered. To speed up the calculation, a list of pre-computed indices of the closest matches can be used.

The function does NOT test if each detection has a surrounding interval, it always returns the closest interval.

madmom.evaluation.beats.find_longest_continuous_segment(sequence_indices)

ind the longest consecutive segment in the given sequence.

Parameters:

sequence_indices : numpy array

Indices of the beats

Returns:

length : int

Length of the longest consecutive segment.

start : int

Start position of the longest continuous segment.

madmom.evaluation.beats.calc_relative_errors(detections, annotations, matches=None)

Errors of the detections relative to the closest annotated interval.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

matches : list or numpy array

Indices of the closest beats.

Returns:

numpy array

Errors relative to the closest annotated beat interval.

Notes

The sequences must be ordered! To speed up the calculation, a list of pre-computed indices of the closest matches can be used.

madmom.evaluation.beats.pscore(detections, annotations, tolerance=0.2)

Calculate the P-score accuracy for the given detections and annotations.

The P-score is determined by taking the sum of the cross-correlation between two impulse trains, representing the detections and annotations allowing for a tolerance of 20% of the median annotated interval [R4].

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

tolerance : float, optional

Evaluation tolerance (fraction of the median beat interval).

Returns:

pscore : float

P-Score.

Notes

Contrary to the original implementation which samples the two impulse trains with 100Hz, we do not quantise the annotations and detections but rather count all detections falling withing the defined tolerance window.

References

[R4]	(1, 2) M. McKinney, D. Moelants, M. Davies and A. Klapuri, “Evaluation of audio beat tracking and music tempo extraction algorithms”, Journal of New Music Research, vol. 36, no. 1, 2007.

madmom.evaluation.beats.cemgil(detections, annotations, sigma=0.04)

Calculate the Cemgil accuracy for the given detections and annotations.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

sigma : float, optional

Sigma for Gaussian error function.

Returns:

cemgil : float

Cemgil beat tracking accuracy.

References

[R5]	A.T. Cemgil, B. Kappen, P. Desain, and H. Honing, “On tempo tracking: Tempogram representation and Kalman filtering”, Journal Of New Music Research, vol. 28, no. 4, 2001.

madmom.evaluation.beats.goto(detections, annotations, threshold=0.175, sigma=0.1, mu=0.1)

Calculate the Goto and Muraoka accuracy for the given detections and annotations.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

threshold : float, optional

Threshold.

sigma : float, optional

Allowed std. dev. of the errors in the longest segment.

mu : float, optional

Allowed mean. of the errors in the longest segment.

Returns:

goto : float

Goto beat tracking accuracy.

Notes

[R6] requires that the first correct beat detection must occur within the first 3/4 of the excerpt. In order to be able to deal with audio with varying tempo, this was altered that the length of the longest continuously tracked segment must be at least 1/4 of the total length [R7].

References

[R6]	(1, 2) M. Goto and Y. Muraoka, “Issues in evaluating beat tracking systems”, Working Notes of the IJCAI-97 Workshop on Issues in AI and Music - Evaluation and Assessment, 1997.

[R7]	(1, 2) Matthew E. P. Davies, Norberto Degara, and Mark D. Plumbley, “Evaluation Methods for Musical Audio Beat Tracking Algorithms”, Technical Report C4DM-TR-09-06, Centre for Digital Music, Queen Mary University of London, 2009.

madmom.evaluation.beats.cml(detections, annotations, phase_tolerance=0.175, tempo_tolerance=0.175)

Calculate the cmlc and cmlt scores for the given detections and annotations.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

phase_tolerance : float, optional

Allowed phase tolerance.

tempo_tolerance : float, optional

Allowed tempo tolerance.

Returns:

cmlc : float

Longest continuous segment of correct detections normalized by the maximum length of both sequences (detection and annotations).

cmlt : float

Same as cmlc, but no continuity required.

References

[R8]	S. Hainsworth, “Techniques for the automated analysis of musical audio”, PhD. dissertation, Department of Engineering, Cambridge University, 2004.

[R9]	A.P. Klapuri, A. Eronen, and J. Astola, “Analysis of the meter of acoustic musical signals”, IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 1, 2006.

madmom.evaluation.beats.continuity(detections, annotations, phase_tolerance=0.175, tempo_tolerance=0.175, offbeat=True, double=True, triple=True)

Calculate the cmlc, cmlt, amlc and amlt scores for the given detections and annotations.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

phase_tolerance : float, optional

Allowed phase tolerance.

tempo_tolerance : float, optional

Allowed tempo tolerance.

offbeat : bool, optional

Include offbeat variation.

double : bool, optional

Include double and half tempo variations (and offbeat thereof).

triple : bool, optional

Include triple and third tempo variations (and offbeats thereof).

Returns:

cmlc : float

Tracking accuracy, continuity at the correct metrical level required.

cmlt : float

Same as cmlc, continuity at the correct metrical level not required.

amlc : float

Same as cmlc, alternate metrical levels allowed.

amlt : float

Same as cmlt, alternate metrical levels allowed.

See also

cml()

madmom.evaluation.beats.information_gain(detections, annotations, num_bins=40)

Calculate information gain for the given detections and annotations.

Parameters:

detections : list or numpy array

Detected beats.

annotations : list or numpy array

Annotated beats.

num_bins : int, optional

Number of bins for the beat error histogram.

Returns:

information_gain : float

Information gain.

error_histogram : numpy array

Error histogram.

References

[R10]

M. E.P. Davies, N. Degara and M. D. Plumbley, “Measuring the performance of beat tracking algorithms algorithms using a beat error histogram”, IEEE Signal Processing Letters, vol. 18, vo. 3, 2011.

class madmom.evaluation.beats.BeatEvaluation(detections, annotations, fmeasure_window=0.07, pscore_tolerance=0.2, cemgil_sigma=0.04, goto_threshold=0.175, goto_sigma=0.1, goto_mu=0.1, continuity_phase_tolerance=0.175, continuity_tempo_tolerance=0.175, information_gain_bins=40, offbeat=True, double=True, triple=True, skip=0, downbeats=False, **kwargs)

Beat evaluation class.

Parameters:

detections : str, list or numpy array

Detected beats.

annotations : str, list or numpy array

Annotated ground truth beats.

fmeasure_window : float, optional

F-measure evaluation window [seconds]

pscore_tolerance : float, optional

P-Score tolerance [fraction of the median beat interval].

cemgil_sigma : float, optional

Sigma of Gaussian window for Cemgil accuracy.

goto_threshold : float, optional

Threshold for Goto error.

goto_sigma : float, optional

Sigma for Goto error.

goto_mu : float, optional

Mu for Goto error.

continuity_phase_tolerance : float, optional

Continuity phase tolerance.

continuity_tempo_tolerance : float, optional

Ccontinuity tempo tolerance.

information_gain_bins : int, optional

Number of bins for for the information gain beat error histogram.

offbeat : bool, optional

Include offbeat variation.

double : bool, optional

Include double and half tempo variations (and offbeat thereof).

triple : bool, optional

Include triple and third tempo variations (and offbeats thereof).

skip : float, optional

Skip the first skip seconds for evaluation.

downbeats : bool, optional

Evaluate downbeats instead of beats.

Notes

The offbeat, double, and triple variations of the beat sequences are used only for AMLc/AMLt.

global_information_gain

Global information gain.

tostring(**kwargs)

Format the evaluation metrics as a human readable string.

Returns:

str

Evaluation metrics formatted as a human readable string.

class madmom.evaluation.beats.BeatMeanEvaluation(eval_objects, name=None, **kwargs)

Class for averaging beat evaluation scores.

fmeasure

F-measure.

pscore

P-score.

cemgil

Cemgil accuracy.

goto

Goto accuracy.

cmlc

CMLc.

cmlt

CMLt.

amlc

AMLc.

amlt

AMLt.

information_gain

Information gain.

error_histogram

Error histogram.

global_information_gain

Global information gain.

tostring(**kwargs)

Format the evaluation metrics as a human readable string.

Returns:

str

Evaluation metrics formatted as a human readable string.

madmom.evaluation.beats.add_parser(parser)

Add a beat evaluation sub-parser to an existing parser.

Parameters:

parser : argparse parser instance

Existing argparse parser object.

Returns:

sub_parser : argparse sub-parser instance

Beat evaluation sub-parser.

parser_group : argparse argument group

Beat evaluation argument group.