New paper by Mol­ly Hen­ry: Amu­sia diag­no­sis should bet­ter rely on sig­nal detec­tion theory

Sig­nal detec­tion the­o­ry – what helps pre­vent­ing mis­di­ag­noses and false pos­i­tives in gen­er­al can’t be bad for diag­nos­ing Amu­sia either, one would think. Our very own Mol­ly Hen­ry and her for­mer super­vi­sor Devin McAuley now demon­strate in a just-accept­ed paper

Fail­ure to apply sig­nal detec­tion the­o­ry to the Mon­tre­al Bat­tery of Eval­u­a­tion of Amu­sia may mis­di­ag­nose amusia

in Music Per­cep­tion that this is indeed the case:

They show that analy­ses based on con­fi­dence rat­ings and ROC-curves out­per­forms sim­ple per­cent­age cor­rect in diag­nos­ing Amusia.

Here is the abstract, and watch out for the full paper to appear soon:

This arti­cle con­sid­ers a sig­nal detec­tion the­o­ry (SDT) approach to eval­u­a­tion of per­for­mance on the Mon­tre­al Bat­tery of Eval­u­a­tion of Amu­sia (MBEA). One hun­dred fifty-five indi­vid­u­als com­plet­ed the orig­i­nal bina­ry-response ver­sion of the MBEA (n = 62) or a con­fi­dence rat­ing ver­sion (MBEA‑C; n = 93). Con­fi­dence rat­ings afford­ed con­struc­tion of empir­i­cal receiv­er oper­a­tor char­ac­ter­is­tic (ROC) curves and deriva­tion of bias-free per­for­mance mea­sures against which we com­pared the stan­dard per­for­mance met­ric, pro­por­tion cor­rect (PC), and an alter­na­tive sig­nal detec­tion met­ric, d’. Across the board, PC was taint­ed by response bias, and under­es­ti­mat­ed per­for­mance as indexed by Az, a non­para­met­ric ROC-based per­for­mance mea­sure. Sig­nal detec­tion analy­ses fur­ther revealed that some indi­vid­u­als per­form­ing worse than the stan­dard PC-based cut­off for amu­sia diag­no­sis showed large response bias­es. Giv­en that PC is con­t­a­m­i­nat­ed by response bias, this sug­gests the pos­si­bil­i­ty that cat­e­go­riz­ing indi­vid­u­als as hav­ing amu­sia or not using a PC-based cut­off may inad­ver­tent­ly mis­clas­si­fy some indi­vid­u­als with nor­mal per­cep­tu­al sen­si­tiv­i­ty as amu­sic sim­ply because they per­formed with large response bias­es. In line with this pos­si­bil­i­ty, a com­par­i­son of amu­sia clas­si­fi­ca­tion using d’- and PC-based cut­offs showed poten­tial mis­clas­si­fi­ca­tion of 33% of the exam­ined cases.