Great point! Only presenting low-confidence cases would indeed introduce some bias, but the dilemma is that the human experts are actually worse than the model at high-confidence cases.

So this just becomes a case of which becomes more significant of an effect: whether the radiologists' bias overall or worse judgement on the random high-confidence cases is more pernicious to performance in general. This point was indeed not addressed by the paper, and it is definitely a worthy avenue for future study.

Aspiring creative in science & tech. AI & blockchain hobbyist. ML engineer at Percept Corporation. Undergrad researcher and CS & EE Major at Dartmouth.

Aspiring creative in science & tech. AI & blockchain hobbyist. ML engineer at Percept Corporation. Undergrad researcher and CS & EE Major at Dartmouth.