1 min readJul 22, 2020
Great point! Only presenting low-confidence cases would indeed introduce some bias, but the dilemma is that the human experts are actually worse than the model at high-confidence cases.
So this just becomes a case of which becomes more significant of an effect: whether the radiologists' bias overall or worse judgement on the random high-confidence cases is more pernicious to performance in general. This point was indeed not addressed by the paper, and it is definitely a worthy avenue for future study.