Great point! Only presenting low-confidence cases would indeed introduce some bias, but the dilemma is that the human experts are actually worse than the model at high-confidence cases.

So this just becomes a case of which becomes more significant of an effect: whether the radiologists' bias overall or worse judgement on the random high-confidence cases is more pernicious to performance in general. This point was indeed not addressed by the paper, and it is definitely a worthy avenue for future study.

Thinking about AI & epistemology. Researching CV & ML as published Assistant Researcher. Studying CS @ Columbia Engineering.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adam Mehdi

Thinking about AI & epistemology. Researching CV & ML as published Assistant Researcher. Studying CS @ Columbia Engineering.