Adrien Foucart, PhD in biomedical engineering.
Get sparse and irregular email updates by subscribing to https://adfoucart.substack.com. This website is guaranteed 100% human-written, ad-free and tracker-free.
It took a long time -- way longer than necessary, to be honest -- but our paper “Evaluating participating methods in image analysis challenges: lessons from MoNuSAC 2020” has finally been published in Pattern Recognition, and is now available online (doi:10.1016/j.patcog.2023.109600, open access for 50 days using this link).
It uses the published results of the MoNuSAC 2020 challenge, which include the prediction maps from some of the teams, to study how using complex metrics such as Panoptic Quality lead to poorer insights and hard to interpret results, compared to using separate, independent metrics for each sub-task (here: segmentation, classification and detection). As we demonstrate in the paper, while the ranking based on the PQ doesn’t really tell us much about the capabilities of the different algorithms, using separate metrics give us otherwise invisible insights on the results of the challenge.
The preprint version of the paper can be downloaded from this website. Only minor modifications were made for the final paywalled version.