Medical Imaging Computing researchers: Meet the Challenges Framework Flagship Node!
Published: 2022-05-24
The Challenges Framework flagship Node organizes a series of Grand Challenges in Medical Image Analysis (www.grand-challenge.org), focused at standardized evaluation of different image analysis algorithms in the field. The goal of this Node is to create a generic open-source platform that allows MIC (Medical Imaging Computing) researchers to make their data and evaluated software available for on-line collaboration and cloud computing. Bram van Ginneken, professor of medical image analysis at Radboud University Medical Center in Nijmegen, The Netherlands, explains the Node’s new approach to running challenges.
Our Node physically consists of storage, computational resources, and web access. It includes a repository with relevant data for challenges, software for download of training data, upload of results, and display of results.
In 2021, we introduced a new way of running challenges on Grand Challenge. Rather than submitting predictions based on public test data, participants can now upload their algorithms directly to a challenge. Our platform takes the algorithms, runs them on secret test data and evaluates the resulting predictions against a secret ground truth. Enabling algorithm submission for challenges was a huge step forward: it makes for a much fairer and objective comparison of the developed machine learning solutions than the prediction submission feature we previously relied on. Since the algorithms needs to be uploaded to Grand Challenge for submission, it also ensures that they are centrally stored and remain accessible even after the challenge is finished. We have hosted 4 such challenges so far and expect to be hosting many more in the coming months and years.
New cloud-based services
2021 also marked the beginning of our collaboration with Amazon Web Services (AWS). Running our services in the cloud, sponsored by AWS, makes our platform more scalable and opens up exciting opportunities for further development of the platform.
On October 19 and November 2, 2021, we hosted an online workshop on how to create and host an algorithm container on Grand Challenge. The event was very well received with almost 50 participants from both within and outside our organization. Our next event will be an open webinar on Grand Challenge in October 2022.
Open access to the Grand Challenges platform is available via Euro-BioImaging. We hope you'll participate in one of the 50 current open challenges on this platform, or organize one yourself, and share the data and your algorithm so it can be run by all our users, over 70,000 and counting!
