Harnessing Differentiable Data Models for Machine Learning Integration in Microscopy


Published August 1, 2023

Imaging technologies are becoming increasingly complex and ever more expensive, reducing the general accessibility and potential reach of cutting-edge techniques. The Special Edition Virtual Pub “Open Hardware in Imaging,” in collaboration with the Euro-BioImaging Industry Board, will highlight developments from scientists and companies who are committed to making biological & biomedical imaging hardware and software solutions openly available to a wide audience.

When: September 22, 2023, from 13:00-15:00 CEST

Where: Online

At this event, Luis Oala, Dotphoton, will present Harnessing Differentiable Data Models for Machine Learning Integration in Microscopy - (full abstract below). Hear this talk and others like it on September 22!

Full program

Register

ABSTRACT

Harnessing Differentiable Data Models for Machine Learning Integration in Microscopy

Luis Oala

Dotphoton

The integration of machine learning with data acquisition in microscopy has emerged as a promising frontier. This abstract summarizes key concepts, results, and implications from a study exploring the potential of physically accurate, differentiable data models in enhancing image acquisition and model training.

Our study introduces drift synthesis, forensics, and optimization, all underpinned by a differentiable data model. Drift synthesis creates physically faithful test cases for model selection. Forensics identifies areas for performance improvement. Optimization employs machine learning to optimize image processing by backpropagating from the task model through the ISP to the raw sensor data.

The microscopy experiments demonstrate the effectiveness of these concepts. Task models trained on physically faithful data models outperformed those trained on common corruptions. Changes in the black level configuration and denoising parameters pose the greatest risk for task model performance. The drift optimization experiment demonstrated how raw data and a differentiable data model can be used to identify unfavorable data models that should be avoided during task model deployment.

These findings suggest that machine-optimized processing could lower hardware costs, making high-quality ISPs more affordable. The use of physically accurate data models for dataset drift validation underscores the importance of precise data models in maintaining task model performance.

In the context of the Open Hardware in Imaging workshop, these findings highlight the potential of machine learning and physically accurate, differentiable data models to innovate image acquisition and model training in microscopy. The differentiable nature of the data model allows for the integration of classical machine learning with data acquisition, opening up new possibilities for research and application in microscopy.

For more details, refer to the full paper Oala, Luis, Marco Aversa, Gabriel Nobis, Kurt Willis, Yoan Neuenschwander, Michèle Buck, Christian Matek et al. "Data Models for Dataset Drift Controls in Machine Learning With Optical Images." Transactions on Machine Learning Research (2023).


More news from Euro-BioImaging

CanSERV user story

August 27, 2025

Multiplex imaging of lymph nodes: How Euro-BioImaging is advancing cancer research through the canSERV funding

The immune system plays an important role in fighting cancer but can also be a pathway of metastasis – therefore understand the interactions of…

August 26, 2025

Euro-BioImaging at IUIS 2025: Building Bridges with the Immunology Community

Vienna, August 17–22, 2025 – More than 5,000 researchers from over 150 countries gathered in Vienna for the 19th International Congress of Immunology…

Euro-BioImaging booth at Turku PET Symposium 2025

August 11, 2025

Euro-BioImaging at Turku PET Symposium 2025

In June 2025, Euro-BioImaging team members were delighted to attend the Turku PET Symposium, organised by the Turku PET Centre, part of our Finnish…