Authors: Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, Joost van de Weijer.
Submitted preprint, 2020
Abstract: For future learning systems incremental learning is desirable, because it allows for: efficient resource usage by eliminating the need to retrain from scratch at the arrival of new data; reduced memory usage by preventing or limiting the amount of data required to be stored – also important when privacy limitations are imposed; and learning that more closely resembles human learning. The main challenge for incremental learning is catastrophic forgetting, which refers to the precipitous drop in performance on previously learned tasks after learning a new one. Incremental learning of deep neural networks has seen explosive growth in recent years. Initial work focused on task incremental learning, where a task-ID is provided at inference time. Recently we have seen a shift towards class-incremental learning where the learner must classify at inference time between all classes seen in previous tasks without recourse to a task-ID. In this paper, we provide a complete survey of existing methods for incremental learning, and in particular we perform an extensive experimental evaluation on twelve class-incremental methods. We consider several new experimental scenarios, including a comparison of class-incremental methods on multiple large-scale datasets, investigation into small and large domain shifts, and comparison on various network architectures.
Find out more
Authors: Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, Tinne Tuytelaars.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021
Abstract: Artificial neural networks thrive in solving the classification problem for a particular rigid task, where the network resembles a static entity of knowledge, acquired through generalized learning behaviour from a distinct training phase. However, endeavours to extend this knowledge without targeting the original task usually result in a catastrophic forgetting of this task. Continual learning shifts this paradigm towards a network that can continually accumulate knowledge over different tasks without the need for retraining from scratch, with methods in particular aiming to alleviate forgetting. We focus on task-incremental classification, where tasks arrive in a batch-like fashion, and are delineated by clear boundaries. Our main contributions concern 1) a taxonomy and extensive overview of the state-of-the-art, 2) a novel framework to continually determine stability-plasticity trade-off of the continual learner, 3) a comprehensive experimental comparison of 10 state-of-the-art continual learning methods and 4 baselines. We empirically scrutinize which method performs best, both on balanced Tiny Imagenet and a large-scale unbalanced iNaturalist datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time and storage.
Find out more
Authors: Corina Kräuter, Ursula Reiter, Clemens Reiter, Volha Nizhnikava, Marc Masana, Albrecht Schmidt, Michael Fuchsjäger, Rudolf Stollberger, Gert Reiter.
Magnetic Resonance in Medicine (MRM), 2020
Abstract: Purpose: To present and validate a method for automated extraction and analysis of the temporal evolution of the mitral valve (MV) vortex ring from MR 4D‐flow data. Methods: The proposed algorithm uses the divergence‐free part of the velocity vector field for Q criterion‐based identification and tracking of MV vortex ring core and region within the left ventricle (LV). The 4D‐flow data of 20 subjects (10 healthy controls, 10 patients with ischemic heart disease) were used to validate the algorithm against visual analysis as well as to assess the method’s sensitivity to manual LV segmentation. Quantitative MV vortex ring parameters were analyzed with respect to both their differences between healthy subjects and patients and their correlation with transmitral peak velocities. Results: The algorithm successfully extracted MV vortex rings throughout the entire cardiac cycle, which agreed substantially with visual analysis (Cohen’s kappa =0.77). Furthermore, vortex cores and regions were robustly detected even if a static end-diastolic LV segmentation mask was applied to all frames (Dice coefficients 0.82 ± 0.08 and 0.94 ± 0.02 for core and region, respectively). Early diastolic MV vortex ring vorticity, kinetic energy and circularity index differed significantly between healthy controls and patients. In contrast to vortex shape parameters, vorticity and kinetic energy correlated strongly with transmitral peak velocities. Conclusion: An automated method for temporal MV vortex ring extraction demonstrating robustness with respect to LV segmentation strategies is introduced. Quantitative vortex parameter analysis indicates importance of the MV vortex ring for LV diastolic (dys)function.
Find out more
Authors: Aymen Azaza, Joost van de Weijer, Ali Douik, Javad Zolfaghari, Marc Masana.
SN Computer Science (Springer), 2020
Abstract: Top-down semantic information is known to play an important role in assigning saliency. Recently, large strides have been made in improving state-of-the-art semantic image understanding in the fields of object detection and semantic segmentation. Therefore, since these methods have now reached a high-level of maturity, evaluation of the impact of high-level image understanding on saliency estimation is now feasible. We propose several saliency features which are computed from object detection and semantic segmentation results. We combine these features with a standard baseline method for saliency detection to evaluate their importance. Experiments demonstrate that the proposed features derived from object detection and semantic segmentation improve saliency estimation significantly. Moreover, they show that our method obtains state-of-the-art results on (FT, ImgSal, and SOD datasets) and obtains competitive results on four other datasets (ECSSD, PASCAL-S, MSRA-B, and HKU-IS).
Find out more
Authors: Jorge Bernal, Aymeric Histace, Marc Masana, Quentin Angermann, Cristina Sánchez-Montes, Cristina Rodríguez de Miguel, Maroua Hammami, Ana García-Rodríguez, Henry Córdova, Olivier Romain, Gloria Fernández-Esparrach, Xavier Dray, F. Javier Sánchez.
International Journal of Computer Assisted Radiology and Surgery (IJCARS), 2018
Abstract: Purpose: Method evaluation for decision support systems for health is a time consuming task. To assess performance of polyp detection methods in colonoscopy videos, clinicians have to deal with the annotation of thousands of images. Current existing tools could be improved in terms of flexibility and ease of use. Methods: We introduce GTCreator, a flexible annotation tool for providing image and text annotations to image-based datasets. It keeps the main functionalities of other similar tools while extending other capabilities such as allowing multiple annotators to work simultaneously on the same task or enhanced dataset browsing. Results: The comparison with other similar tools show that GTCreator allows to obtain fast and precise annotation of image datasets, being the only one which offers full annotation editing and browsing capabilites. As a use case of our tool, we present three different benchmarks generated using our tool covering all stages in polyp characterization: detection, segementation and histology prediction. Conclusions: Our proposed annotation tool has been proven to be efficient for large image dataset annotation, as well as showing potential of use in other stages of method evaluation such as experimental setup or results analysis.
Find out more
Authors: Aymen Azaza, Joost van de Weijer, Ali Douik, Marc Masana.
Journal on Computer Vision and Image Understanding (CVIU), 2018
Abstract: One of the fundamental properties of a salient object region is its contrast with the immediate context. The problem is that numerous object regions exist which potentially can all be salient. One way to prevent an exhaustive search over all object regions is by using object proposal algorithms. These return a limited set of regions which are most likely to contain an object. Several saliency estimation methods have used object proposals. However, they focus on the saliency of the proposal only, and the importance of its immediate context has not been evaluated. In this paper, we aim to improve salient object detection. Therefore, we extend object proposal methods with context proposals, which allow to incorporate the immediate context in the saliency computation. We propose several saliency features which are computed from the context proposals. In the experiments, we evaluate five object proposal methods for the task of saliency segmentation, and find that Multiscale Combinatorial Grouping outperforms the others. Furthermore, experiments show that the proposed context features improve performance, and that our method matches results on the FT datasets and obtains competitive results on three other datasets (PASCAL-S, MSRA-B and ECSSD).
Find out more
