Machine Learning in Montpellier, Theory & Practice
Computer vision tasks are frequently trained using distinctly annotated datasets although the tasks are often related to one another. Multi-task learning, on the other hand, is known to generally improve performance by leveraging the inter-task relationship but often at the high cost of maintaining all-task annotations for each training example. The multi-task partially supervised learning paradigm relaxes this requirement, allowing each input to be annotated only for one of the target tasks. This setting hinders learning joint-task representations yet is potentially helpful for data scarcity scenarios as datasets for individual tasks can be combined for training. In this presentation, we study multi-task relationships for better data exploitation under the partial supervision assumption for the two well-known computer vision tasks, namely object detection and semantic segmentation. Both of these tasks are designed for scene understanding yet differ in data structure and information level: object detection requires box coordinates for object instances while semantic segmentation requires pixel-wise regional categories. To that end, we propose Box-for-Mask and Mask-for-Box strategies to extract information from one task’s annotations to train the other. Ablation studies and experimental results on VOC and COCO datasets show favorable results for the proposed idea.