approved
Synchronization is All You Need - Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairs

We consider the problem of transferring a temporal action segmentation system initially designed for exocentric (fixed) cameras to an egocentric scenario, where wearable cameras capture video data. The conventional supervised approach requires the collection and labeling of a new set of egocentric videos to adapt the model, which is costly and time-consuming. Instead, we propose a novel methodology which performs the adaptation leveraging existing labeled exocentric videos and a new set of unlabeled, synchronized exocentric-egocentric video pairs, for which temporal action segmentation annotations do not need to be collected. We implement the proposed methodology with an approach based on knowledge distillation, which we investigate both at the feature and Temporal Action Segmentation model level. Experiments on Assembly101 and EgoExo4D demonstrate the effectiveness of the proposed method against classic unsupervised domain adaptation and temporal alignment approaches. Without bells and whistles, our best model performs on par with supervised approaches trained on labeled egocentric data, without ever seeing a single egocentric label, achieving a +15.99 improvement in the edit score (28.59 vs 12.60) on the Assembly101 dataset compared to a baseline model trained solely on exocentric data. In similar settings, our method also improves edit score by +3.32 on the challenging EgoExo4D benchmark. Code is available online.

Tags
Data and Resources
To access the resources you must log in
  • Synchronization is All You Need: ...

    Official repository of the paper "Synchronization is All You Need:...

    The resource: 'Synchronization is All You ...' is not accessible as guest user. You must login to access it!
Additional Info
Field Value
Accessibility OnLine
AccessibilityMode Download
Associate Project FAIR
Basic rights Download
CreationDate 2025-03-24
Creator Farinella, Giovanni, giovanni.farinella@unict.it, orcid.org/0000-0002-6034-0432
Field/Scope of use Any use
Group Others
Owner Farinella, Giovanni, giovanni.farinella@unict.it, orcid.org/0000-0002-6034-0432
Programming Language Python
SoBigData Node SoBigData IT
Sublicense rights No
Territory of use World Wide
Thematic Cluster Other
system:type Method
Management Info
Field Value
Author Farinella Giovanni Maria
Maintainer Farinella Giovanni Maria
Version 1
Last Updated 22 June 2025, 01:07 (CEST)
Created 22 June 2025, 01:07 (CEST)