Speaker
Description
Biodiversity monitoring provides essential insights into ecosystem dynamics, enabling informed decision-making for conservation and sustainable development. A key component of this effort is the automated classification of species from imagery, a task where deep learning models have recently achieved significant progress. However, fine-grained visual classification remains a challenge when species from the same genus or family share nearly identical physical traits. To improve these distinctions, we propose a text-enhanced representation learning (TERL) framework that incorporates short, human-produced descriptive texts of a species' salient features as auxiliary information during training. The approach is motivated by the assumption that such textual descriptions provide complementary and often more discriminative cues than visual features alone. Our method teaches the model to link visual patterns with these descriptive traits, making it easier for the model to distinguish between look-alike species. TERL is model-agnostic and can be integrated with any deep learning-based visual classifier, leading to model decision-making that is better aligned with human perception. Notably, the method requires text supervision only during the training phase. Since there is no additional computational overhead during inference, the final model remains a fast, image-only tool that is readily applicable for field researchers. We evaluate the approach on fine-grained ecological images, including camera trap images of moths collected as part of the LEPMON project.
| Status Group | Doctoral Researcher |
|---|---|
| FOR TALKS: Poster Presentation Option | Yes, I’m willing to present as a poster. |