Speaker
Description
Digitized herbarium collections represent a crucial repository of botanical data, offering invaluable historical records for biodiversity research. However, manually transcribing textual information from specimen labels remains a labor-intensive and time-consuming bottleneck for global herbaria, largely due to the high variation and unique styles of handwritten scripts. To address this challenge, we present an end-to-end automated system designed to significantly accelerate information extraction from digitized herbarium labels. This functional pipeline optimizes processing efficiency by sequentially integrating object detection with multimodal transcription and structured parsing.
In the first stage, the system performs automated object detection to localize text labels and relevant metadata regions within high-resolution images, effectively isolating meaningful non-biological components—such as labels, stamps, and barcodes—from the physical plant specimen. Subsequently, the localized image regions are processed using multimodal large language models (LLMs). Leveraging their powerful contextual understanding and generalization capabilities, LLMs perform simultaneous text recognition and semantic parsing of key botanical attributes, including species name, collector, date, and geographic location. While commercial LLM APIs currently yield the highest accuracy, they raise concerns regarding data privacy and long-term recurring costs. To address these limitations, we develop approaches to utilize locally deployed open-weight models as a viable alternative.
Our evaluations show that the pipeline effectively handles variations in label placement and categories, while reliably transcribing text from both printed and handwritten labels. By integrating object detection with LLMs, this workflow significantly mitigates the human effort and temporal costs associated with specimen digitization, ultimately accelerating the public accessibility of structured biodiversity data.
| Status Group | Doctoral Researcher |
|---|---|
| FOR TALKS: Poster Presentation Option | Yes, I’m willing to present as a poster. |