Argonne National Laboratory

Research Highlight | Center for Nanoscale Materials

Software pipeline for automated extraction and annotation of image datasets from science journals
In a Patterns paper, scientists present a toolkit for automatic extraction and annotation of images from the scientific literature by use of rule-based natural language processing and image recognition.
Overview of the EXSCLAIM! pipeline: The journal scraper extracts raw figure-caption pairs from journal articles. The caption distributor divides caption text into segments that are consistent with the images in the figure. The figure separator computes bounding boxes that separate and classify the individual images from the full figure. Finally, the dataset constructor processes all prior information to create a structured self-labeled dataset.

Scientific Achievement

The EXSCLAIM! toolkit was developed for automatic extraction and annotation of images from the scientific literature employing natural language processing and image recognition. This tool was used for the creation of a large electron microscopy dataset with keyword annotations.

Significance and Impact

While EXSCLAIM! is designed for extraction and self-labelling of electron microscopy images of materials, its methodology is transferable to diverse scientific domains generating high volumes of image data.

Research Details

  • Quantitative performance measures for image classification and keyword labeling accuracy are provided for both a crowdsourced-labeled dataset and an extracted dataset of nanostructure figures from Nature-family journals.

DOI: 10.1016/j.patter.2023.100843

Download this highlight

    About Argonne’s Center for Nanoscale Materials
    The Center for Nanoscale Materials is one of the five DOE Nanoscale Science Research Centers, premier national user facilities for interdisciplinary research at the nanoscale supported by the DOE Office of Science. Together the NSRCs comprise a suite of complementary facilities that provide researchers with state-of-the-art capabilities to fabricate, process, characterize and model nanoscale materials, and constitute the largest infrastructure investment of the National Nanotechnology Initiative. The NSRCs are located at DOE’s Argonne, Brookhaven, Lawrence Berkeley, Oak Ridge, Sandia and Los Alamos National Laboratories. For more information about the DOE NSRCs, please visit https://​sci​ence​.osti​.gov/​U​s​e​r​-​F​a​c​i​l​i​t​i​e​s​/​U​s​e​r​-​F​a​c​i​l​i​t​i​e​s​-​a​t​-​a​-​G​lance.

    Argonne National Laboratory seeks solutions to pressing national problems in science and technology by conducting leading-edge basic and applied research in virtually every scientific discipline. Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science.

    The U.S. Department of Energy’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit https://​ener​gy​.gov/​s​c​ience.