arXiv:2602.14285v2 Announce Type: replace-cross

Abstract: Automated scholarly paper review (ASPR) has entered the coexistence phase with traditional peer review, where artificial intelligence (AI) systems are increasingly incorporated into real-world manuscript evaluation. In parallel, research on automated and AI-assisted peer review has proliferated. Despite this momentum, empirical progress remains constrained by several critical limitations in existing datasets. While reviewers routinely evaluate figures, tables, and complex layouts to assess scientific claims, most existing datasets remain overwhelmingly text-centric. This bias is reinforced by a narrow focus on data from computer science publications. Furthermore, existing datasets rarely preserve precise alignment between review comments and specific manuscript versions, obscuring the iterative relationship between peer review and manuscript evolution. In response, we introduce FMMD, a multimodal and multidisciplinary open peer review dataset curated from F1000Research. The dataset addresses the current limitations by integrating manuscript-level visual and structural data with version-specific reviewer reports and editorial decisions. By explicitly aligning review comments with the exact article version under review, FMMD enables granular analysis of the peer review lifecycle. Importantly, its coverage of F1000Research extends ASPR research beyond its traditional focus on computer science to a diverse range of scientific disciplines. FMMD supports a range of research tasks, including visual-semantic consistency classification, figure-related review comment generation, and editorial decision prediction based on multimodal manuscript inputs, thereby providing a comprehensive empirical resource for developing and evaluating multimodal ASPR systems and advancing peer review research.