2024

  • On Learning to Summarize with Large Language Models as References Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Dragomir R. Radev, Arman Cohan. NAACL 2024 pdf
  • Investigating Data Contamination in Modern Benchmarks for Large Language Models Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark B. Gerstein, Arman Cohan. NAACL 2024 pdf
  • Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein. NAACL 2024 pdf
  • On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering Linyong Nan, Ellen Zhang, Weijin Zou, Yilun Zhao, Wenfei Zhou, Arman Cohan. NAACL 2024 Findings pdf
  • Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization Yixin Liu, A. R. Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq R. Joty, Pengfei Liu, Dragomir R. Radev, Chien-Sheng Wu, Arman Cohan. NAACL 2024 Findings pdf
  • On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization Lorenzo Jaime Yu Flores, Arman Cohan. EACL 2024 pdf
  • FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini. Preprint pdf
  • OLMo: Accelerating the Science of Language Models Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hanna Hajishirzi. Preprint pdf
  • Calibrating Long-form Generations from Large Language Models Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, Bhuwan Dhingra. Preprint pdf
  • Evaluating LLMs at Detecting Errors in LLM Responses Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui Zhang. Preprint pdf
  • MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain Expertise Chunyuan Deng, Xiangru Tang, Yilun Zhao, Hanming Wang, Haoran Wang, Wangchunshu Zhou, Arman Cohan, Mark Gerstein. Preprint pdf
  • Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Zhiyong Lu, Mark B. Gerstein. Preprint pdf
  • Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models Martin Riddell, Ansong Ni, Arman Cohan. Preprint pdf

2023

  • A Controllable QA-based Framework for Decontextualization Benjamin Newman, Luca Soldaini, Raymond Fok, Arman Cohan, Kyle Lo. EMNLP 2023 pdf
  • QTSumm: A New Benchmark for Query-Focused Table Summarization Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Xiangru Tang, Yumo Xu, Arman Cohan, Dragomir R. Radev. EMNLP 2023 pdf
  • Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir R. Radev. EMNLP 2023 Findings pdf
  • Medical Text Simplification: Optimizing for Readability with Unlikelihood Training and Reranked Beam Search Decoding Lorenzo Jaime Yu Flores, Heyuan Huang, Kejian Shi, Sophie Chheang, Arman Cohan. EMNLP 2023 Findings pdf
  • Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, E. Zhang, Arman Cohan, Dragomir R. Radev. EMNLP 2023 Findings pdf
  • Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan. EMNLP 2023 Findings pdf
  • Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarios Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan. EMNLP 2023 Industry Track pdf
  • Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan. EMNLP 2023 industry track pdf
  • Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering Avi Caciularu, Matthew E. Peters, J. Goldberger, Ido Dagan, Arman Cohan. ACL 2023 pdf
  • OpenRT: An Open-source Framework for Reasoning Over Tabular Data Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, Dragomir R. Radev. ACL 2023 Demo pdf
  • TESS: Text-to-Text Self-Conditioned Simplex Diffusion Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, J. Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan. EACL 2023 pdf
  • LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo. EACL 2023 (outstanding paper award) pdf
  • Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning Xiangru Tang, Arman Cohan, M. Gerstein. Clinical Natural Language Processing Workshop 2023 pdf
  • Natural language processing in radiology: Clinical applications and future directions Pratheek S Bobba, A. Sailer, J. Pruneski, Spencer Beck, Ali Mozayan, Sara Mozayan, Jennifer Arango, Arman Cohan, S. Chheang. Clinical imaging 2023 pdf
  • When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets Orion Weller, Kyle Lo, David Wadden, Dawn J Lawrie, Benjamin Van Durme, Arman Cohan, Luca Soldaini. Findings 2023 pdf
  • Inference-time Re-ranker Relevance Feedback for Neural Information Retrieval R. Reddy, Pradeep Dasigi, Md Arafat Sultan, Arman Cohan, Avirup Sil, Heng Ji, Hannaneh Hajishirzi. Preprint pdf
  • L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models Ansong Ni, Pengcheng Yin, Yilun Zhao, Martin Riddell, Troy Feng, Rui Shen, Stephen Yin, Ye Liu, Semih Yavuz, Caiming Xiong, Shafiq R. Joty, Yingbo Zhou, Dragomir R. Radev, Arman Cohan. Preprint pdf
  • ODSum: New Benchmarks for Open Domain Multi-Document Summarization Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan. Preprint pdf
  • The Semantic Scholar Open Data Platform Rodney Michael Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, D. Graham, F.Q. Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Christopher Newell, Smita R Rao, Shaurya Rohatgi, P. Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, A. Tanaka, Alex D Wade, Linda M. Wagner, Lucy Lu Wang, Christopher Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld. Preprint pdf
  • DocMath-Eval: Evaluating Numerical Reasoning Capabilities of LLMs in Understanding Long Documents with Tabular Data Yilun Zhao, Yitao Long, Hongjun Liu, Linyong Nan, Lyuhao Chen, Ryo Kamoi, Yixin Liu, Xiangru Tang, Rui Zhang, Arman Cohan. Preprint pdf
  • KnowledgeMath: Knowledge-Intensive Math Word Problem Solving in Finance Domains Yilun Zhao, Hongjun Liu, Yitao Long, Rui Zhang, Chen Zhao, Arman Cohan. Preprint pdf
  • ML-Bench: Evaluating Large Language Models for Code Generation in Repository-Level Machine Learning Tasks Yuliang Liu, Xiangru Tang, Zefan Cai, Junjie Lu, Yichi Zhang, Yan Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, Ruijun Huang, Shuzheng Si, Sheng Chen, Haozhe Zhao, Zheng Li, Liang Chen, Yiming Zong, Yan Wang, Tianyu Liu, Zhiwei Jiang, Baobao Chang, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Arman Cohan, Mark B. Gerstein. Preprint pdf
  • MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning Xiangru Tang, Anni Zou, Zhuosheng Zhang, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark B. Gerstein. Preprint pdf
  • Observable Propagation: A Data-Efficient Approach to Uncover Feature Vectors in Transformers Jacob Dunefsky, Arman Cohan. Preprint pdf

2022

  • SciFact-Open: Towards open-domain scientific claim verification David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi. EMNLP 2022 pdf
  • Generating Scientific Claims for Zero-Shot Scientific Fact Checking Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isabelle Augenstein, Lucy Lu Wang. ACL 2022 pdf
  • Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires Thong Nguyen, Andrew Yates, Ayah Zirikly, Bart Desmet, Arman Cohan. ACL 2022 pdf
  • Zero- and Few-Shot NLP with Pretrained Language Models Iz Beltagy, Arman Cohan, Robert Logan IV, Sewon Min, Sameer Singh. ACL 2022 pdf
  • Embedding Recycling for Language Models Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey. Findings 2022 pdf
  • Overview of the First Shared Task on Multi Perspective Scientific Document Summarization (MuP) Arman Cohan, Guy Feigenblat, Tirthankar Ghosal, Michal Shmueli-Scheuer. SDP 2022 pdf
  • Overview of the Third Workshop on Scholarly Document Processing Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Michal Shmueli-Scheuer, Anita de Waard, Lucy Lu Wang. SDP 2022 pdf
  • Exploring the Challenges of Open Domain Multi-Document Summarization John Giorgi, Luca Soldaini, Bo Wang, Gary D Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan. Preprint pdf
  • SciRepEval: A Multi-Format Benchmark for Scientific Document Representations Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, Sergey Feldman. Preprint pdf
  • Iterative Evidence Searching over Long Structured Documents for Question Answering Arman Cohan. Preprint pdf
  • SciRepEval: A Multi-Format Benchmark for Scientific Document Representations Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, Sergey Feldman. Preprint pdf

2021

  • Cross-Document Language Modeling Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan. EMNLP 2021 pdf
  • FLEX: Unifying Evaluation for Few-Shot NLP Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy. NeurIPS 2021 pdf
  • A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner. NAACL 2021 pdf
  • Beyond Paragraphs: NLP for Long Sequences Iz Beltagy, Arman Cohan, Hannaneh Hajishirzi, Sewon Min, Matthew E. Peters. NAACL 2021 pdf
  • Long Context Question Answering via Supervised Contrastive Learning Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan. NAACL 2021 pdf
  • Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity Sheshera Mysore, Arman Cohan, Tom Hope. NAACL 2021 pdf
  • MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting Anne Lauscher, B. Ko, Bailey Kuehl, Sophie Johnson, David Jurgens, Arman Cohan, Kyle Lo. NAACL 2021 pdf
  • PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan. ACL 2021 pdf
  • Simplified Data Wrangling with ir_datasets Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, Nazli Goharian. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2021 pdf
  • MultiVerS: Improving scientific claim verification with weak supervision and full-document context David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi. NAACL-HLT 2021 pdf
  • LongChecker: Improving scientific claim verification by modeling full-abstract context David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi. Preprint pdf
  • Overview of the Second Workshop on Scholarly Document Processing Iz Beltagy, Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Keith B. Hall, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer, A. Waard, Kuansan Wang, Lucy Lu Wang. Preprint pdf
  • PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan. Preprint pdf
  • Utilizing Evidence Spans via Sequence-Level Contrastive Learning for Long-Context Question Answering Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan. Preprint pdf

2020

  • Fact or Fiction: Verifying Scientific Claims David Wadden, Kyle Lo, Lucy Lu Wang, Shanchuan Lin, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi. EMNLP 2020 pdf
  • SLEDGE: A Simple Yet Effective Zero-Shot Baseline for Coronavirus Scientific Knowledge Search Sean MacAvaney, Arman Cohan, Nazli Goharian. EMNLP 2020 pdf
  • SPECTER: Document-level Representation Learning using Citation-informed Transformers Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld. ACL 2020 pdf
  • Ranking Significant Discrepancies in Clinical Reports Sean MacAvaney, Arman Cohan, Nazli Goharian, Ross W. Filice. European Conference on Information Retrieval 2020 pdf
  • TLDR: Extreme Summarization of Scientific Documents Isabel Cachola, Kyle Lo, Arman Cohan, Daniel S. Weld. Findings 2020 pdf
  • GUIR @ LongSumm 2020: Learning to Generate Long Summaries from Scientific Documents Sajad Sotudeh Gharebagh, Arman Cohan, Nazli Goharian. SDP 2020 pdf
  • On Generating Extended Summaries of Long Documents Sajad Sotudeh, Arman Cohan, Nazli Goharian. SDU@AAAI 2020 pdf
  • ParsiNLU: A Suite of Language Understanding Challenges for Persian Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabadi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, Yadollah Yaghoobzadeh. Transactions of the Association for Computational Linguistics 2020 pdf
  • Longformer: The Long-Document Transformer Iz Beltagy, Matthew E. Peters, Arman Cohan. Preprint pdf
  • SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search Sean MacAvaney, Arman Cohan, Nazli Goharian. Preprint pdf