Characterizing the Expressivity of Local Attention in Transformers.

Jiaoda Li

Jiaoda Li Ryan Cotterell Characterizing the Expressivity of Local Attention in Transformers. 2026 May abs/2605.00768 CoRR https://doi.org/10.48550/arXiv.2605.00768 db/journals/corr/corr2605.html#abs-2605-00768 streams/journals/corr

Selim Jerad Anej Svete Jiaoda Li Ryan Cotterell Unique Hard Attention: A Tale of Two Sides. 977-996 2025 ACL (2) https://doi.org/10.18653/v1/2025.acl-short.76 conf/acl/2025-2 db/conf/acl/acl2025-2.html#JeradSLC25 Jiaoda Li Ryan Cotterell Characterizing the Expressivity of Fixed-Precision Transformer Language Models. 2025 NeurIPS http://papers.nips.cc/paper_files/paper/2025/hash/e9e250537b0345111d50a5f8f392cffc-Abstract-Conference.html conf/nips/2025 db/conf/nips/neurips2025.html#LiC25

Selim Jerad Anej Svete Jiaoda Li Ryan Cotterell Unique Hard Attention: A Tale of Two Sides. 2025 March abs/2503.14615 CoRR https://doi.org/10.48550/arXiv.2503.14615 db/journals/corr/corr2503.html#abs-2503-14615 streams/journals/corr

Jiaoda Li Ryan Cotterell Characterizing the Expressivity of Transformer Language Models. 2025 May abs/2505.23623 CoRR https://doi.org/10.48550/arXiv.2505.23623 db/journals/corr/corr2505.html#abs-2505-23623 streams/journals/corr

Andy Yang Anej Svete Jiaoda Li Anthony Widjaja Lin Jonathan Rawski Ryan Cotterell David Chiang 0001 Probability Distributions Computed by Hard-Attention Transformers. 2025 October abs/2510.27118 CoRR https://doi.org/10.48550/arXiv.2510.27118 db/journals/corr/corr2510.html#abs-2510-27118 streams/journals/corr

Jiaoda Li Yifan Hou Mrinmaya Sachan Ryan Cotterell What Do Language Models Learn in Context? The Structured Task Hypothesis. 12365-12379 2024 ACL (1) https://doi.org/10.18653/v1/2024.acl-long.669 https://aclanthology.org/2024.acl-long.669 conf/acl/2024-1 db/conf/acl/acl2024-1.html#LiHSC24 Jiaoda Li Jennifer C. White Mrinmaya Sachan Ryan Cotterell A Transformer with Stack Attention. 4318-4335 2024 NAACL-HLT (Findings) https://doi.org/10.18653/v1/2024.findings-naacl.269 conf/naacl/2024f db/conf/naacl/naacl2024f.html#LiWSC24 streams/conf/naacl

Jiaoda Li Jennifer C. White Mrinmaya Sachan Ryan Cotterell A Transformer with Stack Attention. 2024 abs/2405.04515 CoRR https://doi.org/10.48550/arXiv.2405.04515 db/journals/corr/corr2405.html#abs-2405-04515

Jiaoda Li Yifan Hou Mrinmaya Sachan Ryan Cotterell What Do Language Models Learn in Context? The Structured Task Hypothesis. 2024 abs/2406.04216 CoRR https://doi.org/10.48550/arXiv.2406.04216 db/journals/corr/corr2406.html#abs-2406-04216

Yifan Hou Jiaoda Li Yu Fei Alessandro Stolfo Wangchunshu Zhou Guangtao Zeng Antoine Bosselut Mrinmaya Sachan Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models. 4902-4919 2023 EMNLP https://doi.org/10.18653/v1/2023.emnlp-main.299 https://aclanthology.org/2023.emnlp-main.299 conf/emnlp/2023 db/conf/emnlp/emnlp2023.html#HouLFSZZBS23

Yifan Hou Jiaoda Li Yu Fei Alessandro Stolfo Wangchunshu Zhou Guangtao Zeng Antoine Bosselut Mrinmaya Sachan Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models. 2023 abs/2310.14491 CoRR https://doi.org/10.48550/arXiv.2310.14491 db/journals/corr/corr2310.html#abs-2310-14491

Jiaoda Li Ryan Cotterell Mrinmaya Sachan Probing via Prompting. 1144-1157 2022 NAACL-HLT https://doi.org/10.18653/v1/2022.naacl-main.84 https://aclanthology.org/2022.naacl-main.84 conf/naacl/2022 db/conf/naacl/naacl2022.html#LiCS22

Jiaoda Li Ryan Cotterell Mrinmaya Sachan Probing via Prompting. 2022 abs/2207.01736 CoRR https://doi.org/10.48550/arXiv.2207.01736 db/journals/corr/corr2207.html#abs-2207-01736

Jiaoda Li Ryan Cotterell Mrinmaya Sachan Differentiable Subset Pruning of Transformer Heads. 1442-1459 2021 9 Trans. Assoc. Comput. Linguistics https://doi.org/10.1162/tacl_a_00436 db/journals/tacl/tacl9.html#LiCS21

Jiaoda Li Duygu Ataman Rico Sennrich Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models. 8556-8562 2021 EMNLP (1) https://doi.org/10.18653/v1/2021.emnlp-main.673 https://aclanthology.org/2021.emnlp-main.673 conf/emnlp/2021-1 db/conf/emnlp/emnlp2021-1.html#LiAS21

Jiaoda Li Ryan Cotterell Mrinmaya Sachan Differentiable Subset Pruning of Transformer Heads. 2021 abs/2108.04657 CoRR https://arxiv.org/abs/2108.04657 db/journals/corr/corr2108.html#abs-2108-04657

Jiaoda Li Duygu Ataman Rico Sennrich Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models. 2021 abs/2109.03415 CoRR https://arxiv.org/abs/2109.03415 db/journals/corr/corr2109.html#abs-2109-03415

Duygu Ataman Antoine Bosselut David Chiang 0001 Ryan Cotterell Yu Fei Yifan Hou Selim Jerad Anthony Widjaja Lin Jonathan Rawski Mrinmaya Sachan Rico Sennrich Alessandro Stolfo Anej Svete Jennifer C. White Andy Yang Guangtao Zeng Wangchunshu Zhou