NeuIRPapers

Must-read Papers on Neural Information Retrieval

Contents

• 1. Survey Papers
• 2. NeuIR Models
  • 2.1. NN Models
  • 2.2. Pretrained Models
• 3. ANN
• 4. Knowledge IR
• 5. Domain Transfer

1. Survey Papers

1. A Deep Look into Neural Ranking Models for Information Retrieval. Jiafeng Guo, Yixing Fan, Liang Pang, Liu Yang, Qingyao Ai, Hamed Zamani, Chen Wu, W. Bruce Croft, Xueqi Cheng. paper

2. NeuIR Models

NN Models

1. Learning Deep Structured Semantic Models for Web Search using Clickthrough Data. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck. CIKM 2013. paper
2. Convolutional Neural Network Architectures for Matching Natural Language Sentences. Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen. NeurIPS 2014. paper
3. Learning Semantic Representations Using Convolutional Neural Networks for Web Search. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, Grégoire Mesnil. WWW 2014. paper
4. A Deep Relevance Matching Model for Ad-hoc Retrieval. Jiafeng Guo, Yixing Fan, Qingyao Ai, W. Bruce Croft. CIKM 2016. paper
5. aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model. Liu Yang, Qingyao Ai, Jiafeng Guo, W. Bruce Croft. CIKM 2016. paper
6. Text Matching as Image Recognition. Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, Xueqi Cheng. AAAI 2016. paper
7. Machine Comprehension Using Match-LSTM and Answer Pointer. Shuohang Wang, Jing Jiang. ICLR 2017. paper
8. Learning to Match Using Local and Distributed Representations of Text for Web Search. Bhaskar Mitra, Fernando Diaz, Nick Craswell. WWW 2017. paper
9. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, Russell Power. SIGIR 2017. paper
10. Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search. Zhuyun Dai, Chenyan Xiong, Jamie Callan, Zhiyuan Liu. WSDM 2018. paper
11. Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking. Sebastian Hofstätter, Markus Zlabinger, Allan Hanbury. ECAI 2020. paper

Pretrained Models

1. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. NAACL-HLT 2019. paper
2. Understanding the Behaviors of BERT in Ranking. Yifan Qiao, Chenyan Xiong, Zhenghao Liu, Zhiyuan Liu. arxiv 2019. paper
3. SciBERT: A Pretrained Language Model for Scientific Text。 Iz Beltagy, Kyle Lo, Arman Cohan. EMNLP 2019. paper
4. Deeper Text Understanding for IR with Contextual Neural Language Modeling. Zhuyun Dai, Jamie Callan. SIGIR 2019. paper
5. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. arxiv 2019. paper
6. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. Kevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. Manning. ICLR 2020. paper

3. ANN

1. Complementing Lexical Retrieval with Semantic Residual Embedding. Luyu Gao, Zhuyun Dai, Zhen Fan, Jamie Callan. arxiv 2020. paper
2. Dense Passage Retrieval for Open-Domain Question Answering. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih. arxiv 2020. paper
3. Latent Retrieval for Weakly Supervised Open Domain Question Answering. Kenton Lee, Ming-Wei Chang, Kristina Toutanova ACL 2019. paper
4. Pre-training Tasks for Embedding-based Large-scale Retrieval. Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, Sanjiv Kumar. arxiv 2020. paper
5. Sparse, Dense, and Attentional Representations for Text Retrieval. Yi Luan, Jacob Eisenstein, Kristina Toutanova, Michael Collins. arxiv 2020. paper
6. REALM: Retrieval-Augmented Language Model Pre-Training. Yi Luan, Jacob Eisenstein, Kristina Toutanova, Michael Collins. arxiv 2020. paper

4. Knowledge IR

1. Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding. Chenyan Xiong, Russell Power, Jamie Callan. WWW 2017. paper
2. Word-entity duet representations for document ranking. Chenyan Xiong, Jamie Callan, Tie-Yan Liu. SIGIR 2017. paper
3. JointSem: Combining Query Entity Linking and Entity based Document Ranking. Chenyan Xiong, Zhengzhong Liu, Jamie Callan, Eduard Hovy. CIKM 2017. paper
4. Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling. Chenyan Xiong, Zhengzhong Liu, Jamie Callan, Tie-Yan Liu. SIGIR 2018. paper
5. Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval. Zhenghao Liu, Chenyan Xiong, Maosong Sun, Zhiyuan Liu. ACL 2018. paper

5. Domain Transfer

1. Meta-Learning in Neural Networks: A Survey. Timothy Hospedales, Antreas Antoniou, Paul Micaelli, Amos Storkey. arXiv 2020. paper
2. Content-Based Weak Supervision for Ad-Hoc Re-Ranking. Sean MacAvaney, Andrew Yates, Kai Hui, Ophir Frieder. SIGIR 2019. paper
3. Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. Wei Yang, Kuang Lu, Peilin Yang, Jimmy Lin. SIGIR 2019. paper
4. Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise. Dan Hendrycks, Mantas Mazeika, Duncan Wilson, Kevin Gimpel. NeurIPS 2018. paper
5. Learning to Reweight Examples for Robust Deep Learning. Mengye Ren, Wenyuan Zeng, Bin Yang, Raquel Urtasun. ICML 2018. paper
6. On the Theory of Weak Supervision for Information Retrieval. Hamed Zamani, W. Bruce Croft. CTIR 2018. paper
7. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, Masashi Sugiyama. NeurIPS 2018. paper
8. Neural Ranking Models with Weak Supervision. Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, JaapKamps, W.Bruce. SIGIR 2017. paper
9. Learning to Learn from Weak Supervision by Full Supervision. Mostafa Dehghani, Aliaksei Severyn, Sascha Rothe, JaapKamps. NeurIPS 2017. paper
10. Training Deep Ranking Model with Weak Relevance Labels. ChengLuo, YukunZheng, JiaxinMao, YiqunLiu, MinZhang, ShaopingMa. ADC 2017. paper
11. Selective Weak Supervision for Neural Information Retrieval. Kaitao Zhang, Chenyan Xiong, Zhenghao Liu, Zhiyuan Liu. WWW 2020. paper