Benchmarks in Natural Language Processing (NLP)
May 18, 2021
Benchmarks helps to assess the performance of pretrained language models in various tasks. A benchmark usually consists of one or more datasets in each task. Here is the list of benchmarks in NLP.
General NLP benchmarks Benchmark Type Paper and and Link GLUE NLU Benchmark for NLU. Paper and Link XGLUE Cross-lingual NLU Benchmark for cross-lingual NLU and NLG. Paper and Link . SuperGLUE NLU Benchmark for NLU. Paper and Link . LinCE Code switching Benchmark for Code switching NLU. Paper and Link . GENIE NLG Benchmark for NLG. Paper and Link . Long Arena Efficient Transformers Benchmark for efficient transformers. Paper and Link GEM NLG Benchmark for Natural Language Generation. Paper and Link . CodeXGLUE Code NLU Benchmark for code intelligence. Paper and Link GLUECoS Code Switching Benchmark for code-switched NLP. Paper and Link DialoGLUE Dialogue Benchmark for Task-Oriented Dialogue. Paper and Link . XTREME Cross-lingual NLU Benchmark for cross-lingual NLU. Paper and Link .
Language-specific NLP benchmarks Benchmark Category Paper and Link RussianSuperGLUE Russian NLU Benchmark for Russian NLU. Paper and Link . IndicGLUE Indian NLU Benchmark for Indian NLU. Paper and Link . CLUE Chinese NLU Benchamark for Chinese NLU. Paper and Link . IndoNLU Indonesian NLU Benchmark for Indonesian NLU. Paper and Link .
Domain-specific NLP benchmarks Benchmark Category Paper and Link BLUE Biomedical NLU Benchmark for biomedical NLU. Paper and Link .com BLURB Biomedical NLU Benchmark for biomedical NLU. Paper and Link . ChineseBLUE Chinese Biomedical NLU Benchamark for Chinese biomedical NLU. Paper and Link . PharmKG Biomedical knowledge graph Benchmark for biomedical knowledge graph. Paper and Link .
