Long Range Arena: A Benchmark For Efficient Transformers

A benchmark suite for long range attention models. Components:

  • ListOps
    • Modeling of long character sequences involving hierarchical brackets
  • Text
    • classify IMDB reviews truncated/padded to 4k characters
  • Retrieval
    • classify the similarity of document vectors from 4k excerpts of computational linguistics papers
  • Image
    • classify cifar-10 grayscale images flattened from 32x32 to 1024 bytes
  • Path
    • classify a flattened 32x32 image with two large dots as being connected by a dashed path or not
  • Path-X
    • classify a flattened 128x128 image with two large dots as being connected by a dashed path or not
  • Avg
    • (all of the above)