EPL DOCUMENTATION

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed giant model training.

EPL

• Easy Parallel Library
  • Overview
  • Examples
  • Publication
• Installation
  • Requirements
  • Install from source
• Quick Start
  • EPL Annotation
  • Launch a parallel training

API Documentation

• Parallelism Strategy API
  • Parallel Strategy Primitive
  • set_default_strategy
  • API Instruction
• Parallelism API Examples
  • Data Parallelism
  • Pipeline Parallelism
  • Tensor Model Parallelism
• Configuration
  • Pipeline Configuration
  • Gradient Checkpoint (GC) Configuration
  • Zero Configuration
  • Offload Configuration
  • Memory-efficient AMP Configuration
  • Optimizer Configuration
  • Cluster Configuration
  • Communication Configuration
  • IO Configuration
  • Auto Parallel Configuration

Tutorials

• Data Parallelism
• Pipeline Parallelism
  • Training setup.
  • Distributed Bert training
  • Evaluation
• MoE Tensor Model Parallelism
  • Training setup.
  • Distributed Training