siiRL documentation

Quickstart

• Installation
  • Requirements
  • Method 1: Install from docker image
  • Method 2: Install from PIP
  • Method 3: Install from custom environment
• Quickstart: GRPO training on GSM8K dataset
  • Introduction
  • Dataset Introduction
  • Step 1: Prepare the dataset
  • Step 2: Download a model for post-training
  • Step 3: Perform GRPO training with the instruct model

Programming guide

• siiRL Complete Architecture Guide
  • Table of Contents
  • 1. siiRL Architecture Overview
  • 2. DistFlow Design Philosophy
  • 3. Program Entry and Startup Flow
  • 4. DAG Planner Deep Dive
  • 5. DAG Worker Deep Dive
  • 6. Data Coordinator Deep Dive
  • 7. Engine Model Execution
  • 8. Core Algorithm Implementation
  • 9. Complete Execution Flow
  • 10. Configuration Parameters
  • 11. Extension Guide
  • Appendix A: Code File Navigation
  • Summary
• Code Structure
  • Directory Structure
  • Core Modules
  • Data Structures
  • Execution Flow
  • Key Concepts
  • Configuration Parameters
  • Extension Points
• siiRL’s Implementation Explained
  • Overall Implementation
  • Environment Abstraction
  • Control Flow: Pipeline
  • DataLoader and DataBuffer
  • TaskGraph Scheduling
  • Build the Distributed Process Group
  • Ray Trainer
  • DAGWorker
  • Training Initiation
• SRPO Code Implementation Explained
  • Overview: What is SRPO?
  • Code Architecture Overview
  • Training Pipeline Definition
  • Core Components Deep Dive
  • Key Configuration Parameters
  • Quick Reference: File Locations
  • References

Data Preparation

• Prepare Data for Post-Training
• Implementing Reward Functions for Datasets

User Define Interface

• Filter Interface
  • Architecture Overview
  • Built-in Filters
  • Custom Filter
• Reward Interface
  • Architecture Overview
  • Quick Start
  • Official Example: GSM8K
  • Custom Examples
  • Function Specification
• Pipeline API
  • Architecture Overview
  • Basic Usage
  • Built-in Pipelines
  • Custom Pipeline Definition
• Metrics Interface
  • Architecture Overview
  • Quick Start
  • Current Metrics System
  • Custom Metrics Implementation
  • Integration with Training Workflow
  • Advanced Examples
  • Configuration and Best Practices
  • File Structure Summary

Configurations

• Configuration Guide
  • Configuration Structure
  • Usage
  • Data Parameters
  • Actor/Rollout/Reference Model
  • Critic Model
  • Reward Model
  • Algorithm Parameters
  • Training Parameters
  • DAG Parameters
  • Complete Example
  • Parameter Reference

Example

• DeepScaleR Example with PPO
• MM-Eureka Example with GRPO
• DeepScaleR Example with CPGD
• Megatron-LM Training Backend
• Embodied SRPO Training

Hardware Support

• Ascend NPU
• Data Collection on Ascend Devices Based on the FSDP Backend
• MetaX(沐曦) GPU