Cluster Architecture

Multi-GPU Parallelism Split Planner

Model division tool for distributed setups. Plan how model layers partition across Tensor Parallelism (TP) and Pipeline Parallelism (PP) ranks.

Topology Design

Configure your target model size and GPU cluster structure.

Target LLM Architecture

Total Available GPUs 8 GPUs

The physical GPU pool size (e.g. 8x H100 SXM nodes).

Tensor Parallelism (TP Rank) 2

Shards individual layer matrices (Attention/FFN) inside the same node.

Pipeline Parallelism (PP Rank) 4

Slices the transformer layer stack sequentially across network boundaries.

Data Parallelism (DP) 1 Concurrent batch replicas

Layers / PP Stage 20 Layer chunk per GPU node

Topology Check Valid Setup

TP * PP matches available GPU pool.