Cluster design priorities
GPU count alone does not define performance. Topology, model build, context, and scheduling must be sized together.
Budget weights, KV cache, concurrency, and context.
Validate IB/RoCE links, topology, and effective bandwidth.
Choose tensor, pipeline, or data parallelism deliberately.
Provide authentication, limits, routing, logs, and model switching.
Production delivery
Failures and recovery belong in acceptance, not only successful requests.
Standardize startup, restart, and boot recovery.
Cover GPUs, network, processes, requests, and capacity.
Measure first-token latency, output rate, concurrency, and context.
Record versions, configuration, evidence, backup, and rollback.
Define the workload before choosing models and compute
Share your target model, users, data boundary, and current environment. We will map capacity limits, delivery steps, and acceptance criteria.