Computing

High Performance Computing for Simulation

Web Application Software Design

Fusion Cloud data processing sulotion

Deep Learning/ Machine Learning

High Performance Computing Service

Optical Communication: Devices to System  

SkyPhy GPU Cluster Engineering

Overview

SkyPhy specializes in full lifecycle large-scale GPU cluster construction, high-performance computing (HPC) infrastructure construction, and generative AI model inference performance optimization. Covering the full technical chain from bare metal server initialization, high-speed RDMA interconnection network engineering to stable online production inference service deployment, SkyPhy converts scattered GPU accelerator hardware resources into high utilization, high throughput, low latency, cost-controllable standardized AI computing infrastructure.

For enterprises, cloud operators and research institutions engaged in large model training, fine-tuning and generative inference businesses, traditional GPU cluster construction often faces pain points including unreasonable network topology, insufficient parallel storage performance, unoptimized communication libraries, low model serving throughput and high token unit cost. SkyPhy relies on deep accumulation in underlying hardware interconnection and AI computing scheduling to deliver end-to-end customized GPU cluster overall solutions, eliminating multi-layer technical bottlenecks for customers and maximizing the return on GPU hardware investment.

Core Capability Modules

1. Full-Lifecycle GPU Cluster Deployment & Stable Operations

We provide one-stop multi-node GPU cluster construction and long-term operation maintenance services, covering all links from hardware delivery to daily stable operation:

•          Bare-metal unified batch provisioning: Complete hardware inspection, hardware fault screening, standard rack deployment and power planning for GPU servers of H100, H200 and other high-end models;

•          System & driver stack standardized deployment: Customized optimized Linux operating system kernels, GPU driver matching, CUDA toolkit environment automatic deployment, conflict-free multi-version environment isolation;

•          Cluster orchestration platform construction: Complete scheduling platform deployment, resource label classification, GPU resource isolation and resource quota control;

•          Full-dimensional real-time monitoring system: Build monitoring metrics covering GPU utilization, memory occupancy, network bandwidth, storage IO, node temperature, link packet loss, alarm automatic notification and fault location tracing;

•          Day-2 long-term operation service: Regular cluster performance inspection, automatic batch patch update, hardware fault rapid response, business migration auxiliary support, periodic performance tuning iteration.

2. Enterprise-Grade High-Performance HPC Infrastructure Construction

Targeting large model pre-training, scientific computing, parallel simulation and other heavy parallel workloads, we build standardized high-performance HPC underlying infrastructure with complete scheduling, storage and networking capabilities:

•          Dual scheduling framework deployment and tuning: Support Slurm for traditional HPC batch tasks and Kubernetes for cloud-native AI inference workloads, realize cross-framework resource unified management, topology-aware task scheduling placement, avoid cross-rack high-latency communication;

•          High-throughput parallel distributed storage architecture: Match GPUDirect Storage technology to realize direct high-speed data access between GPU and storage devices, build multi-tier storage combination of high-speed cache disk and large-capacity persistent storage, solve training data reading IO bottlenecks;

•          Low-latency lossless network architecture planning: Match the computing scale to design multi-layer network topology, isolate computing traffic, storage traffic and management traffic, eliminate network competition jitter;

•          Resource elastic scheduling strategy: Realize dynamic resource scaling according to task load, idle GPU resource pooling reuse, improve overall cluster resource utilization.

3. Professional RDMA High-Speed Interconnection Network Engineering

RDMA interconnection network is the core bottleneck restricting large-scale GPU cluster scaling, and also SkyPhy’s core technical competitive advantage. We own full-stack technical capabilities for InfiniBand and RoCE v2 lossless networks:

•          Complete end-to-end fiber fabric overall design: Customize network topology according to cluster node scale, including leaf-spine multi-layer switching architecture, multi-rail link parallel planning, reasonable bandwidth allocation between racks and within racks;

•          Topology-aware standardized wiring construction: Standardize fiber routing, cable length matching, signal loss control, avoid long-distance signal attenuation causing packet loss and performance attenuation;

•          Deep optimization of NCCL & SHARP communication libraries: Target multi-GPU distributed training communication scenarios, adjust NCCL communication parameters, enable SHARP aggregation switching acceleration, reduce cross-node communication overhead, significantly improve multi-machine parallel training speed;

•          Large-scale multi-rail interconnection deployment: Deploy multi-link parallel RDMA architecture for ultra-large GPU clusters, expand aggregate communication bandwidth, eliminate single link bandwidth bottleneck;

•          Network fault diagnosis and performance tuning: Locate packet loss, congestion, bandwidth mismatch and other hidden network faults, optimize switch buffer, priority flow control, lossless network parameters to stabilize long-running large-scale distributed training tasks.

4. Large Model Inference Acceleration & Comprehensive Optimization

Aiming at online LLM multimodal model inference business, we provide full-stack serving layer optimization services to improve throughput and reduce reasoning cost per token:

•          Multiple mainstream inference engine deep tuning: Carry out targeted parameter optimization for vLLM, SGLang, TensorRT-LLM mainstream inference frameworks, including batch size dynamic adjustment, engine quantization configuration, weight compression optimization;

•          KV Cache intelligent scheduling strategy design: Realize dynamic KV Cache reuse, page cache management, cache space automatic recycling, reduce GPU memory occupation and support more concurrent user requests under the same hardware scale;

•          Dynamic batch & request scheduling optimization: Build multi-model mixed deployment routing system, realize automatic traffic diversion of different size models, priority scheduling of low-latency real-time requests, balance GPU load of each node;

•          Token cost optimization scheme: Combine quantization, speculative decoding, dynamic batching and cache reuse technologies to lower the average computing cost of generating a single token, improve long-term revenue of online inference business;

•          Multimodal unified inference platform construction: Support text, image, audio and video multimodal model co-deployment, unified request gateway and resource isolation control.

Core Competitive Edge: Large-Scale Industrial-Grade RDMA Deployment Capability

Most AI computing service providers only focus on GPU hardware supply and simple cluster assembly, while ignoring the core interconnection network layer that determines the upper limit of cluster scalability. SkyPhy has completed multiple large-scale InfiniBand Quantum-2 NDR and RoCE v2 industrial deployment projects in the region, accumulating mature engineering experience in ultra-large-scale lossless network construction.

We do not only deploy GPU servers, but build a full-performance matching high-speed interconnection fabric for the cluster. Through accurate topology design, multi-rail bandwidth expansion, communication library deep tuning and lossless network fine-tuning, we fundamentally solve the problem of communication bottleneck when dozens, hundreds or even thousands of GPUs work in parallel, ensure that distributed training and high-concurrency inference can maintain linear acceleration effect with the expansion of cluster scale, and avoid the waste of high-end GPU computing power caused by network lag.

Three Standardized Business Cooperation Modes

1. Large-Scale H200-Class GPU Bare-Metal Long-Term Leasing

We continuously and stably source high-performance GPU bare-metal server resources for long-term production business deployment:

•          Hardware configuration: Dedicated physical servers equipped with NVIDIA H100 / H200 high-end GPUs, pre-installed InfiniBand Quantum-2 NDR or RoCE v2 high-speed RDMA interconnection cards;

•          Contract cycle: Provide 1–2 year fixed long-term leasing contracts, reject unstable spot short-term resources, guarantee stable hardware resource supply for customer core production inference and HPC training workloads;

•          Flexible delivery methods: Support single-node small-scale deployment, multi-node cluster batch delivery and full rack exclusive resource commitment modes, match different business scale demands of startups and large enterprises;

•          Supporting value-added services: Free preliminary network topology planning, basic cluster environment pre-deployment, regular hardware inspection and fault quick replacement support.

2. Full Set GPU Cluster Engineering & Custom Optimization Services

Facing cloud computing operators, AI enterprises and research institutions with self-owned hardware resources, we provide full engineering and performance optimization services for self-built GPU clusters:

•          Interconnection network overall planning and design: Select InfiniBand or RoCE v2 solution according to customer budget and business latency requirements, output complete topology drawing, wiring scheme and switch configuration list;

•          Full cluster construction implementation: Complete rack deployment, server system installation, driver environment configuration, scheduling platform deployment, monitoring system access and unified testing acceptance;

•          Inference service full-stack optimization: Tune inference engine parameters, build multi-model routing gateway, design KV Cache and batch scheduling strategies, improve online service throughput and reduce delay;

•          Long-term performance iteration service: Monthly cluster performance evaluation, targeted bottleneck optimization, adapt to customer business growth and model iteration demands.

3. Independent RDMA Interconnection Network Consulting & Troubleshooting Service

For teams that have built GPU clusters but encountered obvious network bottlenecks in distributed training or inference, we provide independent RDMA network professional consulting services:

•          InfiniBand / RoCE v2 fabric architecture review: Check existing network topology defects, link bandwidth mismatch, unreasonable switch configuration and other hidden troubles;

•          Online fault rapid troubleshooting: Locate packet loss, communication delay, NCCL training speed attenuation, SHARP acceleration failure and other typical network problems;

•          Performance engineering tuning: Optimize switch buffer, lossless flow control parameters, NCCL/SHARP communication parameters, multi-rail link load balancing strategy;

•          Output standardized optimization report: Record all network bottleneck points, provide detailed adjustment steps and post-optimization performance comparison data, support internal technical team self-maintenance after service delivery.

Core Technical Technology Stack

Hardware & Interconnection: NVIDIA H200 / H100 GPU, InfiniBand Quantum-2 NDR, RoCE v2, GPUDirect Storage
Distributed Computing & Communication: NCCL, SHARP
Inference Acceleration Engines: vLLM, SGLang, TensorRT-LLM
Cluster Scheduling & Orchestration: Slurm, Kubernetes