Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Lead orchestration and optimization of large-scale LLM pretraining across 1,000+ GPUs. Manage distributed training with PyTorch/DeepSpeed/Megatron-LM, tune networking and memory (InfiniBand/RDMA), and implement checkpointing and robust failure recovery for long-running jobs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Responsible for operating and maintaining regulatory compliance across Boeing’s C-17 ASSC and Amberley Backshop DASR Part 145 maintenance organization and BDA Part 21G manufacturing operations. The role ensures maintenance and defect rectification meet regulatory and organizational standards, supports quality and safety systems, investigates compliance issues, updates command media, and provides leadership in a safe work environment.
Top Skills:
Bda Maintenance Organisation Exposition (Moe)Bda Part 21GDasr Part 145Engineering Australia Chartered StatusMaintenance Management Systems
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Own the full outbound B2B sales cycle for SMB merchants, from prospecting and pipeline creation through discovery, demonstrations, negotiation, and close. Develop tailored multi-product solutions, acquire new logos, manage Salesforce forecasting, and consistently achieve revenue targets. Collaborate with business development, product, marketing, implementation, and operations teams while serving as a consultative advisor across complex, multi-stakeholder deals.
Top Skills:
Salesforce
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Own the full outbound sales cycle for SMB merchants, from prospecting and pipeline creation through discovery, demos, negotiation, and close. Build net-new business through cold outreach, social selling, and targeted campaigns; design multi-product solutions across Square’s ecosystem; manage Salesforce forecasting; collaborate cross-functionally; and consistently achieve revenue targets.
Top Skills:
Salesforce
What you need to know about the Melbourne Tech Scene
Home to 650 biotech companies, 10 major research institutes and nine universities, Melbourne is among one of the top cities for biotech. In fact, some of the greatest medical advancements were conceptualized and developed here, including Symex Lab's "lab-on-a-chip" solution that monitors hormones to predict ovulation for conception, and Denteric's vaccine for periodontal gum disease. Yet, the thousands of people working in the city's healthtech sector are just getting started, to say nothing of the tech advancements across all other sectors.
