Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Lead orchestration and optimization of large-scale LLM pretraining across 1,000+ GPUs. Manage distributed training with PyTorch/DeepSpeed/Megatron-LM, tune networking and memory (InfiniBand/RDMA), and implement checkpointing and robust failure recovery for long-running jobs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
.png)
eCommerce • Fintech • Hardware • Payments • Software • Financial Services
Leads Square’s partnerships strategy across Australia and Japan, building and scaling relationships with financial institutions, technology platforms, distributors, and other strategic partners. Owns partner-driven seller acquisition, revenue growth, product adoption, ecosystem expansion, negotiations, and portfolio performance. Develops executive relationships, influences product and go-to-market strategy, and leads a distributed regional partnerships organization while collaborating across Sales, Product, Marketing, Engineering, Finance, Legal, and Operations.
Top Skills:
Payments TechnologyProduct IntegrationsSaaS
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Manage disposal of surplus inventory and quarantine defective, deficient, or damaged items supporting Australia’s F/A-18F Super Hornet and EA-18G Growler fleets. Responsibilities include assessing inventory impacts, preparing approval documentation, recommending compliant disposal methods, maintaining audit readiness, and coordinating with logistics, engineering, supply chain, and customer stakeholders. The role also provides status reporting and risk notifications while ensuring adherence to defence policies and Commonwealth requirements.
Top Skills:
DeflogmanEscmMilisSap S/4Hana
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Leads Square’s partnerships strategy across Australia and Japan, building and scaling relationships with financial institutions, technology platforms, distributors, and commerce ecosystems. Owns partner acquisition, commercial negotiations, seller growth, product adoption, co-selling, integrations, and portfolio performance. Builds and develops a distributed regional partnerships organization while influencing Product, Sales, Marketing, Engineering, Finance, Legal, and executive leadership. Regular APAC and U.S. travel is required.
What you need to know about the Melbourne Tech Scene
Home to 650 biotech companies, 10 major research institutes and nine universities, Melbourne is among one of the top cities for biotech. In fact, some of the greatest medical advancements were conceptualized and developed here, including Symex Lab's "lab-on-a-chip" solution that monitors hormones to predict ovulation for conception, and Denteric's vaccine for periodontal gum disease. Yet, the thousands of people working in the city's healthtech sector are just getting started, to say nothing of the tech advancements across all other sectors.
