Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Lead orchestration and optimization of large-scale LLM pretraining across 1,000+ GPUs. Manage distributed training with PyTorch/DeepSpeed/Megatron-LM, tune networking and memory (InfiniBand/RDMA), and implement checkpointing and robust failure recovery for long-running jobs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
.png)
Greentech • Hardware • Internet of Things • Machine Learning • Software • Business Intelligence • Agriculture
Field-based hardware technician responsible for deploying, maintaining and diagnosing collars and towers on farms, providing on-call support, collecting and analysing hardware data, building maintenance systems, and feeding field insights to product and engineering teams while regularly travelling across the region.
Top Skills:
Networking
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Field sales role owning a Melbourne territory: prospect and build pipeline through in-person visits, deliver demos, close full-cycle deals across Square's products, partner cross-functionally, maintain Salesforce records, and exceed quota through community outreach and channel partnerships.
Top Skills:
SalesforceSquare
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Individual contributor owning partner marketing for Australia. Build and localize global playbooks, execute joint campaigns across events, digital, content and enablement, track performance, drive leads, self-onboards and product attach, and manage industry, retail and supplier partnerships to amplify growth.
What you need to know about the Melbourne Tech Scene
Home to 650 biotech companies, 10 major research institutes and nine universities, Melbourne is among one of the top cities for biotech. In fact, some of the greatest medical advancements were conceptualized and developed here, including Symex Lab's "lab-on-a-chip" solution that monitors hormones to predict ovulation for conception, and Denteric's vaccine for periodontal gum disease. Yet, the thousands of people working in the city's healthtech sector are just getting started, to say nothing of the tech advancements across all other sectors.
