Review AI-escalated risk cases (KYC, ATO, 2FA, withdrawals), make pass/block/escalate decisions, label outcomes with root-cause annotations to train ROCA, maintain SLAs and accuracy, update SOPs, and support model calibration and BPO onboarding.
Binance is a leading global blockchain ecosystem behind the world’s largest cryptocurrency exchange by trading volume and registered users. We are trusted by 300+ million people in 100+ countries for our industry-leading security, user fund transparency, trading engine speed, deep liquidity, and an unmatched portfolio of digital-asset products. Binance offerings range from trading and finance to education, research, payments, institutional services, Web3 features, and more. We leverage the power of digital assets and blockchain to build an inclusive financial ecosystem to advance the freedom of money and improve financial access for people around the world.
Binance's Risk Operations team is executing an AI-First transformation — moving from manual case review toward a fully autonomous decision pipeline powered by ROCA (Risk Operations AI Agent). This role sits at the critical human-in-the-loop node: reviewing cases where ROCA's confidence score falls below the auto-decision threshold, generating structured feedback that drives continuous model improvement, and progressively expanding the automation boundary.This is not a traditional fraud analyst role.
Think of it like the shift from manual labor to the steam engine era — the people who thrived weren't the ones who resisted the machine, but those who learned to operate it. Here, you are the operator: you direct the AI, validate its judgment, correct its blind spots, and steer it toward better outcomes. The machine handles the volume; you provide the expertise it can't yet replicate.
Responsibilities:
- Review cases escalated by ROCA where confidence score falls below the auto-decision threshold, using AI-assisted signals within the unified case review UI (no multi-system switching)
- Make accurate pass / block / escalate decisions on a daily queue spanning KYC Audit, ATO investigation, Reset 2FA, POA Review, and Withdrawal anomaly case types etc.
- Escalate Key User cases (Priority ≥ 75) and high-risk patterns to Senior Agent / Team Lead with clear rationale documentation
- Maintain SLA compliance (<4h manual resolution time) while sustaining decision accuracy ≥ 98%
- Label every reviewed case as TP / TN / FP / FN with structured root-cause annotations — this data directly feeds ROCA's continuous learning pipeline
- Identify recurring FP/FN patterns and route them through the correct feedback path:
- Type A — Feature weight bias → trigger model retraining / fine-tune
- Type B — Threshold miscalibration → update Decision Hub rules (with A/B validation)
- Type C — SOP gap or logic error → update SOP (version-tracked)
- Actively contribute to improving Precision (>95%) and Recall (>99%) targets by surfacing systematic signal failures to Data and Model teams
- Participate in regular AI model calibration sessions: review misclassification batches, validate ground truth, challenge decision thresholds with data
- Maintain and update Confluence SOP pages for assigned case types; SOP coverage rate target ≥ 80% (directly impacts ROCA first-pass accuracy)
- Flag SOP coverage gaps when ROCA's reasoning chain reveals unhandled scenarios; draft SOP additions for Team Lead review
- Onboard and calibrate BPO agents on AI-assisted review workflows; monitor BPO quality against automated QA benchmarks
Core Review Duties
AI Feedback & Pipeline Contribution
SOP & Knowledge Management
Requirements:
- 2+ years in fraud review, risk operations, or KYC/KYB case management
- Hands-on experience with ATO (Account Takeover) detection and investigation
- Comfort working in AI-augmented workflows: you read model outputs, interpret confidence scores, and make decisions based on AI-generated signals rather than raw data alone
- Structured, precise written communication: you can document a root-cause annotation in <3 sentences that an ML engineer can act on
- Bilingual English/Mandarin is required to be able to coordinate with overseas partners and stakeholders.
- An operator's mindset toward AI tools: you leverage AI signals to make faster, better decisions — you master the tool rather than compete with it
Nice to have:
- Experience in crypto / Web3 / VASP risk environment; familiarity with blockchain transaction patterns (Sybil attacks, wash trading, mule account indicators)
- Exposure to ML model evaluation concepts: Precision, Recall, F1, confusion matrix — you don't need to build models, but you need to speak the language
- Familiarity with prompt engineering or LLM evaluation workflows; experience reviewing or testing AI-generated decisions at scale
- CAMS, CFE, or equivalent AML / fraud certification
- Prior involvement in SOP authoring, QA calibration, or BPO oversight for a risk operations team
Why Binance
• Shape the future with the world’s leading blockchain ecosystem
• Collaborate with world-class talent in a user-centric global organization with a flat structure
• Tackle unique, fast-paced projects with autonomy in an innovative environment
• Thrive in a results-driven workplace with opportunities for career growth and continuous learning
• Competitive salary and company benefits
• Work-from-home arrangement (the arrangement may vary depending on the work nature of the business team)
Binance is committed to being an equal opportunity employer. We believe that having a diverse workforce is fundamental to our success.
By submitting a job application, you confirm that you have read and agree to our Candidate Privacy Notice.
Binance Melbourne, Victoria, AUS Office
Melbourne, VIC, Australia
Similar Jobs
Blockchain • Fintech • Software • Cryptocurrency • Metaverse
Own and scale APAC programmatic advertising strategy to drive user acquisition, retention, and ROAS. Lead media buying across auctions, PMPs, preferred deals and PG; leverage AI/ML and automation; set best practices for targeting, frequency, remarketing, brand safety and DCO; research and scale emerging channels like CTV, audio, contextual and DOOH.
Top Skills:
AIAppsflyerContextual TargetingCtvDcoDoohMachine LearningOpen AuctionPmpPreferred DealsProgrammaticProgrammatic AudioProgrammatic Guaranteed
Blockchain • Fintech • Software • Cryptocurrency • Metaverse
Design and iterate secure, user-friendly trading interfaces (Convert, in-chat cards, AI trading analytics). Conduct user research, produce production-ready UI/UX deliverables (flows, wireframes, prototypes, component specs), define and optimize trading metrics, and collaborate with product, engineering, risk, compliance and operations to deliver consistent, scalable global trading experiences.
Blockchain • Fintech • Software • Cryptocurrency • Metaverse
Manage full financial and management reporting, cash flow, tax and GST compliance, budgeting, statutory and annual reports, audits, and revenue and balance sheet reconciliations.
Top Skills:
Excel
What you need to know about the Melbourne Tech Scene
Home to 650 biotech companies, 10 major research institutes and nine universities, Melbourne is among one of the top cities for biotech. In fact, some of the greatest medical advancements were conceptualized and developed here, including Symex Lab's "lab-on-a-chip" solution that monitors hormones to predict ovulation for conception, and Denteric's vaccine for periodontal gum disease. Yet, the thousands of people working in the city's healthtech sector are just getting started, to say nothing of the tech advancements across all other sectors.
