The role involves developing and maintaining internal production applications using Databricks to optimize processes and deliver products effectively. It requires technical solutions development, legacy system support, and ongoing process improvements based on project management collaboration.
Position Purpose: To be part of an existing team to create, maintain, support and improve upon existing internal production applications to speed up delivery of products to our customers. This will also include the implementation of new business rules and functionality the business deems required.
Key Responsibilities:
- Develop technical solutions that help achieve business and customer goals. Be able to provide clear feedback to technical and project teams upon encountering technological limitations.
- Be part of the ongoing development and support of any new solutions, as well as existing legacy production processing systems to agreed estimates and schedules.
- A demonstrated focus on continuous improvement and process optimisation of new and existing systems.
- Work with project management to assist in work breakdowns and estimates as required.
- Ensure documentation is created and maintained with a view to enabling ongoing development support (and where possible improvement) as well as handing over ongoing processing to the existing production team.
- Ensure ongoing code quality, data quality checking, error reporting and correction occurs.
Essential Subject Skills and Experience:
- 2+ years of E2E In-depth Databricks experience on a real project(s) that have gone to production
- Databricks Genie, AI/BI Dashboards and Analytics reporting.
- Databricks pipelines, warehouse, jobs.
- Databricks Platform set up and admin, cost reduction.
- Databricks ELT, CDC, Streaming - general ETL and ELT and ETL/ELT scaling. Databricks Ingestion tools, e.g., SQL Server Connect
- Development of Databricks applications, ‘App’ and ‘One’ experience.
- Strong experience with Delta Lake, Lakehouse and Medallion architectures, Data architecture
- Meta data catalogs and data governance - Databricks Unity catalog.
- Strong data modelling/data expertise.
- Understanding of AI and Ml.
- General computing experience, AWS, S3, CI/CD, Git, SDLCs, Agile/Iterative.
- Excellent communication and time management skills
- Strong analytical and problem-solving skills
- Be flexible/adaptable and can embrace change
Desirable Skills and Experience:
- DB/Spark performance and optimization.
- Strong Apache Spark and pyspark - created Spark clusters, Spark (and Kafka) Streaming, Data pipelines/orchestration.
- Strong Python and Dash development experience.
- Advance SQL, with materialized views, Common Table Expressions.
- Statistics, Machine Learning, data mining, Experimental design.
- MLFlow, AI training/testing/evaluation and its orchestration.
- GenAI, LLMs, Prompt Engineering, RAG, Fine Tuning, Agents.
- AI ways of working, e.g. CRISP-DM, TDSP etc.
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
Investigate and triage security events across customer environments, conduct incident response and forensic investigations, analyze malware, identify root causes, and recommend remediation. Produce MITRE ATT&CK-aligned incident reports, communicate findings to customers, collaborate with threat intelligence and detection engineering teams, and contribute to detection improvements and team mentoring within a 24/7 MDR SOC.
Top Skills:
BloodhoundCloud EnvironmentsEndpoint Detection And ResponseLinuxmacOSMetasploitMimikatzMitre Att&CkRapid7SIEMUser Behavior AnalyticsWindows
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
Investigate and triage high-priority security events for APAC customers within a 24/7 managed detection and response SOC. Responsibilities include alert review, threat detection, incident response, evidence acquisition, root-cause analysis, technical reporting, remediation recommendations, customer communication, attack-methodology documentation, and feedback to product teams.
Top Skills:
CodingKali LinuxLinuxScriptingSIEMThreat IntelligenceUser Behavior AnalyticsWindows
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Lead chocolate brand portfolio strategy and growth across Australia and New Zealand. Develop brand plans, allocate resources, execute integrated marketing communications and 360-degree campaigns, support revenue and profit goals, lead chocolate innovation platforms with cross-functional and manufacturing teams, and coach assistant brand managers. The role requires strong commercial, strategic, analytical, creative, communication, presentation, project management, and team leadership skills in a consumer goods, matrixed environment.
What you need to know about the Melbourne Tech Scene
Home to 650 biotech companies, 10 major research institutes and nine universities, Melbourne is among one of the top cities for biotech. In fact, some of the greatest medical advancements were conceptualized and developed here, including Symex Lab's "lab-on-a-chip" solution that monitors hormones to predict ovulation for conception, and Denteric's vaccine for periodontal gum disease. Yet, the thousands of people working in the city's healthtech sector are just getting started, to say nothing of the tech advancements across all other sectors.
