Staff Machine Learning Engineer

Skip to main contentThis website uses cookies to improve your web experience. By using the site, you agree to the use of cookies. Privacy PolicyDeclineAccept CookiesEnglishSign InStaff Machine Learning Engineer page is loadedStaff Machine Learning EngineerApplyremote typeHybridlocationsMelbourne, AustraliaRemote, New South Wales, Australiatime typeFull timeposted onPosted Todayjob requisition idR35193Job DescriptionWhat we have builtThe Custom Agent Service handles the full agent lifecycle: an admin configures an agent with instructions, knowledge articles, and actions through a UI; a customer ticket triggers execution; the agent plans, acts through real APIs via the Integration Action Platform, and resolves the issue. The backend is Python, the infrastructure is Kubernetes on AWS, and the agent architectures range from single-pass ReAct loops to our iterative multi-plan executor.We are onboarding our first internal and external EAP customers, so the work ships to real accounts with real tickets.What we need help withAgent execution core. The planning loop, tool dispatch, memory integration, and error recovery that make up the main execution path. You will work directly on the code that decides what the agent does next and make it faster, more reliable, and more capable. This includes integrating new architectures into the production path and hardening them for real traffic.Knowledge retrieval. Agents retrieve and reason over customer knowledge bases at runtime. The retrieval pipeline (embedding, reranking, context assembly) needs to balance answer quality against latency and token cost, across thousands of heterogeneous knowledge bases per deployment.Actions and connectors. Agents call Zendesk APIs, third-party connectors (Shopify, Salesforce, etc.), custom actions configured by admins, and increasingly other agents via A2A. The execution layer needs reliable retries, timeouts, schema validation, and graceful degradation when connectors fail mid-execution. You would also self-service new connector integrations through the Connector SDK.Production instrumentation for model training. Every agent execution generates a trajectory (reasoning steps, tool calls, outcomes, user feedback). We are building toward training domain-specialized models, and that requires clean production data. You would instrument the execution pipeline to capture implicit reward signals (resolution success, escalation patterns, user satisfaction) that feed into the ML team's training pipeline.Security and compliance. PII filtering, audit logging, action versioning, and governance patterns that keep agents within admin-configured bounds. You would work directly with Product Security on security review items as the platform scales.What we are looking for5+ years of backend engineering with strong Python skills. You have shipped production systems, not just models. You understand the difference between getting an agent to work locally and running it across 100,000 accounts.Comfortable across the full agent stack: LLM APIs, prompt engineering, tool calling, memory management, evaluation. You can build an agent loop from scratch, and you know when a framework helps vs. when it gets in the way.You think about what happens when the model returns garbage, the connector times out, and the customer is waiting. You build for the failure case, not just the happy path.You ship working code, review PRs carefully, and communicate clearly about what is done, what is blocked, and what is at risk.Tech StackLanguages: Python (primary), some Go for platform servicesAgent Frameworks: Custom iterative architectures, ReAct, with integration points to open-source toolingInfrastructure: Kubernetes, Spinnaker, AWS (ECS, S3, ElastiCache)Data: Postgres, ElastiCache/Redis (vector + KV), KafkaEvaluation: Braintrust (experiment tracking, scoring, CI/CD integration)Protocols: MCP, REST, gRPCWhy Zendesk for this workZendesk has 100,000+ customers, billions of support interactions, and a...