Software Engineer, Inference AI/ML

Skip to main content Software Engineer, Inference AI/ML CoreWeave Sunnyvale, CA Apply Join or sign in to find your next job Join to apply for the Software Engineer, Inference AI/ML role at CoreWeave Email or phone Password Show Forgot password? Sign in Sign in with Email or New to LinkedIn? Join now By clicking Continue to join or sign in, you agree to LinkedIn’s User Agreement, Privacy Policy, and Cookie Policy. Software Engineer, Inference AI/ML CoreWeave Sunnyvale, CA 1 week ago Over 200 applicants See who CoreWeave has hired for this role Apply Join or sign in to find your next job Join to apply for the Software Engineer, Inference AI/ML role at CoreWeave Email or phone Password Show Forgot password? Sign in Sign in with Email or New to LinkedIn? Join now By clicking Continue to join or sign in, you agree to LinkedIn’s User Agreement, Privacy Policy, and Cookie Policy. Save Report this job CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Learn more at www.coreweave.com.What You’ll DoJoin the Inference team to ship production features that improve latency, reliability, and cost for model serving on our GPU platform. As an IC1, you’ll implement well-scoped changes, learn our operational practices, and grow quickly with mentorship from experienced engineers.About The RoleImplement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve).Write tests, code comments, and short design docs; participate in code reviews.Add basic metrics and dashboards; assist with alarms and runbooks.Follow on-call runbooks and learn incident response in a guided rotation.Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.Who You AreBS/MS in CS, EE, or related field, or equivalent practical experience.Foundations in data structures, algorithms, and networked services. Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics. Exposure to containers and Kubernetes (coursework or projects welcome). Curiosity about GPU inference concepts (micro-batching, KV cache, streaming).PreferredInternship or project that deployed a microservice or ML inference demo.Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus.Familiarity with Grafana/Prometheus/OpenTelemetry or similar...