About this opportunity
Hewlett Packard Enterprise lists this Senior Software Engineer, Inference opportunity in durham, North Carolina. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Design and implement key components of the LLM runtime, including engine integration, batching, and KV cache management. Partner with engineering teams to optimize inference performance, including latency, throughput, and distributed execution capabilities.
Requirements: Requires at least 8 years of software engineering experience with 1-2+ years specifically in LLM inference runtimes or production model serving. Candidates must possess strong proficiency in Go, Python, Kubernetes, and deep knowledge of inference engine internals.
Key Skills: LLM inference, Kubernetes, Go, Python, C++, CUDA, vLLM, TensorRT-LLM, Continuous batching, KV cache management, Distributed execution, GPU optimization, Tensor parallelism, Pipeline parallelism, Performance profiling
Benefits: Health and wellbeing benefits, Professional development programs, Flexible work arrangements
#J-18808-Ljbffr
Worksite address
durham, NC, 27703, US
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.