Job description
The Lead AI/ML Engineer owns the brain of the Navanta AI platform — retrieval, text-to-metrics, model serving, tool orchestration, and the evaluation harness that keeps answers honest. Working under the SVP of Technology and Commercial AI and in close collaboration with the data, platform, and product teams, this role makes “correct and verifiable” the product’s default — the foundation of trust in a regulated banking environment where a confident wrong number loses the account.
Key Responsibilities
Build Navanta’s retrieval and verifications over data systems, with shown queries and citations for every answer
Stand up self-hosted open-weight models serving and embeddings inside each bank’s environment or shared environments for Navanta; evolve RAG to a dedicated standard
Design the MCP tool layer that exposes a small, audited set of read-only tools (metrics, documents, customer 360), eventually growing into read/write tools with heavy amounts of regulated, highly sensitive data
Build and maintain the evaluation harness — golden-question regression, groundedness and retrieval metrics, explicit “I don’t know” behavior — and make it a release gate
Implement LLM guardrails: PII redaction in prompts and context, prompt-injection defenses, and cost and row limits aligned to regulatory security expectations
Partner with data teams so the model selects governed metrics from the semantic layer rather than improvising SQL
Document model architecture, evaluation methodology, and guardrail controls to support customer security reviews and audit readiness
Track latency, cost, and quality trade-offs across model versions and deployment configurations
Core Competencies
Accuracy and evaluation orientation — a demonstrated focus on verifiability and groundedness, not just compelling demos
Production LLM/RAG engineering: retrieval pipelines, tool orchestration, prompt engineering, and guardrail implementation
Security and compliance mindset: PII handling, prompt-injection defense, and least-privilege tool access aligned to NIST CSF 2.0 principles
Cross-functional collaboration with data and platform engineering to deliver a governed, auditable AI system
Key Performance Indicators (KPIs)
Golden-question accuracy — maintained or improved release over release against the verified question set
Groundedness rate: percentage of assistant answers fully supported by retrieved context
PII redaction coverage and zero prompt-injection incidents in production
Model serving latency and cost per query within defined targets
Evaluation harness adoption as a release gate — zero releases without passing the regression suite
Qualifications
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required.
6–10+ years building software, with 2–3+ years shipping production LLM, RAG, or NLP systems used by real people — not prototypes
A demonstrated focus on accuracy and evaluation, not just demos
Strong Python and solid software-engineering fundamentals
Comfort operating self-hosted open-weight models and reasoning about latency, cost, and quality trade-offs
Core Technologies
·Languages: Python
Serving & inference: vLLM, Ollama; GPU / CUDA familiarity, NVIDIA Enterprise (NVAIE)
RAG & retrieval: LlamaIndex or Haystack; Qdrant, pgvector; embeddings
Orchestration: MCP, tool / function calling
Structured querying: text-to-SQL; semantic layers (Cube / dbt MetricFlow)
Evaluation & guardrails: groundedness and eval frameworks, PII redaction, prompt-injection defense
Nice to Have
Experience in regulated or high-stakes domains where a wrong answer is costly
Fine-tuning, adapters, and retrieval-quality optimization
Familiarity with banking and finance terminology
Education and/or Experience
Bachelor’s degree in computer science, mathematics, or a related technical field, or equivalent hands-on experience
Experience in the financial services industry or a regulated, high-accuracy AI application environment strongly preferred
Work Structure & Expectations
Full-time role combining ongoing model operations and evaluation with initiative-based build-out of the data retrieval, guardrail, and serving infrastructure
Close collaboration with data engineering, platform engineering, and product teams; on-call rotation covering reliability in production
Physical Demands
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
While performing the duties of this job, the employee is regularly required to sit and use hands to finger, handle, or touch objects, tools, or controls. The employee frequently is required to talk or hear. The employee is occasionally required to stand; walk; and stoop, kneel, crouch, or crawl. The employee must occasionally lift and/or move up to 10 pounds, usually waist high, up to 50 feet away. Specific vision abilities required by this job include close vision and the ability to adjust focus.
Work Environment
The work environment characteristics described here are representative of those an employee encounters while performing the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
•Typical office environment
•Up to 20% travel time may be required
Who is Navanta?
Navanta is the trusted technology and services partner for community financial institutions, unifying critical systems, security, cloud infrastructure, and support into one seamless, purpose built experience. With more than 35 years of banking expertise — from Managed IT to Core Banking, CRM, and Advisory Services — Navanta helps institutions simplify complexity, reduce risk, and strengthen daily operations. Navanta empowers community bankers and their people to thrive together. Go Bankers, Go.™
Originally posted on Himalayas
Who can apply
Eligible countries: United States. Accepted UTC offsets: UTC-10, UTC-9, UTC-8, UTC-7, UTC-6, UTC-5, UTC+14. Review the full description for employer-specific work authorization, residency and schedule requirements.