Staff Research Engineer, Agent Evals & Post-training
Posted Oct 5, 2026·Open for 1 day (and counting)
Hiring for this role?
Generate a McCoy IQ challenge in 30 seconds.
See how candidates think and approach the work this role demands, before the phone screen. We'll build a video challenge from this posting, and you can edit or share it before it goes live.
Key details
Function
Research
Seniority
Mid
Workplace
On-site
Location
New York, USA
Specialty
Agent Evals
What makes this role novel
Agent evals and post-training for hardware-reasoning systems is a specialized subcategory of AI safety/capability work that emerged only in the last 2–3 years as LLM agents became viable for domain-specific reasoning tasks; the day-to-day work—designing evaluations for agentic behavior in physical domains and iterating on model outputs for hardware contexts—did not exist as a distinct role before ~2022.
Archetype: Research
Job Description
About Nominal
Our mission is to accelerate how the world engineers new hardware. Nominal's connected test and operations platform powers the world's most advanced hardware programs and its most ambitious startups, from spacecraft, racecars, and autonomous vehicles to next-generation defense and energy programs. Our customers include Anduril, Shield AI, Hermeus, Albedo, Shinkei, and Pratt Miller Motorsports, as well as U.S. Navy and U.S. Air Force programs. Now we're expanding across the entire hardware lifecycle, building the foundation, AI-native applications, and agents that accelerate innovators' work and change what's possible to build.
We're backed by Sequoia, General Catalyst, Founders Fund, Lux Capital, and Lightspeed, and our team comes from SpaceX, Apple, Palantir, Anduril, Applied Intuition, and other leading companies.
About Hardware Intelligence
We're the team behind Nominal's agents, AI-native applications, and MCP, and its forward-leaning AI bets. Our mission is to unlock the bottlenecks of the hardware lifecycle with AI. Our agents reason over physical reality, from high-rate telemetry and test campaigns to designs and simulations, where real test results are the ground truth their work is checked against. We believe opinionated AI, built for the real work of hardware programs, will change how the world engineers.
We're collaborative, iterative, and high-agency, and we're human-centered and customer-focused. We build with the newest AI tools every day, and because those tools keep changing, so do we: we stay curious and keep looking for the better way. Our team spans data science and ML, distributed systems, search, and knowledge systems, and we obsess over how agents can be genuinely useful to the engineers who rely on them.
The Role:
As a Staff Research Engineer supporting agent evals & post-training, you'll define how Nominal measures its agents, for customers and for ourselves, and build the path from evals to post-trained models when hardware needs them. Evals come first; post-training follows when air-gapped deployment or cost makes it the right investment.
💼 What You'll Do:
- Build eval suites for our agents, the MCP tool layer, and our internal company agent, grounded in real hardware tasks.
- Invent new benchmarks for agents working over physical engineering data, where none exist today.
- Benchmark our agents against frontier agents using our MCP, and show where and why ours win.
- Build the eval infrastructure: datasets, model-based and human graders, and regression gates in CI.
- Turn evals into reward signals and training data, and lead post-training (fine-tuning, RL, distillation) when we need models we can run anywhere, including air-gapped environments.
- Set Nominal's strategy for measurement and model improvement, working with hardware experts on what "correct" means.
🚀 What you'll bring:
- 8+ years in ML engineering or research, including evals or post-training work you led in production.
- Statistical rigor: you design evals that don't fool you, and you know when a difference is real.
- Deep experience evaluating LLM or agent systems, including model-graded evals and their limits.
- Hands-on post-training experience: fine-tuning, RL from feedback, or distillation on real tasks.
- The judgment to know when to measure, when to train, and when a better prompt or tool is the answer.
- A track record of setting technical direction across a team and raising the bar for the engineers around you.
- You build with modern AI coding agents (Claude Code, Cursor, Codex) every day, and stay curious and open to better ways of working. The tools keep changing, and so do we.
⚡️ Nice to have:
- You've built evals or post-training at a frontier lab or an AI-native company.
- You've published benchmarks or eval methods that others use.
- You've trained or served open-weight models in restricted, on-prem, or air-gapped environments.
- You've worked in test, reliability, or verification engineering for physical systems.
Benefits/Perks
- 🏥 100% coverage of medical, dental, and vision insurance
- 🏖️ Unlimited PTO and sick leave
- 🍽️ Free lunch, snacks, and coffee
- 🚀 Professional Development Stipend
- ✈️ Annual company retreat
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, or national origin.
ITAR Requirements
To conform to U.S. Government export regulations, applicant must be a (i) U.S. citizen or national, (ii) U.S. lawful, permanent resident (aka green card holder), (iii) Refugee under 8 U.S.C. § 1157, or (iv) Asylee under 8 U.S.C. § 1158, or be eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here.
Audit details(provenance, verification trail, raw fields)
Core fields
Posting ID
nominal:am9icG9zdDp7LtWwyt-ZLX3N-M7SHGRVTitle
Staff Research Engineer, Agent Evals & Post-training
Function
Research
Location
New York, NY
Workplace mode
unspecified
Posted at
2026-10-05 21:18:00Z
Compensation
undisclosed
Provenance
First seen (our tracker)
2026-10-06 11:40:05Z
Last seen
2026-10-06 11:40:05Z
Last updated
2026-10-07 05:20:34Z
Removed at
still open
Days open
Open for 1 day (and counting)
ATS adapter
gem
ATS slug
nominalVerification trail
This posting hasn't been probed by our closure verifier yet. Stream C runs on a rolling schedule against postings approaching the close-decision threshold.
LLM enrichment
Enriched at 2026-10-07 04:32:44Z. Enrichment runs once per posting, never re-runs.
Seniority
ic_l4
Role archetype
research
Specialty
agent_evals
Workplace mode
unknown
City (normalized)
New York
Country (normalized)
United States
Comp range
—
Tech stack
—
Novel role archetype?
yes
See how we measure for definitions, or our corrections log for known issues. Found something wrong? Flag a correction.