AI Software Engineer, Legal Prompting & LLM Dev.

The AI Software Engineer, Legal Prompting & LLM Development is responsible for building production-grade applications that interact with state-of-the-art Large Language Models (LLMs) to deliver cutting-edge, time-saving solutions to attorneys throughout our firm and in support of our clients. This role sits at the intersection of software engineering, prompt engineering, and emerging agentic AI infrastructure — building not just LLM-powered applications, but the surrounding ecosystem of tools, integrations, and protocols that allow AI to interact safely and effectively with legal workflows and firm data.The AI Software Engineer will develop application code as well as English prose, persona-based prompts to LLMs, while also designing and operating the infrastructure that connects those models to the firm's systems, including Model Context Protocol (MCP) servers, retrieval-augmented generation (RAG) pipelines, vector stores, APIs, and agent orchestration frameworks. Works primarily within the Microsoft technology stack (Azure, .NET, C#, SQL Server) and with languages such as TypeScript/JavaScript, Python, and other modern languages as needed, integrating with internal and third-party APIs to deliver production-grade solutions. Works across programming languages and can produce reliable backend code while also writing creative, lucid, and effective prose instructions. A remote work schedule is available for this position.A career at Nixon Peabody is the opportunity to do work that matters. It’s the chance to use your knowledge to shape what’s ahead. To share, to innovate, to learn at a firm that taps the power of collective thinking.We’ve created a dynamic, energizing environment that promotes success for our clients and each other. We offer fast growth, connectedness and training in business as well as law. And our rigorous standards assure you are part of a diverse team of top talent at every turn.If you’re someone who’s looking toward the future, we’d love to hear from you.Location: Boston, MA; Chicago, IL; Los Angeles, CA; New York City, NY; Rochester, NY; San Francisco, CA; Washington, DCLLM Application DevelopmentDesign, develop, and deploy LLM-integrated applications that enhance legal workflows across transactional, litigation, regulatory, and advisory practice areas.Develop backend services across the Microsoft stack and in languages such as TypeScript/JavaScript, Python, C#, and others as needed, that interact with LLM providers (OpenAI, Anthropic, etc.), external APIs, SQL and NoSQL databases, and document management systems.Build and maintain RESTful and event-driven APIs that expose AI capabilities to internal applications and downstream consumers.Prompt Engineering & EvaluationWrite and refine persona-based prompts, system instructions, and few-shot examples to guide LLMs in delivering accurate, defensible, and legally appropriate responses.Build prompt evaluation harnesses, regression test suites, and offline/online evaluation pipelines (e.g., LLM-as-judge, golden datasets) to measure quality, hallucination rates, and latency.Continuously test and iterate on prompts and code to optimize model performance, cost, and user experience.MCP Servers & Tool IntegrationDesign, build, and operate Model Context Protocol (MCP) servers that expose firm systems — document management (e.g., iManage, NetDocuments), time and billing, CRM, research platforms, and internal knowledge bases — as secure, governed tools for AI agents.Define tool schemas, authentication flows, rate limiting, and audit logging for MCP endpoints, ensuring outputs are scoped to user permissions and ethical walls.Maintain a catalog of reusable MCP tools and resources that can be composed across multiple AI products at the firm.Retrieval, RAG & Knowledge InfrastructureBuild and tune retrieval-augmented generation pipelines, including chunking strategies, embedding model selection, hybrid search (lexical + semantic), and reranking.Work with vector databases (e.g., Pinecone, Weaviate, pgvector, Azure AI Search) and orchestration frameworks (e.g., LangChain, LlamaIndex, Semantic Kernel) to ground LLM outputs in firm and client data.Agentic Workflows & OrchestrationDevelop multi-step and multi-agent workflows that combine planning, tool use, and human-in-the-loop checkpoints for sensitive legal tasks.Implement guardrails, content filters, PII redaction, and citation/verification layers to ensure responsible use.MLOps, Observability & SecurityContainerize services (Docker) and deploy via CI/CD pipelines to cloud environments (Azure preferred; AWS/GCP a plus), using infrastructure-as-code (Terraform, Bicep) where appropriate.Instrument applications with logging, tracing, and LLM-specific observability tools (e.g., LangSmith, Arize, Weights & Biases, OpenTelemetry) to monitor quality, cost, and drift in production.Partner with Information Security and the Office of the General Counsel to ensure solutions meet client outside counsel guidelines, data residency requirements, and confidentiality obligations.Collaboration & TranslationCollaborate with attorneys, legal professionals, and product teams to understand domain-specific needs and translate them into technical solutions.Assess the integration of LLMs into existing legal workflow systems and recommend improvements.Perform other duties as assigned.To perform this job successfully, you must be able to perform each essential job responsibility listed above, satisfactorily, with or without reasonable accommodation. Nixon Peabody retains the right to change or assign other duties to this position. The requirements listed below are representative of the skills and abilities required.