About this role
We are seeking a Machine Learning Staff Engineer to lead the development of core backend services powering our Agentic AI Platform. In this role, you will be the primary technical driver for building reasoning-driven agents, multi-agent orchestration, and outcome-based workflows.
You will collaborate with the Principal AI Architect to bridge the gap between high-level design and scalable implementation. You will own the development of high-performance APIs and orchestration runtimes, integrating frameworks like LangChain, LangGraph, and LangSmith. This role is highly hands-on and requires a technical leader who can navigate a fast-paced environment while maintaining high engineering standards.
Key Responsibilities
Platform Backend Development
Lead Implementation: Act as the lead engineer for agent runtime orchestration services, focusing on reasoning, planning, and tool invocationAPI & SDK Design: Own the development of robust APIs and SDKs that enable internal and external teams to build and deploy agent workflowsState & Memory Management: Build and optimize stateful dialog management and memory services for multi-turn, context-aware agentsAgent Communication: Implement multi-agent communication (A2A) protocols and shared context systemsSystem Architecture & Execution
Microservices Leadership: Execute the transition to cloud-native, event-driven backend services using microservices or service mesh architecturesWorkflow Design: Build and manage task scheduling and long-running operations using tools like Temporal or AirflowPerformance Optimization: Optimize LLM orchestration for performance and cost through caching, batching, and token monitoringIntegrations & Data Services
Data Pipeline Ownership: Implement and maintain RAG pipelines and integrations with vector databases (e.g., Pinecone, Weaviate, FAISS)Telemetry Integration: Ensure all backend services support LangSmith-based evaluation and comprehensive observabilitySecurity & Reliability
Tenant Isolation: Implement RBAC, API authentication, and multi-tenant isolation logicEngineering Excellence: Set the standard for automated testing (unit, integration, load) and implement OpenTelemetry for full system visibilityMentorship: Provide technical guidance and code reviews for more junior backend engineers on the team Required Qualifications:
6+ years of backend engineering experience1-2+ years of hands-on experience in AI/ML platform developmentExpert Proficiency: Strong programming skills in Java and PythonFramework Expertise: Hands-on experience with LangChain, LangGraph, or LangSmithDistributed Systems: Deep understanding of event-driven architectures, microservices, and KubernetesCloud & Data: Solid experience with AWS, Terraform, and a mix of relational (PostgreSQL) and Vector databasesOrchestration: Practical experience with workflow engines like Temporal or Airflow and messaging systems like KafkaPreferred Qualifications:
Experience with AI-driven planning systems or dialog state trackingBackground in enterprise security (RBAC, SSO, OAuth2)Experience working in a high-growth SaaS or startup environmentActive contributions to open-source backend or AI frameworks Please note this is a hybrid role with onsite expectations of 3x/week (Tues - Thurs) from our San Mateo, CA headquarters.
The annual base salary range for this position is $211,000 - $261,000. This role is also eligible for a target bonus.
Compensation is based on a variety of factors, including but not limited to location, experience, job-related skills, and level.
Freshworks offers multiple options for dental, medical, vision, disability, and life insurance. Equity + ESPP, flexible PTO, flexible spending, commuter benefits, and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.
At Freshworks, we have fostered an environment that enables everyone to find their true potential, purpose, and passion, welcoming colleagues of all backgrounds, genders, sexual orientations, religions, and ethnicities. We are committed to providing equal opportunity and believe that diversity in the workplace creates a more vibrant, richer environment that boosts the goals of our employees, communities, and business. Fresh vision. Real impact. Come build it with us.