Principal Search/AI Engineer - Lucene Platform

Description : Role Summary : We're looking for a Principal Search/AI Engineer to join our team of search experts and serve as technical lead on large scale search modernization projects and programs for Fortune-1000 clients.You will collaborate with our established search practice to design, build, and optimize AI-powered search platforms that deliver rich content and product discovery experiences.In this role, you'll drive technical excellence across multi-year digital transformation initiatives, lead solution architecture, and mentor engineering teams while implementing cutting-edge search solutions enhanced with AI and agentic capabilities for our retail, e-commerce, and enterprise clients.Responsibilities : Key Responsibilities : Search Platform Leadership : - Lead technical architecture and strategy for complex search modernization programs spanning multiple years and teams.- Drive end-to-end ownership of Lucene-based search platforms (Solr/Elasticsearch/OpenSearch) including schema design, index pipelines, scaling, and monitoring.- Architect high-throughput, low-latency Java services that expose search functionality via REST/gRPC APIs.- Design and implement advanced search relevance strategies including semantic search, hybrid approaches, and AI-enhanced ranking.- Perform sophisticated relevance tuning leveraging BM25, TF-IDF, vector similarity, and learning-to-rank methodologies.AI and Agentic Search Innovation : - Prototype and productionize next-generation search features including faceted navigation, autocomplete/type-ahead, spell-checking, vector search, and hybrid retrieval.- Lead integration of AI-powered conversational search, intelligent query understanding, and agentic workflows to enable natural-language and multi-turn search experiences.- Architect and implement agentic AI frameworks (LangGraph, LangChain) for multi-agent search workflows and intelligent content discovery.Technical Leadership and Collaboration : - Partner with data scientists and ML engineers to deploy and optimize PyTorch-based ranking models for query understanding, embeddings, and re-ranking.- Lead design and implementation of data pipelines (Python/Java/Spark/Dataflow) to generate training signals from user behavior analytics.- Mentor and guide engineering teams on search best practices, performance optimization, and emerging AI technologies.- Drive technical decision-making across cross-functional teams and stakeholder groups.Quality and Operational Excellence : - Establish comprehensive metrics, alerting, and automated testing frameworks to guarantee SLA compliance at enterprise scale.- Lead performance profiling and capacity planning for indexes serving hundreds of millions of documents or SKUs.- Drive continuous improvement initiatives for search relevance, performance, and user experience metrics.Requirements : Required Qualifications : - 8+ years building enterprise-scale back-end systems in Java (Spring Boot or similar) with demonstrated focus on performance, scalability, and reliability.- 5+ years hands-on leadership experience with Lucene-based search engines (Apache Solr, Elasticsearch, or OpenSearch) including advanced index design, relevance tuning, and cluster operations.- 3+ years experience as technical lead on large-scale search or platform modernization programs for enterprise clients.- Proven track record delivering AI-enhanced search solutions handling 100M+ documents/SKUs at Fortune-1000 scale (e-commerce, media, enterprise content, or similar).- Deep understanding of information retrieval concepts including tokenization, analyzers, BM25, TF-IDF, vector similarity, semantic search, and machine learning integration.- Strong proficiency in Python for data manipulation, ML model integration, and prototyping.- Extensive experience with cloud-native architectures, Docker/Kubernetes, CI/CD, and enterprise deployment (AWS, GCP, or Azure).- Demonstrated leadership in mentoring engineers and driving technical excellence across teams.Nice to have : Preferred/Bonus Skills : - Expert-level understanding of Lucene-based search engines and their underlying architecture, performance characteristics, and optimization strategies.- Hands-on experience with modern search platform ecosystem including Algolia, Coveo, Lucidworks, Constructor.io, or Vertex AI Search.- Proven experience implementing agentic AI frameworks (LangGraph, LangChain) and orchestrating multi-agent search workflows.- Advanced experience embedding and serving PyTorch models for Learning-to-Rank, embeddings, conversational AI, and generative re-ranking.- Deep knowledge of vector databases and similarity search (OpenSearch/ES K-NN, Milvus, Pinecone, Weaviate, Chroma).- Experience designing streaming data pipelines (Apache Kafka, Pub/Sub, Apache Flink) for real-time indexing and personalization.- Active contributions to open-source information retrieval/ML projects, published research, or recognized technical thought leadership.- Experience with cloud-native search services (AWS CloudSearch, GCP Vertex AI Search, Azure Cognitive Search) and hybrid architectures.- Background in natural language processing, transformer models, and large language model integration.- Technical marketing and thought leadership experience including creating and publishing blog posts, white papers, presenting at professional conferences, and maintaining active presence on professional social media platforms.Leadership And Soft Skills : - Proven technical leader who can balance architectural vision with pragmatic delivery timelines.- Excellent communication skills for presenting complex technical concepts to C-level executives, product stakeholders, and engineering teams.- Passionate advocate for engineering excellence, continuous learning, and staying current with search, ML, and AI agent technologies.- Strong collaborative mindset aligned with Grid Dynamics' co-innovation model and client partnership approach.- Experience building and scaling high-performing engineering teams in consulting or client services environments.We offer : - Opportunity to work on bleeding-edge projects.- Work with a highly motivated and dedicated team.- Competitive salary.- Flexible schedule.- Benefits package medical insurance, sports.- Corporate social events.- Professional development opportunities.- Well-equipped office. (ref: hirist.tech)