Posted 12 days ago
Description
We're partnering with an innovative UK technology company developing a next-generation AI platform for critical defence and healthcare applications. As they continue to scale, they're looking for a Principal Data & AI Engineer to provide technical leadership across the design, architecture, and delivery of enterprise-grade AI and data platforms.
This is a rare opportunity to take ownership of the data and AI engineering function within a rapidly growing business, shaping the technical direction of products that solve complex, real-world problems. You'll work alongside AI researchers, software engineers, and product teams to build highly secure, scalable data platforms that power advanced AI capabilities in mission-critical environments.
If you're passionate about modern data engineering, large-scale AI systems, and influencing technical strategy while remaining hands-on, this role offers genuine autonomy and long-term career progression.
What You'll Be Doing
- Lead the architecture and development of scalable AI and data platforms supporting advanced machine learning and LLM workloads.
- Design and build high-performance data pipelines capable of processing structured and unstructured data from multiple secure sources.
- Architect robust data ingestion, transformation and orchestration frameworks using modern cloud-native tooling.
- Build and optimise Retrieval-Augmented Generation (RAG) pipelines and enterprise knowledge retrieval platforms.
- Develop and optimise vector search capabilities using technologies such as Weaviate, Pinecone, Milvus or Elasticsearch.
- Engineer highly available APIs and backend services using Python (FastAPI/Flask) and modern software engineering practices.
- Design secure, containerised platforms using Docker and Kubernetes for deployment into restricted environments.
- Drive best practice around data modelling, governance, observability, testing and platform reliability.
- Mentor and provide technical leadership to engineers across the wider software and AI teams.
- Work closely with stakeholders to translate complex business challenges into scalable technical solutions.
What We're Looking For
- Significant commercial experience as a Senior, Lead or Principal Data Engineer, AI Engineer or Platform Engineer.
- Strong expertise in Python and SQL, alongside modern software engineering principles.
- Deep understanding of distributed data processing and orchestration technologies (e.g. Spark, Kafka, Airflow, Dagster or similar).
- Experience building AI-enabled platforms using Large Language Models (LLMs), embeddings, vector databases and RAG architectures.
- Hands-on experience with vector databases such as Pinecone, Weaviate, Milvus or Elasticsearch.
- Strong understanding of cloud-native architecture, Docker and Kubernetes.
- Experience designing resilient microservice architectures and RESTful APIs.
- Knowledge of data governance, security, metadata management and production data operations.
- Comfortable leading technical direction, mentoring engineers and influencing architecture decisions.
- Excellent communication skills with the ability to engage technical and non-technical stakeholders.
- Degree in Computer Science, Software Engineering, Data Engineering or a related discipline (or equivalent commercial experience).
- Experience deploying AI platforms into secure or highly regulated environments.
- Knowledge of MLOps tooling and model lifecycle management.
- Experience with Azure, AWS or private cloud infrastructure.
- Familiarity with Infrastructure as Code (Terraform, Helm, Ansible or similar).
- Experience within Defence, National Security, Healthcare, Government or other regulated industries.
- Exposure to knowledge graphs, semantic search or graph databases.
- Experience leading technical strategy within a fast-growing technology business.
Security Requirements
Due to the nature of the work, applicants must be eligible for SC Security Clearance
#J-18808-Ljbffr
