Staff Software Engineer, Web-Scale Crawling & Ingestion

Staff Software Engineer, Web-Scale Crawling & Ingestion

Nebius Group | Greater London, ENG, GB

Posted 13 days ago

Apply Now

Description

Nebius is hiring a Staff Software Engineer to build the content acquisition and crawling infrastructure for a cutting-edge AI search platform. You will design and operate web-scale crawling, ingestion workflows, and scheduling to handle billions of URLs with high throughput and reliability.

You will work with indexing and ML teams to ensure content quality and observability, while balancing coverage, freshness, and resource use.


#J-18808-Ljbffr