Data Engineer

Outbrain is the world’s largest content discovery platform, bringing personalised, relevant online, mobile and video content to audiences while helping publishers understand their audiences through data. Outbrain serves over 190 billion personalised content recommendations every month and reaches over 561 million unique visitors from across the globe.

Visual Revenue, an agile team within Outbrain, is looking for a highly motivated, self-reliant Data Engineer to join our New York office. Visual Revenue is a unique platform that processes, analyzes, and learns web and social traffic for hundreds of the biggest news sites around the world in real time. Our systems deal with the intricate complexity of petabyte-scale web traffic data (over 400M new events per day) and the needs of our diverse publisher base. You must be a self-starter with strong design and coding skills who loves shipping great product. You will be responsible for the design and development of the machine learning products and data systems that provide rich insight into web traffic performance using Visual Revenue's unique datasets. You will challenge yourself and others to constantly come up with better solutions behind the ultimate real-time analytics platform for news editors.

* Design, build, and deploy high-performance production platforms/infrastructure to support data warehousing, real-time ETL, and batch big-data - processing * Detect data quality issues all the way down to root cause, and implement fixes and data audits to prevent/capture such issues * Create compelling PoC’s for data solutions using emerging technologies for real-time and big data ingestion and processing * Optimize code and design architecture to streamline data science processes * Build, automate and deploy specialized research environments

* 3+ years of experience in software engineering * Strong programming experience with any of: Python, Scala or Java * In-depth understanding of object oriented programming concepts * Experience with batch and real-time data processing frameworks like Storm, Kafka, or Spark * Effective knowledge of SQL, especially Hive, and NoSQL databases * Expertise with version control systems, especially with Git - Proficiency with Unix/Linux environment **Preferred Qualifications:** * Experience with data mining, machine learning, and underlying algorithms * Experience with statistical methods and experimentation (A/B testing)