Data Infrastructure Engineer

Shopify is a leading cloud-based, multichannel commerce platform designed for small and medium-sized businesses. Merchants can use the software to design, set up and manage their stores across multiple sales channels, including web, mobile, social media such as Pinterest and Facebook, brick-and-mortar locations, and pop-up shops. The platform also provides a merchant with a powerful back-office and a single view of their business.

Shopify's platform is expanding at an incredible rate. Along the way, it generates vast swaths of information, which if understood correctly, can lead to far better decisions, and a more successful company. Hadoop powers these critical insights for all levels of decision makers at and building on top of Shopify, and these insights need to be delivered reliably. This is why we're looking to add an experienced data infrastructure engineer to our Data Engineering team. Please note that all members of the Data Engineering team participate in a 24/7 on call rotation.

• Ensuring that Hadoop stays online, secure, and performant
• Day-to-day system administration of the production environment
• Capacity planning
• Development of configuration management and automation tools
• Network troubleshooting and monitoring
• Scalability, performance, and availability of the Hadoop and Spark infrastructure
• Collaborating with other Data Engineers and Data Analysts throughout your day-to-day activities

• Experienced in planning, deployment, and maintenance of large Linux installations
• Proficient with configuration management tools, like Chef
• Skilled with scripting languages such as Ruby or Python
• Experienced with a variety of open source software
• Experienced with Hadoop administration: YARN, MapReduce, Hive, Impala
• Experienced with database administration: query optimization, OLTP and OLAP schema design, hardware considerations
• Passionate about working on a team: collaborating on problems, asking questions, delivering feedback, and supporting others in their goals

It’d be great if you have experience with:

• Apache Spark
• Data analytics (basic query and report authoring)
• Cloudera Hadoop Distribution 5 and Cloudera Manager
• Networking with Arista or Juniper gear
• MySQL administration and performance tuning
• Amazon Web Services and Heroku
• Cassandra or other NoSQL data stores
• Splunk or other log aggregation tools
• Working on a distributed/remote team, as the Data Engineering team is distributed across Shopify’s offices