Big Data Architect

Lytx

(San Diego, California)
Full Time
Job Posting Details
About Lytx
At Lytx, we harness the power of data to change human behavior and help good companies become even better. The Lytx DriveCam™ Program, sets the standard for driver safety in the industries we serve, and our RAIR® Compliance Services helps DOT-regulated fleets comply with safety regulations, complementing the DriveCam® Program. We protect more than 1,300 commercial and government fleet clients worldwide who drive billions of miles.
Summary
We’re looking for hungry, humble and capable people who want to be part of building something special. Ideally, people who want to play a part in shaping the future where your work makes a direct contribution and helps to save lives. Recognized in this industry as the “best in breed” for our software development, Lytx is scaling and looking for a big data architect. The Big Data Architect will be the technical leader designing, deploying and managing the big data infrastructure as we build on our impressively successful platform. We are looking for someone highly conversant in large scale data sets, but able to solve massive real world business problems utilizing the latest technologies like Hadoop 2.0, Spark, Storm, and Kafka. If this sounds exciting, please read on!
Responsibilities
* Build world class high-volume real-time data ingestion frameworks and automate ingesting various data sources into Hadoop. * Provide leadership and expertise in the development of new products/services/processes, frequently operating at the leading edge of technology. * Lead the transformation of existing data and analytic infrastructure to a highly scalable, flexible, and performant big data platform using appropriate open source technologies. * Articulate architectural differences between big data solution methods and the advantages/disadvantages of each. * Interpret and deliver impactful plans that specify strategy and improve data integration, data quality and data delivery in support of big data business initiatives and road maps to achieve results. * Collaborate with end users, development staff, and business analysts to ensure that prospective data architecture plans maximize the value of client data across the organization. * Other duties as assigned.
Ideal Candidate
* Understanding the concepts and technology ecosystem around both real-time and batch processing in Hadoop. * Experience with various messaging systems, such as Kafka or RabbitMQ. * Expert level experience with the Hadoop ecosystem - HDFS, MapReduce, Hive, HBase, Oozie, Sqoop, Storm, and Kafka. * Experience with Cloudera/MapR/Hortonworks. * 8+ years of experience with progressively increasing responsibilities in the areas of software analysis, design, development and architecture. * 3+ years of hands-on experience designing and implementing data architecture, modeling, processing batch and real time data streams with relational and non-relational databases. * 2+ years experience installing and administering one or more Hadoop distributions such as Hortonworks or Cloudera. * Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge of NoSQL platforms. * Development knowledge of J2EE and design patterns in a distributed, multi-platform, heterogeneous computing environment. **Desired Experience & Qualifications:** * Good understanding of Lambda Architecture, along with its advantages and drawbacks. * Experience with using Storm to build stream-processing system. * Experience with using Spark for real-time analytics. * 4+ years of experience leading technical teams. * Hands on experience with Hortonworks 2.2+ or Cloudera 4.5+. * In-depth knowledge on big-data technologies and understanding of latest trends and products in the market. * Experience working with Amazon Web Services (AWS) - EC2, VPC, S3. * Prior experience as a tech lead or architect is required.

Questions

Answered by on
This question has not been answered
Answered by on

There are no answered questions, sign up or login to ask a question

Want to see jobs that are matched to you?

DreamHire recommends you jobs that fit your
skills, experiences, career goals, and more.