Senior Software Engineer for Big Data

At Lytx, we harness the power of data to change human behavior and help good companies become even better. The Lytx DriveCam™ Program, sets the standard for driver safety in the industries we serve, and our RAIR® Compliance Services helps DOT-regulated fleets comply with safety regulations, complementing the DriveCam® Program. We protect more than 1,300 commercial and government fleet clients worldwide who drive billions of miles.

The Senior Software Engineer – Big Data will be an integral part of the core team designing and implementing the Hadoop ecosystem for Data Science, Data Analytics, Data Warehouse and Business Intelligence by leveraging variety of data types. One of the primary roles will include designing and developing large scale real-time and batch data ingestion and integrations using the Hadoop Platform. If you love building new big data platforms, guiding or assisting software and data engineering teams toward the development of big data solutions, then this is the role for you!

* Design and develop end-to-end Hadoop data ingestion process. Sufficiently decompose and transform the NoSQL, Binary, Sequential Flat file or XML data to Data warehouse through MapReduce, Pig, Hive, Avro and etc. * Design and develop Hadoop data integration in both real-time and scheduled mode leveraging appropriate Hadoop open integration framework (Storm, Flume, Sqoop, Oozie, and etc.) * Manage Hadoop clusters and guide the operational best practices for Hadoop cluster configuration and job processing. * Build Libraries, user defined functions (UDF), and frameworks around Hadoop designed to be reusable and scalable for enterprise wide use. * Manage the performance and allocations in open distributed file system (HDFS) leveraging the Hadoop data lifecycle and retention policy. * Implement standards based monitoring and tuning practices for the 20+ nodes Hadoop ecosystem. * Translate the functional and non-functional requirements from business, product and engineering teams to appropriate Hadoop technology solution design. * Document the big data solution design and clearly communicate the target solution patterns to the technical audiences.

**Required Qualifications:** * Minimum 4 years of experience of Java application development experience; strong knowledge of object-oriented-programming (OOP) with J2EE architecture expertise. * Must have an experience of setting up J2EE application build, deploy and promote configurations experience; Java library/source control management experience preferred. * Strong knowledge of SQL (Oracle, NoSQL, SQL Server) databases. * Solid backgrounds in fundamentals of computer science and development process. * Experience with open source frameworks and tools. * Familiarly with Hadoop, Map-Reduce, Hive and other big-data based open source projects. * Understanding of virtualization and cloud computing concepts and technologies. * Experience with Unix-like operation systems is required. * The ability to think clearly and quickly while taking ownership of important issues is imperative. * Strong foresight and the ability to anticipate challenges or obstacles. * Drives change in ambiguous situations and is a champion for those new approaches. * Other duties as assigned. **Preferred Qualifications:** * 1+ years experience of designing and developing Hadoop based solutions with strong knowledge of related framework including MapReduce, YARN, Hive, Sqoop, Flume and Oozie. * Experience in setting up, optimizing and sustaining Hadoop ecosystem at mid-to-large scale. * Familiarity with AWS technologies – EC2, S3, VPC. * Expert level Java coding experience. * Greenplum/Teradata database development experience for large scale of data warehouse. * Experience with distributed systems, persistence, caching and concurrent programming. * Experience in Storm/Kafka will be a huge plus.