Senior Hadoop Developer

NTT DATA is a leading IT services provider and global innovation partner headquartered in Tokyo, with business operations in over 40 countries. Our emphasis is on long-term commitment, combining global reach with local intimacy to provide premier professional services varying from consulting and systems development to outsourcing.

A large client that sells commercial and industrial appliances desires to augment their reporting and operational infrastructure using big data. They generate large volumes of web server log data consisting of results from custom API calls, company portal website use, and device telemetry. The project focus is the ingestion log data with some filtering, marrying it with other data from production databases, and exposing the results to their Tableau reporting platform.

* Oversee and maintain clustering platform, and work with Data Architect to implement any future changes, as well as make proactive adjustments to better performance * Perform root cause analysis on all processes and resolve all production issues needed to support the ETL applications * Perform periodic data validation and routine tests on databases/systems as well as conduct gap analysis to determine what new tests or processes might enhance operational confidence * Develop new ETL processes or software components to satisfy use-cases as suggested by client

* 4+ years of experience with Hadoop development and implementation. Good Knowledge on Hadoop Architecture and HDFS. Proven understanding with Hadoop, HBase, Hive, Pig, and HBase. * At least five (5) years Information Technology experience, including at least one (1) year of hands-on experience programming on a high-scale or distributed system * Hands-on experience with the Hadoop stack, and experience with at least two of: Cassandra, Spark, Kafka * Familiarity with Linux (command-line), Java programming language, and a scripting language (bash/Perl/Python or other) * Bachelor’s degree in Computer Science, Management Information Systems, or a related field * Demonstrated experience with a Hadoop platform (Hortonworks, Cloudera, MapR) * Experience with "productionalizing" Hadoop applications (e.g. administration, configuration management, monitoring, debugging, and performance tuning) * Demonstrated presentation and communication skills