Big Data Engineer

Position Summary: 
As a Big Data Engineer, you will provide technical expertise and aptitude to Hadoop technologies as they relate to the development of analytics at our client company. You will be responsible for the planning and execution of big data analytics, predictive analytics and machine learning initiatives. You will help lead the plan, build, and run states within the Enterprise Analytics Team and act in a lead role driving user story analysis. By creating optimization and stability to the platforms, you will play a key role in the architecture design and data modeling of the platform and analytic applications. You will engage in solving and supporting real business issues with your Hadoop distributed systems and Open Source framework knowledge. 

You will perform detailed analysis of business problems and technical environments and use this data in designing the solution and maintaining data architecture. You will focus on creating strategy, researching emerging technology, and applying technology to enable business solutions within the organization. 

Other responsibilities of the Big Data Engineer include: 

* Designing and developing software applications, testing, and building automation tools. 
* Designing efficient and robust Hadoop solutions for performance improvement and end-user experiences. 
* Working in a Hadoop ecosystem implementation/administration, installing software patches along with system upgrades and configuration. 
* Conducting performance tuning of Hadoop clusters while monitoring and managing Hadoop cluster job performance, capacity forecasting, and security. 
* Defining compute (Storage & CPU) estimations formula for ELT & Data consumption workloads from reporting tools and Ad-hoc users. 
* Analyzing Big Data Analytic technologies and applications in both business intelligence analysis and new service offerings, adopting and implementing these insights and standard methodologies. 

Required Qualifications: 
* 8+ years- experience supporting various enterprise platforms, performance tuning and application performance optimizations. 

* 5+ years' experience working in Linux servers and platform optimization. 

* 4+ years- experience in architecture and implementation of large and highly complex projects using Hortonworks (Hadoop Distributed File System) with Isilon commodity hardware. 

* 4+ years' experience with Big Data platforms and tools and Hadoop implementation experience including the following: 

- Hands on experience in the platform operations. 

- Performance and delivery of Hadoop ecosystem (Hadoop, Hive, Spark, Hbase, Ambari, Kafka, Pyspark & R). 

-Application performance tuning. 

-Experience as a DBA on any RDBMS or Hive Database. 

-Experience managing Big Data platform operations or any other large platform. 


Preferred Qualifications: 
* Understanding of Machine Learning and Artificial Intelligence advanced analytics. 
* Experience in Data Warehouse and BI Analytics. 
* Experience with Spark, R, Python, Scala, Java/C++, and SQL/RDBM. 

Education: 
Bachelor's Degree.