Requirements:

• Should have a bachelor’s degree in field of Statistics, Business Intelligence or an equivalent
• Should have got relevant experience in working with Hadoop, Spark environment and other related tools such as Hive, Presto, Ranger etc.
• Should have got relevant experience deploying Hadoop/Spark with cluster manager YARN, Kubernetes, standalone.
• Should have relevant experience in setting up Hadoop/Spark/Kafka cluster and configuring the environment to work with other ETL & reporting tools.
• Should have got administration experience on Linux systems and have proficiency in Shell scripting
• Should have got experience in working on node administration – such as cluster management, job management, resource management, installation, configuration , security.
• Should be capable of handling user security and integration among tools within the environment.
• Should have got experience implementing Single Sign On capability with SAML or equivalent.
• Should be capable of working towards performance optimization of the platform
• Should be capable of managing backup & recovery tasks, high availability, monitoring of cluster
• Should have experience working with capacity planning, scaling up/down of nodes
• Should have experience troubleshooting application errors, build custom schedules for start-up services and apply best practices for cluster configuration
• Should have experience working with Apache Hadoop Distribution, experience with Cloudera/Hortonworks Distribution is an addition.
• Should have got experience managing distributed systems on cloud – EMR, HDInsight, Databricks , etc.
• Should have experience administering databases ( SQLServer, PostgreSQL Redshift, Snowflake, etc.)
• Should have administration experience with noSQL databases (Hbase, mongoDB, Redis, etc.)
• Should have experience working in a Datawarehouse development environment
• Should have experience building data pipelines with Apache Airflow.
• Should have expertise in coding with either of the languages - Java, python, Scala
• SQL expertise is a must
• Should have good communication skills to be able to interact with multiple teams