BigData Platform Engineer

Job Details

  • ID#37679279
  • Address 92626 , Costamesa,

    California

    Costamesa USA
  • Job type

    Permanent

  • Salary USD Depends on Experience Depends on Experience
  • Hiring Company

    Vertogic

  • Showed04th April 2022
  • Date02nd April 20222022-04-02T03:00:00-0400
  • Deadline01st June 2022
  • Category

    Et cetera

BigData Platform Engineer

Vacancy expired!

About the Role: We are looking for an expert platform engineer with in-depth knowledge on Big Data analytics & public cloud platforms to help us run this peta-byte scale BigData platform and provide the best possible experience for our clients.

What You’ll Do Here • Responsible for continuous platform enhancements, upgrades, availability, reliability and security of the Ascend Sandbox platform.• Provide end-to-end observability of our Ascend Sandbox platform.• Responsible for resolving incidents reported by Sandbox users and take preventive actions.• Help Sandbox users with troubleshooting failed MapReduce/Hive/Spark applications.• Help Sandbox users to improve the performance and optimize their MapReduce/Hive/Spark applications.• Participate in follow-the-sun on-call rotation to address any emergency production incidents affecting the Sandbox platform.

What You'll Need To Succeed Must Have skills: • Deep understanding of Linux, networking fundamentals and security.
  • Solid professional coding experience with at least one scripting language - Shell, Python etc.• Experience working with AWS cloud platform and infrastructure.• Experience managing large BigData clusters in production (at least one of Cloudera, Hortonworks, EMR)• Excellent knowledge and solid work experience providing observability for BigData platforms using tools like Prometheus, InfluxDB, Dynatrace, Grafana, Splunk etc.• Experience managing BigData clusters with compute decoupled from storage (Eg: S3) on public cloud platforms.• Expert knowledge on Hadoop Distributed File System (HDFS) and Hadoop YARN.• Decent knowledge of various Hadoop file formats like ORC, Parquet, Avro etc.• Deep understanding of Hive (Tez), Hive LLAP, Presto and Spark compute engines.• Ability to understand query plans and optimize performance for complex SQL queries on Hive and Spark.• Hands on experience supporting Spark with Python (PySpark) and R (SparklyR, SparkR) languages.• Experience working with Data Analysts, Data Scientists and at least one of these related analytical applications like SAS, R-Studio, JupyterHub, H2O etc.• Able to read and understand code (Java, Python, R, Scala), but expertise in at least one scripting language.• Experience managing JVM based applications in production.• Excellent written and oral communication.

    Nice to have skills: • Previous experience leading or playing critical role in production migration from one BigData analytics platform to other, preferably from CDH to CDP.• Experience with workflow management tools like Airflow, Oozie etc.• Implementation history of Terraform, Packer, Ansible, Chef, Jenkins or any other similar tooling.• Prior working knowledge of Active Directory and Windows OS based VDI platforms like Citrix, AWS Workspaces etc.• Professional coding experience in at least one programming language, preferably Java.• Experience with other public cloud platforms like Azure and Google Cloud Platform is a bonus.

Vacancy expired!

Similar jobs

BigData Platform Engineer

Vertogic - BigData Platform Engineer

BigData Platform Engineer

Vertogic - BigData Platform Engineer

BigData Platform Engineer

Vertogic - BigData Platform Engineer

BigData Platform Engineer

Vertogic - BigData Platform Engineer