Big Data Administrator

**Job Summary**:
**Key Responsibilities**:
**Informatica Administration**:
1. Install, configure, and maintain Informatica PowerCenter and Informatica Cloud Data Integration environments, ensuring optimal performance and availability.

2. Manage and monitor Informatica repository, domain, and services, ensuring smooth operations across development, testing, and production environments.

3. Configure and administer Informatica workflows, sessions, and sessions logs for ETL processing, ensuring data pipelines run effectively.

4. Handle user access management, security, and role assignments within the Informatica environment.

6. Implement performance tuning of Informatica workflows and jobs to optimize resource utilization and processing times.

**Cloudera Administration**:
1. Install, configure, and maintain the Cloudera Distribution of Hadoop (CDH), including components like HDFS, Hive, Impala, Spark, YARN, and HBase.

2. Manage the Cloudera Manager and perform regular system checks to monitor the health and performance of the Hadoop ecosystem.

3. Implement and enforce data security policies across Cloudera platform services, including configuring authentication (Kerberos), encryption, and user access management.

4. Oversee the Cloudera Cluster management, including node configurations, service management, and ensuring efficient resource allocation.

5. Perform troubleshooting and performance optimization of HDFS, Hive, Spark, and other Cloudera components to minimize job failures and reduce processing times.

6. Coordinate with other teams for provisioning additional resources (e.g., compute, storage) in the Cloudera ecosystem as required by growing data demands.

**System Monitoring and Troubleshooting**:
1. Continuously monitor and manage both Informatica and Cloudera environments to ensure high availability, mínimal downtime, and maximum performance.

2. Set up and configure alerting and logging mechanisms for both platforms to proactively address performance bottlenecks, job failures, and resource utilization.

3. Troubleshoot and resolve issues related to system performance, data processing errors, and infrastructure failures across the Informatica and Cloudera environments.

4. Investigate and resolve data discrepancies, failed jobs, and system performance issues by working with Data Engineers and Business Intelligence teams.

**Performance Tuning and Optimization**:
1. Optimize Informatica workflows, mappings, and transformations to enhance execution times and reduce resource consumption.

2. Tune Hadoop components such as HDFS, YARN, Hive, and Spark to ensure efficient data processing and minimize latency.

3. Perform routine system checks and implement corrective actions to maintain optimal performance across both the Informatica and Cloudera platforms.

**Data Backup and Recovery**:
1. Manage and configure data backups and disaster recovery processes for Informatica and Cloudera environments.

2. Ensure data is recoverable in the event of failure or system downtime, and manage recovery procedures based on business continuity plans.

**Required Qualifications**:
1. Education: Major in Computer Science or related filed.

2. Years of experience: 4+

3. Informatica Administration: Strong experience in installing, configuring, and managing Informatica platforms, such as PowerCenter, IDQ, EDC, MDM, Axon, etc.

4. Cloudera Administration: Proficient experience with administering Cloudera Distribution of Hadoop (CDH), including components like HDFS, Hive, Impala, YARN, and Spark.

5. Expertise in managing Informatica repositories, services, workflows, and performance tuning.

6. Ensure end-to-end data pipeline visibility by maintaining logs, error reporting, and alerting mechanisms for ETL jobs, using platforms like Cloudera Manager or custom logging solutions.

7. Troubleshoot bottlenecks and optimize resource consumption in Cloudera, Informatica, and Alteryx to reduce costs and improve efficiency.

8. Implement monitoring and alerting systems to track system health, job performance, and data processing metrics.