Senior Data Engineer

1 month ago

Cape Town, Western Cape, Saudi Arabia CDW Full time

KEY DUTIES

  • CDW is undergoing a roll out of our Azure data platform, and we are looking for a Data Engineer to join a growing team to ingest, cleanse and model data into single source of truth Kimball datasets, used for Business Intelligence reporting and Data Science machine learning use cases.
  • Ingest data from various sources, including on premise SQL databases, REST API's, Apache Kafka streams, etc.
  • Define and apply cleansing rules to the data to ensure meets quality expectations.
  • Define and model data into single source of truth Kimball datasets, e.g. dimensions and facts.
  • DevOps approach to software; create high quality code using traditional software practices, such as building, linting, unit and integration testing, repositories, CI/CD, peer reviews, etc.
  • DevOps approach to support; monitor pipelines to ensure businesscritical data pipelines are processed meeting time constraint and data quality expectations including implementing reactive changes where applicable.
  • Drives and Integrates into the data team's WOW, including backlog refinement, sprint planning, demonstrations, and retrospectives, including translation of user requirements into technical requirements including complexity estimation, implement into sprint cycles, collaboration into process improvement, etc.
  • Based on requirements, perform ad-hoc analysis of structured and unstructured data across multiple data sources to inform solution design.
  • Document datasets in data catalogue, including ownership, stewardship, dictionaries, glossaries, lineage, sensitivity, etc.
  • Ownership of Work Items and works with data owners / stewards to ensure high quality, aligned deliveries, and compliance with legislation, such as GDPR, e.g. PII vs Non-PII, data retention, etc
  • Document solution design in wiki.

KNOWLEDGE AND EXPERIENCE

Must Have:

  • Architecture, modelling and leadership skills
  • Strong Azure data skills, including:
  • Azure Data Factory V2
  • Azure Data Lake Storage V2
  • Azure Databricks o Azure Function Apps& Logic Apps
  • Azure Stream Analytics
  • Azure Resource Manager skills Terraform, Azure Portal, Az CLI and Az PowerShell
  • Strong PySpark, Delta Lake, Unity Catalog and Python skills.
  • Includes ability to write unit and integration tests in Python with unittest, pytest, etc.
  • Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc.
  • Strong repositories, CI/CD skills.
  • Strong knowledge of Kimball data modelling, such as star schema, snowflake, etc.
  • Strong SQL skills.
  • Strong data analysis skills.
  • Excellent written and verbal communication skills
  • A minimum of 2 years of experience as an Architect
  • A demonstrable track record of getting stuff done whilst managing competing pressures and deadlines and retaining an eye for detail and quality.
  • A passion for technology and its ability to have a positive impact on business.

Nice To Have:

  • Azure DevOps git, multistage YAML, etc.
  • Other languages, such as C#, PowerShell
  • IaC, e.g. Terraform, ARM, Bicep, etc.
  • Test driven development TDD
  • Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc.
  • Power BI Engineering experience
  • Certified SCRUM Developer CSD
  • Machine learning and Artificial intelligence

  • Azure Data Engineer

    1 month ago

    Cape Town, Western Cape, Saudi Arabia CDW Full time

    KEY DUTIESCDW is undergoing a roll out of our Azure data platform, and we are looking for a Data Engineer to join a growing team to ingest, cleanse and model data into single source of truth Kimball datasets, used for Business Intelligence reporting and Data Science machine learning use cases.Ingest data from various sources, including on premise SQL...

  • Azure Data Engineer

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia CDW Full time

    KEY DUTIESCDW is undergoing a roll out of our Azure data platform, and we are looking for a Data Engineer to join a growing team to ingest, cleanse and model data into single source of truth Kimball datasets, used for Business Intelligence reporting and Data Science machine learning use cases.Ingest data from various sources, including on premise SQL...

  • Data Scientist

    1 month ago

    Cape Town, Western Cape, Saudi Arabia Principa Full time

    PRIMARY PURPOSE OF THE JOB:The Analyst / Data Scientist is one of the individuals contributing to our Consulting and Analytics team working on the development and delivery of reporting, credit risk strategy, predictive models and prescriptive models across the consumer credit life cycle in various industries.You need to be passionate about data and...

  • Data Scientist

    2 weeks ago

    Cape Town, Western Cape, Saudi Arabia Principa Full time

    PRIMARY PURPOSE OF THE JOB:The Analyst / Data Scientist is one of the individuals contributing to our Consulting and Analytics team working on the development and delivery of reporting, credit risk strategy, predictive models and prescriptive models across the consumer credit life cycle in various industries.You need to be passionate about data and...

  • Data Scientist

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Principa Full time

    PRIMARY PURPOSE OF THE JOB:The Analyst / Data Scientist is one of the individuals contributing to our Consulting and Analytics team working on the development and delivery of reporting, credit risk strategy, predictive models and prescriptive models across the consumer credit life cycle in various industries.You need to be passionate about data and...

  • Senior Data Analyst: Performance Marketing

    2 months ago

    Cape Town, Western Cape, Saudi Arabia National Sea Rescue Institute Full time

    Minimum RequirementsBachelor's or advanced degree in a quantitative field e.g., Statistics, Mathematics, Economics, Computer Science.Proven experience 8-10 years in data analysis, preferably within a performance marketing or digital advertising environment.Proficient in data visualization tools e.g., Tableau, Power BI and statistical analysis tools e.g., R,...

  • Senior Data Analyst: Performance Marketing

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia National Sea Rescue Institute Full time

    Minimum RequirementsBachelor's or advanced degree in a quantitative field e.g., Statistics, Mathematics, Economics, Computer Science.Proven experience 8-10 years in data analysis, preferably within a performance marketing or digital advertising environment.Proficient in data visualization tools e.g., Tableau, Power BI and statistical analysis tools e.g., R,...

  • Cloud Data Engineer

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Nebula Full time

    Job Opportunity at 1Nebula: The purpose of this position is to ensure that data is properly collected, stored, processed, and made available to other stakeholders within the organization. What You will do within this role Your key responsibilities: Design and develop data pipelines and data architectures to facilitate data access and management in the cloud....

  • Cloud Data Engineer

    2 months ago

    Cape Town, Western Cape, Saudi Arabia Nebula Full time

    Job Opportunity at 1Nebula: The purpose of this position is to ensure that data is properly collected, stored, processed, and made available to other stakeholders within the organization. What You will do within this role Your key responsibilities: Design and develop data pipelines and data architectures to facilitate data access and management in the cloud....

  • Chief Data Officer

    4 days ago

    Cape Town, Western Cape, Saudi Arabia GVW Group Full time

    Summary Description:Aculocity is a dynamic and innovative IT Services company committed to leveraging cutting-edge Artificial Intelligence AI and Machine Learning ML technologies to drive business growth and excellence. We are seeking a talented and experienced Chief Data Officer with strong AI/ML background to lead our AI/ML initiatives and contribute to...

  • Senior Frontend Engineer

    5 days ago

    Cape Town, Western Cape, Saudi Arabia Zenysis Technologies Full time

    DescriptionZenysis is a global technology startup that builds an integration and advanced analytics software that empowers governments and international organizations to deliver lifesaving health programs in a truly-data driven way.We are a diverse team of software engineers, diplomats, public health and development experts, and we are on a mission to...

  • Senior Specialist: Senior Specialist: Machine Learning Engineer

    2 months ago

    Cape Town, Western Cape, Saudi Arabia Vodafone Global Enterprise Full time

    Role Purpose/Business Unit:Are you ready to join Vodacom's exciting world of Digital IT and become part of a high performing, dynamic technology team? With our customers at the heart of our purpose, you will be part of the evolution of our IT technology landscape, harnessing the latest technical innovations available.The SS Machine Learning Engineer delivers...

  • Senior Specialist: Senior Specialist: Machine Learning Engineer

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Vodafone Global Enterprise Full time

    Role Purpose/Business Unit:Are you ready to join Vodacom's exciting world of Digital IT and become part of a high performing, dynamic technology team? With our customers at the heart of our purpose, you will be part of the evolution of our IT technology landscape, harnessing the latest technical innovations available.The SS Machine Learning Engineer delivers...

  • Cloud Data Architect

    2 months ago

    Cape Town, Western Cape, Saudi Arabia Nebula Full time

    Job Opportunity at 1Nebula: The purpose of this position is to ensure that the organization's data is properly managed, stored, and used to support the organization's business goals and objectives. What You will do within this role Your key responsibilities: Design and develop cloud-based data strategies, architectures, and solutions that enable the...

  • Cloud Data Architect

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Nebula Full time

    Job Opportunity at 1Nebula: The purpose of this position is to ensure that the organization's data is properly managed, stored, and used to support the organization's business goals and objectives. What You will do within this role Your key responsibilities: Design and develop cloud-based data strategies, architectures, and solutions that enable the...

  • Data & Digital Academy Internships 2024

    2 months ago

    Cape Town, Western Cape, Saudi Arabia Sanlam Full time

    Sanlam: Data & Digital Academy Internship Programme 2024An exciting opportunity is available for soon-to-be or recent graduates to join the Sanlam Data & Digital Academy. We are seeking bright minds who are eager to build and develop technical, analytical, and business skills while working on real business cases within the Sanlam Business Intelligence and...

  • Data & Digital Academy Internships 2024

    2 weeks ago

    Cape Town, Western Cape, Saudi Arabia Sanlam Full time

    Sanlam: Data & Digital Academy Internship Programme 2024An exciting opportunity is available for soon-to-be or recent graduates to join the Sanlam Data & Digital Academy. We are seeking bright minds who are eager to build and develop technical, analytical, and business skills while working on real business cases within the Sanlam Business Intelligence and...

  • Data & Digital Academy Internships 2024

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Sanlam Full time

    Sanlam: Data & Digital Academy Internship Programme 2024An exciting opportunity is available for soon-to-be or recent graduates to join the Sanlam Data & Digital Academy. We are seeking bright minds who are eager to build and develop technical, analytical, and business skills while working on real business cases within the Sanlam Business Intelligence and...

  • Director Data Strategy and Reporting, EMEA

    2 weeks ago

    Cape Town, Western Cape, Saudi Arabia Marriott International Full time

    Role Summary The Director Data Strategy & Reporting, EMEA is the lead for data strategy, data science, data engineering and commercial performance monitoring in Europe, Middle East and Africa. The role leads a team of data experts and is responsible for planning, developing, implementing, and evaluating data-related strategy and adoption of data products.The...

  • Director Data Strategy and Reporting, EMEA

    4 weeks ago

    Cape Town, Western Cape, Saudi Arabia Marriott International Full time

    Role Summary The Director Data Strategy & Reporting, EMEA is the lead for data strategy, data science, data engineering and commercial performance monitoring in Europe, Middle East and Africa. The role leads a team of data experts and is responsible for planning, developing, implementing, and evaluating data-related strategy and adoption of data products.The...