150 Data Pipelines jobs in South Africa

Johannesburg | R900PH | 12-month Contract

This is an excellent opportunity for a Data Engineer to design, build, and optimize large-scale data pipelines and cloud-based data platforms that underpin investment decision-making, risk management, and regulatory reporting

COMPANY

This company is one of South Africa's largest investment managers, responsible for managing more than R600 billion in assets across both institutional and retail clients. Their focus is on delivering long-term, sustainable returns through innovation, discipline, and a client-first approach. With a strong presence locally and across key African markets, they are investing heavily in modern data platforms and engineering capabilities to strengthen their trading, risk, and analytics functions.

THE ROLE

As a Data Engineer, you will be responsible for building and maintaining the data ecosystem that supports trading applications. Your focus will be on designing ETL/ELT pipelines that ingest data from multiple trading systems (such as Alchemy, Murex, and market data feeds) into Snowflake and enterprise data lakes, ensuring accuracy, performance, and scalability. You'll work on API-led integrations (leveraging MuleSoft and custom Python/Java services) to enable seamless data flow between applications, while embedding governance and security controls required in a regulated financial environment.

• Design, build, and optimize data pipelines for ingestion, transformation, and delivery of financial datasets.
• Model and manage Snowflake data warehouses and cloud-based data lakes.
• Develop APIs and integration solutions for downstream systems, risk engines, and reporting platforms.
• Monitor and optimize pipeline performance to handle large, complex data volumes for valuations and risk reporting.
• Enforce data governance, lineage, and security policies, including data masking and access controls.
• Collaborate with business stakeholders, analysts, and reporting teams to ensure data meets evolving business and regulatory requirements.

THE REQUIREMENTS

At least 5 years' experience in Data Engineering (financial services experience highly advantageous).

Strong SQL skills with both relational and NoSQL databases.

Expertise in ETL/ELT pipeline design using tools such as Airflow, Talend, or custom scripting.

Proficiency in Python, Java, or Scala for building integrations and transformations.

Experience with Snowflake or other cloud data warehouses (AWS Redshift, BigQuery).

Solid knowledge of data modeling, data lakes, and dimensional warehousing methodologies.

Exposure to big data frameworks such as Spark or Hadoop.

Familiarity with MuleSoft or other API integration platforms.

Location

City of Ekurhuleni

Remuneration

R 957,408 to R 1,313,316 - Basic salary per annum (plus benefits)

Assignment Type

Permanent

Minimum Requirements:

• B Degree in Computer Science / Information Systems / Technology and Engineering or relevant equivalent NQF Level 7 qualification
• TOGAF / SABSA Certification
• 5 years' experience in application architecture, solution architecture design including: strategic, tactical and operational thinking and leadership with strong abilities in relationship management; developing and management of application architecture artefacts; established a tract record of managing ICT staff and ICT services; and demonstrated project management competency and the execution of multiple projects, including managing resources across multiple projects
• A valid driver's license

Core Responsibilities:

• Design and analyse software projects. Create new applications or improve existing applications, run software tests, develop product prototypes and create technical documents and manuals relating to application development. Evaluate application technologies and make recommendations for best uses for the City. Ensure compliance with ICT standards and the enterprise architecture as defined by the City. Train other team members in areas of programming and software development
• Provide input to strategic capability and leadership to ensure alignment with the City's and ICT strategy
• Ensure effective operational processes to ensure effective and efficient execution of assigned tasks related to the delivery of architecture projects and thereby contribute towards the achievement of performance goals by the Department
• Perform governance and risk management functions by maintaining compliance with policies and procedures to ensure promotion of good governance, compliance with applicable legislative and policy prescripts and effective and efficient management of risks by the Department
• Manage effective financial processes by performing assessment of the potential of emerging technologies and innovation ideas to contribute towards prudent financial management by the Department at large
• Ensure customer service orientation to promote people centered and efficient service delivery by the Department in line with the Batho Pele principles

Enquiries: Tel:

The successful candidate will be required to sign a performance contract as per Council resolution.

Candidates may be requested to submit proof of the NQF level applicable to the qualifications they have obtained.

Should candidates not be informed of the outcome of the application within six weeks of the closing date, it should be regarded as having been unsuccessful. The City of Ekurhuleni subscribes to the principles of the Employment Equity Act and reserves the right to appoint, not appoint or to appoint laterally.

Sign up to Apply for this Job

Job category: Others: IT and Telecommunication

Location: Bedfordview

Contract: Permanent

Remuneration: Market Related

EE position: No

Applications are invited for the position of Manager: Data Integration & Operations (Peromnes Grade 7) based at Head Office, Bruma. The successful applicant will be reporting to the Head of Data Management. The purpose of this role is to ensure effective data management, formulate quality data collection techniques, implement efficient and secure procedures for data handling and analysis, and establish data sharing rules and proper governance.

1. Evaluate and recommend data management tools and technologies.
2. Implement and manage master data management solutions for core data entities.
3. Ensure consistent and accurate master data across the organization.
4. Lead data profiling, cleansing, and validation initiatives.
5. Monitor and report on data quality metrics and drive continuous improvement.
6. Define and implement effective data pipeline jobs to Extract, Transform and Load (ETL) data.
7. Collaborate with expert teams to design and maintain data architectures that support business needs.
8. Conduct regular data security audits and risk assessments.
9. Ensure compliance with data privacy regulations (e.g., POPIA, HIPAA).
10. Develop and manage strategic relationships with key IT product and service providers.
11. Manage staff in accordance with HC policies and ensure effective performance management.

• B.Eng. or BSc in Computer Science or related field such as Electronics/Electrical/Computer Engineering/Data Management/Information Systems.
• Postgraduate degree in Computer Science or related field will be an advantage .

• Minimum 5 years’ experience in information technology, data management, and engineering within an operations environment, preferably air traffic services/air traffic management, with at least 3 years at management level.
• Proven experience in data management and familiarity with database technologies.

Interested applicants are requested to register their CV’s and apply on by no later than 28 February 2025.

ATNS is an equal opportunity employer that strives to achieve a diverse workforce broadly representative of our people. This position will be filled in line with the objectives of ATNS’ Employment Equity Plan.

People with disabilities are encouraged to apply.

Empowering Africa's tomorrow, together…one story at a time.
With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary
Technology Enablement owns the data processes that enable sustainable, accurate and timeous financial information in support of internal and external reporting. The individual will operate in Technology Enablement with primary focus to own master data principals and design and implement own the mapping rules between product systems and ledger on all Interface implementations. Support master data and manual journal postings as normal BAU where required.

Job Description
Accountability: Design and Maintain Data Interfaces

• Understand the product system and the file formats and interface methods required to interface into the ledger.
• Own all master data principals and ensure adherence to master data principals in all postings into ledger
• Develop a strategy to ensure that operations, systems, and procedures are appropriate to absorb the impact of changes to increase the financial operations for Absa Group and specifically in the Financial Control environment.
• Knowledge expert in how interfacing into the ledger works and what is required for a successful implementation.
• Manage stakeholder engagement in order to ensure consensus and approval of the data interfaces.
• Be pro-active and manage changes impacting SAP S/4 HANA, enable long-term planning and input development.
• Maintain DI and replication mapping tables
• Liaise and engage with Functions Technology to discuss system and technology enhancements to increase performance of data integrity.
• Obtain and sustain knowledge and experience on the end-to-end processes, including product systems and general ledgers, to support business units with enhanced service delivery.
• Manage adherence to the standard interface testing procedure for all new systems before interfacing with SAP S/4 HANA to ensure prevention of rejections and miss-representation of financial information.
• Support business and own the process where manual journals needs to be processed as part of data clean ups or where the automated journal tool do not have the required functionality to ensure a successful posting.
• Manage research, design, promotion, development and testing of innovative improvements or new solutions that will improve processes and ensure financial data integrity.

Accountability: Business direction and support

• Work with different Stakeholders to achieve excellent business results through continues engagements
• Contribute to the development of business unit strategy.
• Contribute to the development and implementation of the vision for the team.
• Agree targets and take accountability for the overall achievement of performance objectives in the business area.
• Define customer experience metrics.

Accountability: Risk management, compliance, and controls

• Ensure that the team understands applicable risk management, compliance, and control requirements, including required compliance training.
• Keep up to date with all the relevant legislation/regulatory requirements.
• Own and agree corrective action items with Internal and External Audit for findings related to data interfaces and integrity.
• Participate in the development of the Business Continuity Plan by reviewing and ensuring the critical financial processes are adequately covered and tested.
• Perform proper maintenance of all actions needed for the completion of DEA (Design Effectiveness Assessment) and OET (Operating Effectiveness Testing).

Accountability: People and talent management

• Exhibits Absa Way, Purpose and Values and champions them in the team towards development of a corporate culture.
• With the support from the HR Business Partner, interview and recruit direct reports and provide support to them during the recruitment of their teams on request.
• Effectively assign and balance staff allocations to attain the required performance levels.
• Develop suitable induction training solutions for the area of business.
• Ensure that staff is developed in accordance with personal development plans aligned to organisational priorities and individual development areas.
• Ensure that the people and talent management record keeping system is kept up to date.
• Establish and maintain a succession plan for the management team in the area using the formal Talent Management process for identified talent and an informal process for remaining roles.
• Perform quarterly "health checks" with staff and confirm that they are meeting their required level of performance.
• Diversity - manage and make sure that every team member is treated with respect in terms of culture, gender and religion.
• Create effective workforce and recruitment demand plans to ensure that current and future business requirements can be met.

Knowledge & Skills:

• Understanding of business processes
• Excellent ETL skills
• Solid financial background
• Solid accounting knowledge
• Data and data analyses skills
• Knowledge and experience of the full SDLC life cycle
• Networking and relationship management
• Excellent written and verbal communication skills
• Understanding of Absa processes and systems
• Commercial awareness

Education And Experience Required

• B-degree in accounting or information technology (NQF level no. 7)
• At least 5 years banking experience
• At least 5 – 8 years financial reporting experience
• People Management Experience

Competencies:

• Applying expertise and technology
• Working with people
• Analyzing
• Relating and networking
• Crating and innovating
• Persuading and influencing
• Entrepreneurial and commercial thinking

Education
Bachelor`s Degrees and Advanced Diplomas: Physical, Mathematical, Computer and Life Sciences (Required)

Absa Bank Limited is an equal opportunity, affirmative action employer. In compliance with the Employment Equity Act 55 of 1998, preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.
Absa Bank Limited reserves the right not to make an appointment to the post as advertised

Empowering Africa's tomorrow, together…one story at a time.

With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary

Technology Enablement owns the data processes that enable sustainable, accurate and timeous financial information in support of internal and external reporting. The individual will operate in Technology Enablement with primary focus to own master data principals and design and implement own the mapping rules between product systems and ledger on all Interface implementations. Support master data and manual journal postings as normal BAU where required.

Job Description

Accountability: Design and Maintain Data Interfaces

• Understand the product system and the file formats and interface methods required to interface into the ledger.
• Own all master data principals and ensure adherence to master data principals in all postings into ledger
• Develop a strategy to ensure that operations, systems, and procedures are appropriate to absorb the impact of changes to increase the financial operations for Absa Group and specifically in the Financial Control environment.
• Knowledge expert in how interfacing into the ledger works and what is required for a successful implementation.
• Manage stakeholder engagement in order to ensure consensus and approval of the data interfaces.
• Be pro-active and manage changes impacting SAP S/4 HANA, enable long-term planning and input development.
• Maintain DI and replication mapping tables
• Liaise and engage with Functions Technology to discuss system and technology enhancements to increase performance of data integrity.
• Obtain and sustain knowledge and experience on the end-to-end processes, including product systems and general ledgers, to support business units with enhanced service delivery.
• Manage adherence to the standard interface testing procedure for all new systems before interfacing with SAP S/4 HANA to ensure prevention of rejections and miss-representation of financial information.
• Support business and own the process where manual journals needs to be processed as part of data clean ups or where the automated journal tool do not have the required functionality to ensure a successful posting.
• Manage research, design, promotion, development and testing of innovative improvements or new solutions that will improve processes and ensure financial data integrity.

Accountability: Business direction and support

• Work with different Stakeholders to achieve excellent business results through continues engagements
• Contribute to the development of business unit strategy.
• Contribute to the development and implementation of the vision for the team.
• Agree targets and take accountability for the overall achievement of performance objectives in the business area.
• Define customer experience metrics.

Accountability: Risk management, compliance, and controls

• Ensure that the team understands applicable risk management, compliance, and control requirements, including required compliance training.
• Keep up to date with all the relevant legislation/regulatory requirements.
• Own and agree corrective action items with Internal and External Audit for findings related to data interfaces and integrity.
• Participate in the development of the Business Continuity Plan by reviewing and ensuring the critical financial processes are adequately covered and tested.
• Perform proper maintenance of all actions needed for the completion of DEA (Design Effectiveness Assessment) and OET (Operating Effectiveness Testing).

Accountability: People and talent management

• Exhibits Absa Way, Purpose and Values and champions them in the team towards development of a corporate culture.
• With the support from the HR Business Partner, interview and recruit direct reports and provide support to them during the recruitment of their teams on request.
• Effectively assign and balance staff allocations to attain the required performance levels.
• Develop suitable induction training solutions for the area of business.
• Ensure that staff is developed in accordance with personal development plans aligned to organisational priorities and individual development areas.
• Ensure that the people and talent management record keeping system is kept up to date.
• Establish and maintain a succession plan for the management team in the area using the formal Talent Management process for identified talent and an informal process for remaining roles.
• Perform quarterly "health checks" with staff and confirm that they are meeting their required level of performance.
• Diversity - manage and make sure that every team member is treated with respect in terms of culture, gender and religion.
• Create effective workforce and recruitment demand plans to ensure that current and future business requirements can be met.

Knowledge & Skills:

• Understanding of business processes
• Excellent ETL skills
• Solid financial background
• Solid accounting knowledge
• Data and data analyses skills
• Knowledge and experience of the full SDLC life cycle
• Networking and relationship management
• Excellent written and verbal communication skills
• Understanding of Absa processes and systems
• Commercial awareness

Education and Experience Required

• B-degree in accounting or information technology (NQF level no. 7)
• At least 5 years banking experience
• At least 5 – 8 years financial reporting experience
• People Management Experience

Competencies:

• Applying expertise and technology
• Working with people
• Analyzing
• Relating and networking
• Crating and innovating
• Persuading and influencing
• Entrepreneurial and commercial thinking

Education

Bachelor`s Degrees and Advanced Diplomas: Physical, Mathematical, Computer and Life Sciences (Required)

Absa Bank Limited is an equal opportunity, affirmative action employer. In compliance with the Employment Equity Act 55 of 1998, preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.

Absa Bank Limited reserves the right not to make an appointment to the post as advertised

Empowering Africa's tomorrow, together…one story at a time.
With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary
Technology Enablement owns the data processes that enable sustainable, accurate and timeous financial information in support of internal and external reporting. The individual will operate in Technology Enablement with primary focus to design, build and maintain SAP S/4 HANA interfaces.

Job Description
Accountability: Design and Maintain Data Interfaces

• Develop data and information architecture strategy, blueprint and roadmap and track progress towards achieving the strategic goals set out.
• Develop a strategy to ensure that operations, systems, and procedures are appropriate to absorb the impact of changes to increase the financial operations for Absa Group and specifically in the Financial Control environment.
• Provide leadership on SAP S/4 HANA functionalities for current and future financial system landscape.
• Manage stakeholder engagement in order to ensure consensus and approval of the data interfaces.
• Be pro-active and manage changes impacting SAP S/4 HANA, enable long-term planning and input development.
• Design, build and manage SAP S/4 HANA DI processes
• Maintain DI and replication mapping tables
• Liaise and engage with Functions Technology to discuss system and technology enhancements to increase performance of data integrity.
• Obtain and sustain knowledge and experience on the end-to-end processes, including product systems and general ledgers, to support business units with enhanced service delivery.
• Obtain and maintain technical knowledge, skills, and benchmarking to keep up to date in specialist areas to perform at the required standards.
• Manage adherence to the standard interface testing procedure for all new systems before interfacing with SAP S/4 HANA to ensure prevention of rejections and miss-representation of financial information.
• Manage research, design, promotion, development and testing of innovative improvements or new solutions that will improve processes and ensure financial data integrity.

Accountability: Business direction and support

• Work with different Stakeholders to achieve excellent business results through continues engagements
• Contribute to the development of business unit strategy.
• Contribute to the development and implementation of the vision for the team.
• Agree targets and take accountability for the overall achievement of performance objectives in the business area.
• Define customer experience metrics.

Accountability: Risk management, compliance, and controls

• Ensure that the team understands applicable risk management, compliance, and control requirements, including required compliance training.
• Keep up to date with all the relevant legislation/regulatory requirements.
• Own and agree corrective action items with Internal and External Audit for findings related to data interfaces and integrity.
• Participate in the development of the Business Continuity Plan by reviewing and ensuring the critical financial processes are adequately covered and tested.
• Perform proper maintenance of all actions needed for the completion of DEA (Design Effectiveness Assessment) and OET (Operating Effectiveness Testing).

Accountability: People and talent management

• Exhibits Absa Way, Purpose and Values and champions them in the team towards development of a corporate culture.
• With the support from the HR Business Partner, interview and recruit direct reports and provide support to them during the recruitment of their teams on request.
• Effectively assign and balance staff allocations to attain the required performance levels.
• Develop suitable induction training solutions for the area of business.
• Ensure that staff is developed in accordance with personal development plans aligned to organisational priorities and individual development areas.
• Ensure that the people and talent management record keeping system is kept up to date.
• Establish and maintain a succession plan for the management team in the area using the formal Talent Management process for identified talent and an informal process for remaining roles.
• Perform quarterly "health checks" with staff and confirm that they are meeting their required level of performance.
• Diversity - manage and make sure that every team member is treated with respect in terms of culture, gender and religion.
• Create effective workforce and recruitment demand plans to ensure that current and future business requirements can be met.

Knowledge & Skills

• Understanding of business processes
• Excellent ETL skills
• Solid SAP S/4 DI skills
• Solid financial background
• Solid accounting knowledge
• Networking and relationship management
• Excellent written and verbal communication skills
• Understanding of Absa processes and systems
• Commercial awareness

Education And Experience Required

• B-degree in information technology (NQF level no. 7)
• At least 5 years SAP DI experience
• At least 5 years banking experience
• At least 5 – 8 years financial reporting experience
• People Management Experience

Competencies

• Applying expertise and technology
• Working with people
• Analyzing
• Relating and networking
• Crating and innovating
• Persuading and influencing
• Entrepreneurial and commercial thinking

Education
Bachelors Degrees and Advanced Diplomas: Data Science, Bachelors Degrees and Advanced Diplomas: Information Technology, Bachelor`s Degrees and Advanced Diplomas: Manufacturing, Engineering and Technology

Absa Bank Limited is an equal opportunity, affirmative action employer. In compliance with the Employment Equity Act 55 of 1998, preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.
Absa Bank Limited reserves the right not to make an appointment to the post as advertised

Empowering Africa's tomorrow, together…one story at a time.

With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.

Job Summary

Technology Enablement owns the data processes that enable sustainable, accurate and timeous financial information in support of internal and external reporting. The individual will operate in Technology Enablement with primary focus to design, build and maintain SAP S/4 HANA interfaces.

Job Description

Accountability: Design and Maintain Data Interfaces

• Develop data and information architecture strategy, blueprint and roadmap and track progress towards achieving the strategic goals set out.
• Develop a strategy to ensure that operations, systems, and procedures are appropriate to absorb the impact of changes to increase the financial operations for Absa Group and specifically in the Financial Control environment.
• Provide leadership on SAP S/4 HANA functionalities for current and future financial system landscape.
• Manage stakeholder engagement in order to ensure consensus and approval of the data interfaces.
• Be pro-active and manage changes impacting SAP S/4 HANA, enable long-term planning and input development.
• Design, build and manage SAP S/4 HANA DI processes
• Maintain DI and replication mapping tables
• Liaise and engage with Functions Technology to discuss system and technology enhancements to increase performance of data integrity.
• Obtain and sustain knowledge and experience on the end-to-end processes, including product systems and general ledgers, to support business units with enhanced service delivery.
• Obtain and maintain technical knowledge, skills, and benchmarking to keep up to date in specialist areas to perform at the required standards.
• Manage adherence to the standard interface testing procedure for all new systems before interfacing with SAP S/4 HANA to ensure prevention of rejections and miss-representation of financial information.
• Manage research, design, promotion, development and testing of innovative improvements or new solutions that will improve processes and ensure financial data integrity.

Accountability: Business direction and support

• Work with different Stakeholders to achieve excellent business results through continues engagements
• Contribute to the development of business unit strategy.
• Contribute to the development and implementation of the vision for the team.
• Agree targets and take accountability for the overall achievement of performance objectives in the business area.
• Define customer experience metrics.

Accountability: Risk management, compliance, and controls

• Ensure that the team understands applicable risk management, compliance, and control requirements, including required compliance training.
• Keep up to date with all the relevant legislation/regulatory requirements.
• Own and agree corrective action items with Internal and External Audit for findings related to data interfaces and integrity.
• Participate in the development of the Business Continuity Plan by reviewing and ensuring the critical financial processes are adequately covered and tested.
• Perform proper maintenance of all actions needed for the completion of DEA (Design Effectiveness Assessment) and OET (Operating Effectiveness Testing).

Accountability: People and talent management

• Exhibits Absa Way, Purpose and Values and champions them in the team towards development of a corporate culture.
• With the support from the HR Business Partner, interview and recruit direct reports and provide support to them during the recruitment of their teams on request.
• Effectively assign and balance staff allocations to attain the required performance levels.
• Develop suitable induction training solutions for the area of business.
• Ensure that staff is developed in accordance with personal development plans aligned to organisational priorities and individual development areas.
• Ensure that the people and talent management record keeping system is kept up to date.
• Establish and maintain a succession plan for the management team in the area using the formal Talent Management process for identified talent and an informal process for remaining roles.
• Perform quarterly "health checks" with staff and confirm that they are meeting their required level of performance.
• Diversity - manage and make sure that every team member is treated with respect in terms of culture, gender and religion.
• Create effective workforce and recruitment demand plans to ensure that current and future business requirements can be met.

Knowledge & Skills:

• Understanding of business processes
• Excellent ETL skills
• Solid SAP S/4 DI skills
• Solid financial background
• Solid accounting knowledge
• Networking and relationship management
• Excellent written and verbal communication skills
• Understanding of Absa processes and systems
• Commercial awareness

Education and Experience Required

• B-degree in information technology (NQF level no. 7)
• At least 5 years SAP DI experience
• At least 5 years banking experience
• At least 5 – 8 years financial reporting experience
• People Management Experience

Competencies:

• Applying expertise and technology
• Working with people
• Analyzing
• Relating and networking
• Crating and innovating
• Persuading and influencing
• Entrepreneurial and commercial thinking

Education

Bachelors Degrees and Advanced Diplomas: Data Science, Bachelors Degrees and Advanced Diplomas: Information Technology, Bachelor`s Degrees and Advanced Diplomas: Manufacturing, Engineering and Technology

Absa Bank Limited is an equal opportunity, affirmative action employer. In compliance with the Employment Equity Act 55 of 1998, preference will be given to suitable candidates from designated groups whose appointments will contribute towards achievement of equitable demographic representation of our workforce profile and add to the diversity of the Bank.

Absa Bank Limited reserves the right not to make an appointment to the post as advertised