70 Data Preprocessing jobs in South Africa
Manager, Data Engineering
Posted 21 days ago
Job Viewed
Job Description
Business Segment: Insurance & Asset Management
Location: ZA, GP, Roodepoort, Ellis Street 4
To develop and maintain complete data architecture across several application platforms, provide capability across application platforms. To design, build, operationalise, secure and monitor data pipelines and data stores to applicable architecture, solution designs, standards, policies and governance requirements thus making data accessible for the evaluation and optimisation for downstream use case consumption. To execute data engineering duties according to standards, frameworks, and roadmaps.
QualificationsMinimum Qualifications
- Type of Qualification: First Degree
- Field of Study: Business Commerce
- Type of Qualification: First Degree
- Field of Study: Information Studies
- Type of Qualification: First Degree
- Field of Study: Information Technology
Experience Required
- Software Engineering
5-7 years: Experience in building databases, warehouses, reporting and data integration solutions. Experience building and optimising big data data-pipelines, architectures and data sets. Experience in creating and integrating APIs. Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. - 8-10 years: Deep understanding of data pipelining and performance optimisation, data principles, how data fits in an organisation, including customers, products and transactional information. Knowledge of integration patterns, styles, protocols and systems theory.
- 8-10 years: Experience in database programming languages including SQL, PL/SQL, SPARK and or appropriate data tooling. Experience with data pipeline and workflow management tools.
Please note: All our recruitment processes comply with the applicable local laws and regulations. We will never ask for money or any form of
Manager, data engineering
Posted 1 day ago
Job Viewed
Job Description
Director of Data Engineering
Posted 1 day ago
Job Viewed
Job Description
Strategic Leadership:
- Define and drive the strategic vision and roadmap for the Data Engineering pillar, aligned to the company's broader business goals.
- Identify emerging technologies, methodologies, and practices that can deliver value to our customers and keep the business at the forefront of the industry.
- Provide architectural oversight and direction for client and internal projects.
- Engage with clients during the early discovery and solution design phases to understand business goals and shape optimal data architectures and most suitable technologies (currently, Qlik Data Integration (QDI), Qlik Talend, and Microsoft technologies).
- Design, develop, implement and showcase demo pipelines in our current technology stack that demonstrate the power and relevance of our platform capabilities.
- Collaborate with other pillars (Sales and Customer) to support marketing, sales, pre-sales, including scoping, solution design, and estimation.
- Lead, mentor and grow the data engineering team, developing both technical and consulting capabilities.
- Drive internal enablement programs to ensure continuous upskilling across all teams on our core platforms: Qlik Data Integration (QDI), Qlik Talend, and Microsoft technologies.
- Champion a culture of learning, experimentation, and collaboration.
- Define and implement best practices and frameworks for data integration, pipeline orchestration, and data architecture.
- Establish and maintain robust project delivery methodologies that ensure repeatable, scalable, and high-quality implementations.
- Work with the other pillar heads to ensure projects are resourced effectively and delivered on time and budget.
Serve as the thought leader for our key platforms - Qlik Data Integration, Qlik Talend, and Microsoft Azure / Data Factory / Fabric. This may involve interviews, speaking engagements and other marketing activities both internal and external.
Promote innovative uses of technology to solve customer problems in creative, commercially viable ways.
Ensure the alignment of short, medium, and long-term technology choices with customer goals and budget realities.
- 8+ years of experience in data engineering or data integration, with at least 3 years in a leadership role.
- Proven experience designing and leading complex data architectures and pipelines across a range of industries.
- Deep understanding of modern data platforms including:
- Qlik Data Integration (Qlik Replicate, Compose, Enterprise Manager)
- Qlik Talend (Data Integration, Data Quality, ESB)
- Microsoft (Azure Synapse, Data Factory, SQL, Power Platform, Fabric)
- Experience in pre-sales, solutioning, and commercial scoping of data projects.
- Familiarity and experience with data governance, data quality, and metadata management best practices.
Skills & Competencies:
Strong business acumen and the ability to translate between business needs and technical solutions.
Excellent communication and stakeholder engagement skills with both technical and non-technical audiences.
Strategic thinker with a hands-on, solutions-oriented approach.
Collaborative leadership style and ability to work cross-functionally.
Entrepreneurial mindset with the ability to innovate within constraints.
Director of Data Engineering
Posted today
Job Viewed
Job Description
Data Engineering Practice Lead
Posted 13 days ago
Job Viewed
Job Description
Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it's a place where you can grow, belong and thrive.
The Director, Data Engineering is a leadership role accountable for operationally managing a Data Engineering Team.
The Data Engineering Practice lead will lead the design, development of the enterprise data platform and pipelines.
This role is further responsible for overseeing SAP data migrations, ensuring seamless integration and transformation of legacy systems into modern cloud-based architectures.
The person will drive innovation, scalability, and governance in data engineering practices while managing a high-performing team.
**Key Responsibilities:**
+ Manages a team of Managers and Data Engineers to ensure achievement of team objectives/KPIs and through continuous coaching and mentoring helps the team to understand and apply data to solve business solutions.
+ Develop the new Data Platform - Execute the data engineering strategy per the selected platform.
+ SAP Data Migration - Lead end-to-end SAP data migration projects, including planning, mapping, transformation, and validation.
+ Engineering Team Leadership - Build and manage a team of data engineers and technical leads. Foster a culture of innovation, collaboration, and continuous improvement.
+ Data Pipeline Development - Oversee development of robust ETL/ELT pipelines using Azure Data Factory, Databricks, Snowflake Streams/Tasks, and SAP tools. Ensure pipelines are optimized for performance, reliability, and cost-efficiency.
+ Data Management - partner with data management practice lead and governance lead to ensure adherence to data quality, lineage, and security standards. Implement monitoring and alerting for data pipeline health and SLA compliance.
+ Stakeholder Engagement - Collaborate with business units and the internal analytics teams and IT to align data engineering efforts with strategic goals. Translate business requirements into scalable technical solutions.
+ Ensure industry standards best practices are applied to development activities and continuously ensures the team remains abreast of latest technology and tools in the area of data engineering.
+ Works across multiple areas on data set to ensure solutions are successfully executed, within agreed upon time frames.
**Knowledge and Attributes:**
+ Excellent ability to thrive in a dynamic, fast-paced environment.
+ Strong quantitative and qualitative analysis skills.
+ Strong desire to acquire more knowledge to keep up to speed with the ever-evolving field of data science.
+ Strong curiosity to sift through data to find answers and more insights.
+ Excellent understanding of the information technology industry within a matrixed organization and the typical business problems such organizations face.
+ Excellent ability to translate technical findings clearly and fluently to non-technical team business stakeholders to enable informed decision-making.
+ Excellent ability to create a storyline around the data to make it easy to interpret and understand.
+ Self-driven and able to work independently yet acts as a team player.
+ Excellent ability to apply data science principles through a business lens.
+ Strong desire to create strategies and solutions that challenge and expand the thinking of peers and senior leadership teams.
+ Excellent ability to think strategically about how to use data to drive competitive advantages.
+ Excellent management and leadership skills.
**Academic Qualifications and Certifications:**
+ Bachelor's degree or equivalent in Data Science, Business Analytics, Mathematics, Economics, Engineering, Computer Science or a related field.
+ Relevant programming certification preferred.
+ Agile certification preferred.
**Required experience:**
+ Significant demonstrable experience with one or more programming languages, statistical packages and database languages.
+ Significant demonstrable experience with data warehouse and data lake technical architectures, infrastructure components, ETL/ ELT, and reporting/analytic tools.
+ Significant demonstrable experience applying statistical methods to solve business problems.
+ Significant demonstrated experience with distributed data platform tools such as MS Azure, SAP Snowflake, Spark, MySQL, etc.
+ Significant demonstrated experience in working in micro-services architecture working with APIs development.
+ Significant demonstrable experience of full-stack software development with prolific coding abilities.
+ Significant demonstrated experience with Agile Development Methodologies and Test-Driven Development.
+ Solid line manager experience leading and managing a team of Data Engineering Teams
**Workplace type** **:**
Hybrid Working
**About NTT DATA**
NTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. We invest over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the digital future. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure, and connectivity. We are also one of the leading providers of digital and AI infrastructure in the world. NTT DATA is part of NTT Group and headquartered in Tokyo.
**Equal Opportunity Employer**
NTT DATA is proud to be an Equal Opportunity Employer with a global culture that embraces diversity. We are committed to providing an environment free of unfair discrimination and harassment. We do not discriminate based on age, race, colour, gender, sexual orientation, religion, nationality, disability, pregnancy, marital status, veteran status, or any other protected category. Join our growing global team and accelerate your career with us. Apply today.
Data Engineering Practice Lead
Posted 13 days ago
Job Viewed
Job Description
Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it's a place where you can grow, belong and thrive.
The Director, Data Engineering is a leadership role accountable for operationally managing a Data Engineering Team.
The Data Engineering Practice lead will lead the design, development of the enterprise data platform and pipelines.
This role is further responsible for overseeing SAP data migrations, ensuring seamless integration and transformation of legacy systems into modern cloud-based architectures.
The person will drive innovation, scalability, and governance in data engineering practices while managing a high-performing team.
**Key Responsibilities:**
+ Manages a team of Managers and Data Engineers to ensure achievement of team objectives/KPIs and through continuous coaching and mentoring helps the team to understand and apply data to solve business solutions.
+ Develop the new Data Platform - Execute the data engineering strategy per the selected platform.
+ SAP Data Migration - Lead end-to-end SAP data migration projects, including planning, mapping, transformation, and validation.
+ Engineering Team Leadership - Build and manage a team of data engineers and technical leads. Foster a culture of innovation, collaboration, and continuous improvement.
+ Data Pipeline Development - Oversee development of robust ETL/ELT pipelines using Azure Data Factory, Databricks, Snowflake Streams/Tasks, and SAP tools. Ensure pipelines are optimized for performance, reliability, and cost-efficiency.
+ Data Management - partner with data management practice lead and governance lead to ensure adherence to data quality, lineage, and security standards. Implement monitoring and alerting for data pipeline health and SLA compliance.
+ Stakeholder Engagement - Collaborate with business units and the internal analytics teams and IT to align data engineering efforts with strategic goals. Translate business requirements into scalable technical solutions.
+ Ensure industry standards best practices are applied to development activities and continuously ensures the team remains abreast of latest technology and tools in the area of data engineering.
+ Works across multiple areas on data set to ensure solutions are successfully executed, within agreed upon time frames.
**Knowledge and Attributes:**
+ Excellent ability to thrive in a dynamic, fast-paced environment.
+ Strong quantitative and qualitative analysis skills.
+ Strong desire to acquire more knowledge to keep up to speed with the ever-evolving field of data science.
+ Strong curiosity to sift through data to find answers and more insights.
+ Excellent understanding of the information technology industry within a matrixed organization and the typical business problems such organizations face.
+ Excellent ability to translate technical findings clearly and fluently to non-technical team business stakeholders to enable informed decision-making.
+ Excellent ability to create a storyline around the data to make it easy to interpret and understand.
+ Self-driven and able to work independently yet acts as a team player.
+ Excellent ability to apply data science principles through a business lens.
+ Strong desire to create strategies and solutions that challenge and expand the thinking of peers and senior leadership teams.
+ Excellent ability to think strategically about how to use data to drive competitive advantages.
+ Excellent management and leadership skills.
**Academic Qualifications and Certifications:**
+ Bachelor's degree or equivalent in Data Science, Business Analytics, Mathematics, Economics, Engineering, Computer Science or a related field.
+ Relevant programming certification preferred.
+ Agile certification preferred.
**Required experience:**
+ Significant demonstrable experience with one or more programming languages, statistical packages and database languages.
+ Significant demonstrable experience with data warehouse and data lake technical architectures, infrastructure components, ETL/ ELT, and reporting/analytic tools.
+ Significant demonstrable experience applying statistical methods to solve business problems.
+ Significant demonstrated experience with distributed data platform tools such as MS Azure, SAP Snowflake, Spark, MySQL, etc.
+ Significant demonstrated experience in working in micro-services architecture working with APIs development.
+ Significant demonstrable experience of full-stack software development with prolific coding abilities.
+ Significant demonstrated experience with Agile Development Methodologies and Test-Driven Development.
+ Solid line manager experience leading and managing a team of Data Engineering Teams
**Workplace type** **:**
Hybrid Working
**About NTT DATA**
NTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. We invest over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the digital future. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure, and connectivity. We are also one of the leading providers of digital and AI infrastructure in the world. NTT DATA is part of NTT Group and headquartered in Tokyo.
**Equal Opportunity Employer**
NTT DATA is proud to be an Equal Opportunity Employer with a global culture that embraces diversity. We are committed to providing an environment free of unfair discrimination and harassment. We do not discriminate based on age, race, colour, gender, sexual orientation, religion, nationality, disability, pregnancy, marital status, veteran status, or any other protected category. Join our growing global team and accelerate your career with us. Apply today.
Director of data engineering
Posted today
Job Viewed
Job Description
Identify emerging technologies, methodologies, and practices that can deliver value to our customers and keep the business at the forefront of the industry.
Provide architectural oversight and direction for client and internal projects.
Solution Advisory & Pre-Sales: Engage with clients during the early discovery and solution design phases to understand business goals and shape optimal data architectures and most suitable technologies (currently, Qlik Data Integration (QDI), Qlik Talend, and Microsoft technologies).
Design, develop, implement and showcase demo pipelines in our current technology stack that demonstrate the power and relevance of our platform capabilities.
Collaborate with other pillars (Sales and Customer) to support marketing, sales, pre-sales, including scoping, solution design, and estimation.
People & Enablement: Lead, mentor and grow the data engineering team, developing both technical and consulting capabilities.
Drive internal enablement programs to ensure continuous upskilling across all teams on our core platforms: Qlik Data Integration (QDI), Qlik Talend, and Microsoft technologies.
Champion a culture of learning, experimentation, and collaboration.
Process & Delivery Excellence: Define and implement best practices and frameworks for data integration, pipeline orchestration, and data architecture.
Establish and maintain robust project delivery methodologies that ensure repeatable, scalable, and high-quality implementations.
Work with the other pillar heads to ensure projects are resourced effectively and delivered on time and budget.
Technology & Innovation: Serve as the thought leader for our key platforms - Qlik Data Integration, Qlik Talend, and Microsoft Azure / Data Factory / Fabric.
This may involve interviews, speaking engagements and other marketing activities both internal and external.
Promote innovative uses of technology to solve customer problems in creative, commercially viable ways.
Ensure the alignment of short, medium, and long-term technology choices with customer goals and budget realities.
Key Requirements: 8 years of experience in data engineering or data integration, with at least 3 years in a leadership role.
Proven experience designing and leading complex data architectures and pipelines across a range of industries.
Deep understanding of modern data platforms including: Qlik Data Integration (Qlik Replicate, Compose, Enterprise Manager) Qlik Talend (Data Integration, Data Quality, ESB) Microsoft (Azure Synapse, Data Factory, SQL, Power Platform, Fabric) Experience in pre-sales, solutioning, and commercial scoping of data projects.
Familiarity and experience with data governance, data quality, and metadata management best practices.
Skills & Competencies: Strong business acumen and the ability to translate between business needs and technical solutions.
Excellent communication and stakeholder engagement skills with both technical and non-technical audiences.
Strategic thinker with a hands-on, solutions-oriented approach.
Collaborative leadership style and ability to work cross-functionally.
Entrepreneurial mindset with the ability to innovate within constraints.
Be The First To Know
About the latest Data preprocessing Jobs in South Africa !
Director of data engineering
Posted 1 day ago
Job Viewed
Job Description
Senior Manager: Data Engineering | Gauteng
Posted 21 days ago
Job Viewed
Job Description
Join this listed FSP as a Senior Manager: Data Engineering. You will be responsible for leading this company’s enterprise data strategy and data platform. Lead the data engineering function responsible for unlocking value from the company’s data assets. Set data strategy, research, and implementation of new data tools and processes to drive ROI. Be a data thought leader for the company.
Responsibilities: Data and Analytics Strategy:
- Create and execute a long-term strategy for leveraging data and analytics as a key business enabler and competitive differentiator across the business. Research tools to plan, implement, and build all business and technical aspects of different data and analytics systems and platforms.
- Take responsibility for developing and delivering a key element of the organisation’s data management system.
- Translate data and analytics strategy and vision into data architecture, defining and implementing models, rules, and standards for design and governance.
- Lead a collaborative digital product/service design process by gathering input across all product/service portfolios to define a design system that is viable and scalable, and which strengthens the brand. Lead a cross-portfolio, multidisciplinary team that contributes to the evolution of the system.
- Lead the review of existing operations and the implementation of innovation processes across the division or function to ensure the required continuous improvement outcomes are delivered.
- Communicate the function’s strategy and its relationship to the organisation’s mission, vision, and values; clarify the actions needed to implement it within the area of responsibility; motivate people to commit to these and to doing extraordinary things to achieve the organisation’s business goals.
- Manage and report on the performance of the function/business area; set appropriate performance objectives for direct reports and hold them accountable for achieving these; take appropriate corrective action where necessary to ensure the achievement of annual business objectives.
- Manage a portfolio of major projects in line with overall project management strategy.
- Identify and manage stakeholders up to top management level, finding out their needs/issues/concerns and reacting to these by leading and coordinating the development of stakeholder engagement plans to support the communication of business information and decisions.
- Makes sense of complex, high quantity, and sometimes contradictory information to effectively solve problems. Ensures the organization has reliable ways to gather needed information.
- Anticipates and adopts innovations in business-building digital and technology applications. Stays apprised of technology trends and of how they can make a substantial difference.
- Creates new and better ways for the organization to be successful. Fosters a climate that supports and rewards creativity and innovation; removes obstacles to creativity.
- Holds self and others accountable to meet commitments. Emphasizes accountability, measures organizational performance against goals, and addresses poor team performance.
- Consistently achieves results, even under tough circumstances. Eliminates barriers to organizational performance.
- Develops and delivers multi-mode communications that convey a clear understanding of the unique needs of different audiences.
- Builds strong-identity teams that apply their diverse skills and perspectives to achieve common goals.
- Builds partnerships and works collaboratively with others to meet shared objectives.
- Makes good and timely decisions that keep the organization moving forward.
- Adapts approach and demeanor in real time to match the shifting demands of different situations.
- Rebounds from setbacks and adversity when facing difficult situations.
- Operates as a recognized expert to manage projects and monitor applications using enterprise systems and tools.
- Acts as the organisation’s authority and established expert on understanding and using tools in the domain of data engineering.
- Python, Pyspark, Spark, Databricks, SQL, SAS
- Applies expertise to act as the organisational authority on implementing, configuring, and optimizing IT systems and services.
- Acts as the organisational authority and established expert on acquiring, organizing, protecting, and processing data to fulfill business objectives.
- Uses comprehensive knowledge and skills to work independently while providing guidance and training to others on developing, monitoring, interpreting, and understanding policies and procedures.
- Works at an advanced level to extract patterns from large amounts of data.
- Grade 12/SAQA Accredited Equivalent (Essential)
- 3-year degree in Information Technology/Sciences/Informatics/Applied Mathematics/Statistics/Data Mining (Essential)
- Master’s Degree in Information Technology/Sciences/Informatics/Applied Mathematics/Statistics/Data Mining (Advantageous)
- 5 or more years’ experience of leading teams (Essential)
Senior manager: data engineering | gauteng
Posted 1 day ago
Job Viewed