91 Data Engineer Midsenior jobs in Cape Town

Imagine being a data and programming guru and understanding the intricacies of banking and insurance. At Monocle, we believe work is more than a job. It is a chance to make a difference in the world of finance. An opportunity to fulfil your potential. To make friendships with ambitious, positive, likeminded people. To gain invaluable industry experience and learn critical skills that will make you a highly sought-after management consultant. Monocle strives to provide our clients with a cradle-tograve implementation approach. This means that we not only propose fit-for-purpose solutions, but we also execute the development and implementation of the technical infrastructure required to realise the full benefit of these solutions.

Monocle is a people-centric, independent consulting firm, specialising in banking and insurance. We understand that our greatest asset is our people and because of this, we do things a bit differently.

Monocle embraces an ongoing and active approach to amplifying the talents of our people, empowering them to unlock their full potential and feel fulfilled in their careers. Our vibrant culture thrives through our shared growth mindset, collaboration, support, and social connection, fostering a space where everyone can flourish. We are committed to continuous learning and provide ongoing technical training, industry certification support, and personal development opportunities through mentorship and executive engagement. Ultimately, we strive to create an environment that nurtures ambition, supports well-being, embraces diversity, upholds integrity, and inspires individuals to make a positive and lasting impact on the world around us.

• Unparalleled growth and exposure – Monocle is uniquely positioned in the market to undertake projects across a wide spectrum of critical and exciting areas of the financial services industry. Our managers deliver mission critical projects at the most prestigious banks and insurers in Johannesburg, Cape Town, London and Amsterdam.
• Unlimited training and development – Investment in our people’s development is at the heart of Monocle’s company ethos. That is why we prioritise the upskilling of every employee.
• Unique and vibrant company culture – At Monocle, we believe friends work better together than colleagues. We love nothing more than partaking in a wide variety of activities through our company sponsored clubs.
• Ultra-competitive compensation – At Monocle, we want the best talent to join our team, so we understand that those individuals need to be recognised and rewarded for their true value.

Design and implement scalable data pipelines using Cloud services such as Glue, Redshift, S3, Lambda, EMR, Athena, Microsoft Fabric & Databricks.

Develop and maintain ETLprocesses to transform and integrate data from various sources.

Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.

Optimize and tune performance of data pipelines and queries.

Ensure data quality and integrity through robust testing and validation processes.

Implement data security and compliance best practices.

Monitor and troubleshoot data pipeline issues and ensure timely resolution.

Stay updated with the latest developments in Data Engineering technologies and best practices.

Holds a Bachelor’s degree from an accredited university.

Industry experience: A minimum of two years of hands-on experience is required. Prior experience in the financial services industry would be beneficial but not mandatory.

Strong foundation in data engineering: We value hands-on experience and proven skills in building and managing data solutions using on-premises technologies or Cloud. While a Bachelor's degree in Computer Science, Engineering, or a related field is a plus, your ability to demonstrate expertise matters most.

Experience with core Cloud Data Services: Familiarity with Glue, Redshift, S3, Lambda, EMR, Athena, Microsoft Fabric or Databricks.

Experience with Big Data technologies: Knowledge of big data technologies such as Apache Spark, Hadoop, or Kafka.

Scripting & Programming proficiency: Programming skills in Python, Pandas & SQL

Database Management: Experience working with relational databases like AWS RDS, MS SQL, Azure SQL DB or Postgres.

Solid Data Engineering background: Knowledge and experience of data modelling, ETL processes, and data warehousing.

Infrastructure as code (IaC) proficiency: Experience with tools like AWS CloudFormation, Terraform or Azure ARM/Bicep for automating infrastructure provisioning and deployments is crucial.

DevOps fluency: We seek a candidate with experience in CI/CD tools to streamline software development and delivery.

Communication and collaboration: Excellent communication, problem-solving, and analytical skills are key. The ability to present complex technical concepts in a clear and concise way.

Cloud Certification (a plus): While not mandatory, possessing a relevant Cloud certification demonstrates your commitment to professional development and validates your understanding of Cloud services and best practices.

Strong problem-solving and analytical skills.

Excellent communication and collaboration skills.

Attention to detail and a commitment to data accuracy.

Be eager to learn new approaches and technologies.

Works well under pressure to meet client objectives.

Is sociable and enjoys interacting with others, both at work and at social events.

Works well independently and in a team.

During the interview process, you will meet with both People and Culture representatives and Monocle Executives. A critical aspect of the interview process will be determining whether Monocle will be a good fit for you and your career aspirations, so please come prepared with any questions you may have.

If you have a passion for working with people, a relevant qualification, and think you have what it takes to join a team of your smartest and most driven peers, apply to Monocle today!

A 45-Minute, 60-questions, online competency assessment test.

A personal meeting with People & Culture to get to know you and tell you more about Monocle.

To discuss your skills, business knowledge, experience, project fit and the role requirements.

A discussion with either our CEO, COO or CFO on various topics.

A final offer is made by the P&C team to the candidate.

*Please note that on clicking 'Apply Now' you will be redirected to our third-party career portal.

? Join Our Dynamic Data Engineering Team! ?

Are you ready to take your data engineering career to the next level? We are on the lookout for talented individuals to join our innovative team and work on cutting-edge projects that make a real impact.

? Virtual Recruitment Drive: 24th & 25th July Complete assessments, interviews, and find out the outcome of your application on the same day!

Why Join Us?

• Exciting Projects: Work on complex data engineering projects within a niche domain involving patents and intellectual property.

• High-Profile Clients: Collaborate with industry giants like Google, Apple, and Tesla.

• Innovative Environment: Contribute to the development of a unified data platform that serves as the single source of truth for our business.

• Career Growth: Benefit from a mix of experience levels within the team, fostering growth and mentorship.

• Collaborative Culture: Be part of a team that values inclusivity and diverse backgrounds.

• Work in a Way That Works for You: We promote a healthy work/life balance across the organisation . With numerous wellbeing initiatives, shared parental leave, study assistance , and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals. Enjoy working flexible hours, flexing the times when you work in the day to help you fit everything in and work when you are the most productive.

Open Positions:

• Principal Engineer: Lead technical projects and mentor junior team members. Significant experience with PySpark is essential.

• Senior Data Engineer: Tackle complex data engineering tasks with strong skills in SQL, Python, and cloud technologies (preferably AWS).

• Intermediate Data Engineer: Support senior engineers with at least 3 years of experience in SQL, Python, and cloud technologies.

• Junior Data Engineer: Assist in various data engineering tasks with foundational knowledge in SQL and Python.

What We Offer: We know that your well-being and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer:

• Medical Aid

• Retirement Plan inclusive of Risk Benefits (Disability, Critical Illness, Life Cover & Funeral Cover)

• Modern Family Benefits, including adoption and surrogacy

• Study Leave

Ready to Make an Impact? Submit your application today. Should you meet the requirements one of our recruiters will reach out to explain the process and learn more about your experience.

Apply Now and Be Part of Our Journey!

We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form or please contact 1- .

Criminals may pose as recruiters asking for money or personal information. We never request money or banking details from job applicants. Learn more about spotting and avoiding scams here .

Please read our Candidate Privacy Policy .

We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law.

USA Job Seekers:

EEO Know Your Rights .

? Join Our Dynamic Data Engineering Team! ?

Are you ready to take your data engineering career to the next level? We are on the lookout for talented individuals to join our innovative team and work on cutting-edge projects that make a real impact.

? Virtual Recruitment Drive: 24th & 25th July Complete assessments, interviews, and find out the outcome of your application on the same day!

Why Join Us?

• Exciting Projects: Work on complex data engineering projects within a niche domain involving patents and intellectual property.

• High-Profile Clients: Collaborate with industry giants like Google, Apple, and Tesla.

• Innovative Environment: Contribute to the development of a unified data platform that serves as the single source of truth for our business.

• Career Growth: Benefit from a mix of experience levels within the team, fostering growth and mentorship.

• Collaborative Culture: Be part of a team that values inclusivity and diverse backgrounds.

• Work in a Way That Works for You: We promote a healthy work/life balance across the organisation . With numerous wellbeing initiatives, shared parental leave, study assistance , and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals. Enjoy working flexible hours, flexing the times when you work in the day to help you fit everything in and work when you are the most productive.

Open Positions:

• Principal Engineer: Lead technical projects and mentor junior team members. Significant experience with PySpark is essential.

• Senior Data Engineer: Tackle complex data engineering tasks with strong skills in SQL, Python, and cloud technologies (preferably AWS).

• Intermediate Data Engineer: Support senior engineers with at least 3 years of experience in SQL, Python, and cloud technologies.

• Junior Data Engineer: Assist in various data engineering tasks with foundational knowledge in SQL and Python.

What We Offer: We know that your well-being and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer:

• Medical Aid

• Retirement Plan inclusive of Risk Benefits (Disability, Critical Illness, Life Cover & Funeral Cover)

• Modern Family Benefits, including adoption and surrogacy

• Study Leave

Ready to Make an Impact? Submit your application today. Should you meet the requirements one of our recruiters will reach out to explain the process and learn more about your experience.

Apply Now and Be Part of Our Journey!

We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form or please contact 1- .

Criminals may pose as recruiters asking for money or personal information. We never request money or banking details from job applicants. Learn more about spotting and avoiding scams here .

Please read our Candidate Privacy Policy .

We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law.

USA Job Seekers:

EEO Know Your Rights .

Stay safe online. Now more than ever it’s important to be cybersafe. See Santam’s tips to help you stay safe online. Learn more.

You made it. Here we ensure we receive all your documents to get you onto our payroll system.

Our recruitment process

Step 1

Our vacancies are advertised between 3-5 days depending on the market availability of the skills required.

Our recruitment process

Step 2

Could take a couple of days depending on the volume and the screening process, which various according to the complexity of the role.

Our recruitment process

Step 3

Long listed candidates are sent to the hiring manager to review and consider for the next stage of the selection process.

Our recruitment process

Step 4

Candidates may be contacted for a short screening interview to get to know them better and to ask any clarity seeking questions.

Our recruitment process

Step 5

We use a competency interview model. Interview dates will be confirmed via e-mail. During the interview you will be introduced to the panel and the process will be explained. Remain calm and sell yourself. This is our opportunity to match your skills to the job. You only get one shot!

Our recruitment process

Step 6

Should we like what you bring to the table, depending on the job you are being considered for, you may be invited to complete a psychometric assessment. Further employment checks are conducted the ensure the validity of your credentials.

Our recruitment process

Step 7

Prior to an offer being made, there may be a second interview to meet with the Head of Department.Should there be a need for any further interviews, be sure to bring the same charm that got you through your first interview.

Our recruitment process

Step 8

You made it. Here we ensure we receive all your documents to get you onto our payroll system.

Our recruitment process

Step 1

Our vacancies are advertised between 3-5 days depending on the market availability of the skills required.

Our recruitment process

Step 2

Could take a couple of days depending on the volume and the screening process, which various according to the complexity of the role.

Our recruitment process

Step 3

Long listed candidates are sent to the hiring manager to review and consider for the next stage of the selection process.

Our recruitment process

Step 4

Candidates may be contacted for a short screening interview to get to know them better and to ask any clarity seeking questions.

Our recruitment process

Step 5

We use a competency interview model. Interview dates will be confirmed via e-mail. During the interview you will be introduced to the panel and the process will be explained. Remain calm and sell yourself. This is our opportunity to match your skills to the job. You only get one shot!

Our recruitment process

Step 6

Should we like what you bring to the table, depending on the job you are being considered for, you may be invited to complete a psychometric assessment. Further employment checks are conducted the ensure the validity of your credentials.

Our recruitment process

Step 7

Prior to an offer being made, there may be a second interview to meet with the Head of Department.Should there be a need for any further interviews, be sure to bring the same charm that got you through your first interview.

Our recruitment process

Step 8

You made it. Here we ensure we receive all your documents to get you onto our payroll system.

There is not just one way to be successful at Santam, nor is there one way to interview with us. Interviewing is a two-way process - it helps us learn about you, and it helps you learn about us, our people, and what you could do here.

Santam has been officially recognised by the Top Employers Institute as a Top Employer 2024 for excellence in people practices. This is the eighth consecutive year that we have achieved this accolade.

• Design and develop scalable data pipelines and ETL (Extract, Transform, Load) processes to collect, process, and integrate data from various sources into a centralized data warehouse or data lake.
• Implementing data models, schemas, and database structures to organize and optimize data storage for efficient data retrieval, writing and analysis.
• Collaborating with quantitative and analysts to understand their data requirements and provide them with the necessary infrastructure and tools for data exploration, analysis, and visualization.
• Ensuring data quality and consistency by implementing data validation, cleaning, and transformation procedures.
• Monitoring and maintaining the performance, availability, and security of data systems, including database servers, data warehouses, and data processing frameworks.
• Troubleshooting and resolving data-related issues and performance bottlenecks in collaboration with the investment teams.
• Evaluating and integrating new data technologies, tools, and frameworks to improve data infrastructure and processing capabilities.
• Documenting data pipelines, workflows, and technical specifications to facilitate knowledge sharing and maintain system.
• Keeping up-to-date with industry trends, best practices, and emerging technologies in the field of data engineering.

General:

• Assist and collaborate on team wide projects.
• Contribute to the analytics platform to know what is required from a Data Engineering perspective.
• Assist and investigate ad-hoc data related queries.

REQUIRED EXPERIENCE

• Up to 3 years’ experience with a specific focus in database management, data analysis and have produced ETL processes at scale are essential.
• Basic understanding of financial instruments and markets.
• Exceptional skill and experience in Python and SQL.

REQUIRED QUALIFICATIONS

• University Post Graduate Degree (Computer Science/Information Systems/Maths/Stats/Business Science/Actuarial).
• Strong programming skills in Python and SQL.

KEY COMPETENCIES

• Excellent interpersonal skills, excellent communication skills oral and written.
• Aptitude in decision-making and problem-solving.
• Strong work ethic, self-motivated, driven and results oriented.
• Ability to listen and respond to new ideas, input and diverse perspectives.
• High levels of self-confidence, assertive as well as healthy levels of EQ.
• Energetic problem-solver.
• Structured work style, attention to detail and accuracy is of the utmost importance.

We are looking for a skilled Data Engineer to join our clients’ investment team to oversee their database architecture, data flow, and data processes. The role involves efficiently managing the reception, transfer, processing, analysis, and integration of third-party systems for investment research and business operational processes. The ideal candidate will possess expertise in version control and database solutions, while also adhering to the companies’ collaborative team approach in line with their value framework.

• Design and develop scalable data pipelines and ETL processes to collect, process, and integrate data from various sources into a centralized data warehouse or data lake.
• Implement data models, schemas, and database structures to organize and optimize data storage for efficient data retrieval, writing, and analysis.
• Collaborate with quantitative analysts to understand their data requirements and provide necessary infrastructure and tools for data exploration, analysis, and visualization.
• Ensure data quality and consistency by implementing data validation, cleaning, and transformation procedures.
• Monitor and maintain the performance, availability, and security of data systems, including database servers, data warehouses, and data processing frameworks.
• Troubleshoot and resolve data-related issues and performance bottlenecks in collaboration with the investment teams.
• Evaluate and integrate new data technologies, tools, and frameworks to improve data infrastructure and processing capabilities.
• Document data pipelines, workflows, and technical specifications to facilitate knowledge sharing and maintain systems.
• Stay up to date with industry trends, best practices, and emerging technologies in the field of data engineering.
• Assist and collaborate on team-wide projects.
• Contribute to the analytics platform from a Data Engineering perspective.
• Assist and investigate ad-hoc data-related queries.

• Up to 3 years' experience with a specific focus on database management, data analysis, and producing ETL processes at scale is essential.
• Basic understanding of financial instruments and markets.
• Exceptional skill and experience in Python and SQL.
• Linux server administration.
• Git version control.
• Solid experience in managing and analysing large datasets.

• Matric / Grade 12 – Essential
• University Post Graduate Degree (Computer Science/Information Systems/Mathematics/Stats/Business Science/Actuarial).

This is an exciting opportunity offering fantastic growth and exposure as you become part of a leading systematic investment framework team.

For more information contact:
Chanel du Plessis
Senior IT Recruitment Consultant
+27 87 378 8857
Alternatively, email your CV to:

Job category: FMCG, Retail, Wholesale and Supply Chain

Location: Cape Town

Contract: Permanent

Remuneration: Market Related

EE position: No

Data Engineer who will assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of data at scale and consolidating this data into a bigger data ecosystem.

• Assist in designing and implementing scalable and robust processes for ingesting and transforming large data sets.
• Assist in the design and implementation of data pipelines from a variety of data sources and support the maintenance thereof.
• Ingest large, complex data sets that meet functional and non-functional requirements.
• Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
• Build bulk and delta data patterns for optimal extraction, transformation, and loading of data.
• Supports the organisation's cloud strategy and alignment to data architecture and data governance.
• Engineer data in the appropriate formats for downstream consumption for analytics or Enterprise applications.
• Assist in the development of APIs to expose the data to Enterprise Applications and 3rd party vendors.
• Assist in identifying, designing and implementing robust process improvement activities to drive efficiency and automation for greater scalability.
• Work with various stakeholders across the organisation to understand data requirements and apply technical knowledge of data management to solve key business problems.
• Provide support in the operational environment with all relevant support teams for data services.
• Create and maintain functional requirements and system specifications in support of data architecture and detailed design specifications for current and future designs.
• Support test and deployment of new services and features.

• Bachelor’s degree in Computer Science, Business Informatics, Mathematics, Statistics or Engineering with 4 - 5 years relevant data engineering experience.
• A strong understanding of data structures, algorithms, and effective software design.
• Significant experience working with structured and unstructured data at scale and different data stores such as key-value, document, columnar, etc. as well as traditional RDBMS and data warehouses.
• Good programming, performance tuning and troubleshooting skills using programming languages such as Python, Scala, Java and C.
• Practical experience with Apache Spark and AWS services such as Redshift, Glue, Lambda, EMR, S3, IAM, RDS, etc.
• Experience wrangling terabytes of big, complicated, imperfect data.
• Experience with designing and implementing Cloud (AWS) solutions including use of APIs available.
• Experience with DevOps architecture, implementation and operation would be advantageous.
• Experience with version control systems such as Git, SVN.
• Excellent verbal and written communication skills; must work well in an agile, collaborative team environment.
• Knowledge of Engineering and Operational Excellence using standard methodologies.
• Some experience in applying SAFe/Scrum/Kanban methodologies would be advantageous.
• Knowledge and understanding of business process management lifecycle which covers the design, modelling, execution, monitoring, and optimization as well as business process re-engineering.
• Good problem-solving skills: The ability to exercise judgment in solving technical, operational, and organizational challenges.

We are looking for a Data Engineer to join our team in Cape Town, on an initial 12-month contract, onsite.

The Data Engineer will work within the Business Intelligence team of the information technology division and will be responsible to work closely with our Financial services division for the design, development and implementation of scalable and robust processes, in accordance with specified guidelines.

This role includes extracting, transforming, and consolidating data, developing and maintaining a data platform that will ensure optimal performance and stability.

This position requires strong problem-solving skills, technical expertise in BI tools and SQL, and excellent communication abilities to effectively support users and maintain high service standards.

KEY RESPONSIBILTIES:

• Design, develop, optimize, and maintain data architecture and data pipelines that adhere to ETL principles and business goals.
• Develop complex queries and data solutions using SQL and SSIS according to defined architectural standards.
• Assist in the design and implementation of data pipelines from a variety of data sources and support the maintenance thereof.
• Ingest large, complex data sets that meet functional and non-functional requirements.
• Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
• Build bulk and delta data patterns for optimal extraction, transformation, and loading of data.
• Engineer data in the appropriate formats for downstream consumption for analytics or Enterprise applications.
• Work with various stakeholders to understand data requirements and apply technical knowledge of data management to solve key business problems.
• Provide support in the operational environment with all relevant support teams for data services.
• Support test and deployment of new services and features.

Education Background:

• Bachelors degree in computer science, Business Informatics or equivalent.

Profession Experience:

• Minimum of 3 years of relevant data engineering experience, preferably within the retail sector.
• Experience in credit and credit risk will be advantageous.

Skills :

• Expertise with SQL, SSIS and data analysis and experience with at least one programming language (Python).
• Knowledge of working with DBT would be advantageous.
• Experience developing and maintaining data warehouses in big data solutions.
• Worked with BI tools such as Power BI, Tableau, Pyramid or others.
• Conceptual knowledge of data and analytics, such as dimensional modelling, ETL, ELT, reporting tools, data governance, data warehousing, and structured and unstructured data.
• Knowledge and experience regarding cloud technologies, Python and Databrick will be advantageous.

We are looking for a versatile Data Engineer with a strong blend of data engineering and software engineering skills. The ideal candidate is someone who thrives on improving existing systems, enjoys business-as-usual (BAU) operational tasks, and is comfortable engaging with business stakeholders to understand and translate requirements into technical solutions.

This is not a greenfield development role — instead, it focuses on supporting, maintaining, and optimising our current data and backend solutions to ensure performance, stability, and scalability.

Key Responsibilities:

• Monitor and support existing data pipelines and backend processes to ensure smooth daily operations.
• Investigate and resolve issues related to databases (e.g., performance tuning, index management, RDS memory optimisation).
• Analyse and fix API-related errors, latency issues, and general data integration problems.
• Optimise SQL queries, functions, and stored procedures as needed.
• Conduct load testing and scenario testing to ensure robustness and reliability of solutions.
• Collaborate with business users to gather and understand requirements, then translate them into scalable technical solutions.
• Make incremental improvements to existing systems — including data models, ETL processes, and reporting solutions.
• Contribute to backend work and system maintenance where applicable.
• Maintain clean documentation and follow good development practices, ideally with some GitOps experience (not mandatory).

Key Skills and Experience:

• 4+ years of experience in data engineering, software engineering, or related roles.
• Strong understanding of relational databases (e.g., PostgreSQL, MySQL, AWS RDS).
• Proficiency in SQL and Python for data processing and scripting.
• Solid troubleshooting skills with experience in diagnosing system performance issues and debugging backend systems.
• Comfortable working in a BAU environment, making enhancements to existing systems rather than building from scratch.
• Ability to work closely with business stakeholders to spec and implement changes.
• Familiarity with tools like Git, CI/CD pipelines, and basic DevOps practices (GitOps experience is a plus).
• Understanding of API structures and integration concepts.
• Detail-oriented mindset with a passion for system stability and continuous improvement.

Nice to Have:

• Experience with cloud platforms (e.g., AWS, Azure, GCP).
• Exposure to monitoring tools (e.g., Datadog, CloudWatch).
• Familiarity with data visualisation or reporting tools.
• Previous experience working in cross-functional teams and agile environments.

What We’re Looking For:

• Someone who takes pride in keeping systems running smoothly and efficiently.
• A practical problem-solver who enjoys getting into the details and fixing root causes.
• A communicator who can bridge the gap between technical teams and business users.
• A collaborative team player who can work independently and proactively.

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

* If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.