274 Big Data jobs in South Africa

We are looking for an experienced Big Data Developer to join an international banking technology team in Málaga, Spain. In this role, you will contribute to the development of business applications within the Regulatory & Compliance domain, covering the full software lifecycle from problem analysis to deployment.

You'll work with modern big data technologies, collaborate with users to understand business needs, and provide innovative solutions that meet regulatory and compliance requirements. This is a fantastic opportunity to advance your career in a global environment while enjoying the lifestyle benefits of living in Spain.

• Participate in the end-to-end software lifecycle, including analysis, design, development, testing, and deployment.
• Collaborate with business users to identify requirements and deliver strategic technology solutions.
• Optimise and analyse code, applying best practices such as threat modelling and SAST.
• Manage tools and processes for documentation and Application Lifecycle Management (ALM).
• Plan and deliver projects using Agile methodology.
• Support incident resolution, including planned interventions.
• Execute unit, integration, and regression testing.
• Manage release processes and deployment tools.

Required:

• 3+ years of experience as a Big Data Developer.
• Bachelor's degree in Computer Science, Telecommunications, Mathematics, or a related field.
• Proficiency with GitHub.
• Strong knowledge of databases (Oracle PL/SQL, PostgreSQL).
• Experience with Java and JavaScript.
• Hands-on ETL experience.
• Fluency in English (Spanish is advantageous).

Preferred:

• Familiarity with microservices frameworks (Spring Boot), OpenShift.
• Knowledge of Flink, Drools, Kafka, DevOps tools.
• Agile methodology experience with tools such as Jira and Confluence.
• Exposure to S3, Elastic, and Angular.
• Experience in Transactional Regulatory Reporting.
• Innovative mindset and ability to generate strategic ideas.

Other Requirements:

• Availability to travel.
• Willingness to relocate to Málaga, Spain.

Contract

4 to 25 years

Negotiable

03 September 2025

We are seeking a skilled Data Engineer to design and develop scalable data pipelines that ingest raw, unstructured JSON data from source systems and transform it into clean, structured datasets within our Hadoop-based data platform. The ideal candidate will play a critical role in enabling data availability, quality, and usability by engineering the movement of data from the Raw Layer to the Published and Functional Layers.

Key Responsibilities:

• Design, build, and maintain robust data pipelines to ingest raw JSON data from source systems into the Hadoop Distributed File System (HDFS).
• Transform and enrich unstructured data into structured formats (e.g., Parquet, ORC) for the Published Layer using tools like PySpark, Hive, or Spark SQL.
• Develop workflows to further process and organize data into Functional Layers optimized for business reporting and analytics.
• Implement data validation, cleansing, schema enforcement, and deduplication as part of the transformation process.
• Collaborate with Data Analysts, BI Developers, and Business Users to understand data requirements and ensure datasets are production-ready.
• Optimize ETL/ELT processes for performance and reliability in a large-scale distributed environment.
• Maintain metadata, lineage, and documentation for transparency and governance.
• Monitor pipeline performance and implement error handling and alerting mechanisms.

Technical Skills & Experience:

• 3+ years of experience in data engineering or ETL development within a big data environment.
• Strong experience with Hadoop ecosystem tools: HDFS, Hive, Spark, YARN, and Sqoop.
• Proficiency in PySpark, Spark SQL, and HQL (Hive Query Language).
• Experience working with unstructured JSON data and transforming it into structured formats.
• Solid understanding of data lake architectures: Raw, Published, and Functional layers.
• Familiarity with workflow orchestration tools like Airflow, Oozie, or NiFi.
• Experience with schema design, data modeling, and partitioning strategies.
• Comfortable with version control tools (e.g., Git) and CI/CD processes.

Nice to Have:

• Experience with data cataloging and governance tools (e.g., Apache Atlas, Alation).
• Exposure to cloud-based Hadoop platforms like AWS EMR, Azure HDInsight, or GCP Dataproc.
• Experience with containerization (e.g., Docker) and/or Kubernetes for pipeline deployment.
• Familiarity with data quality frameworks (e.g., Deequ, Great Expectations).

Qualifications:

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.

• Relevant certifications (e.g., Cloudera, Databricks, AWS Big Data) are a plus.

• In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

• If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.

Big DataApache HadoopApache HivePySparkSQLJSONData Engineering

BankingFinancial Services

About the Role

We are seeking a highly skilled and motivated Data Management & Business Intelligence (BI) Specialist to join our team on a full-time basis.

The successful candidate will be responsible for designing, implementing, and maintaining enterprise-grade data management frameworks and BI solutions, following globally recognized standards such as the DAMA Data Management Body of Knowledge (DMBOK).

This role requires both strategic insight and technical execution to ensure data quality, governance, and accessibility across the organization.

Key Responsibilities

• Develop, maintain, and optimize data models, ensuring alignment with business processes and analytics needs.
• Oversee data governance frameworks, ensuring compliance with policies, standards, and regulatory requirements.
• Design and manage data warehouses for reporting, analytics, and operational intelligence.
• Implement and monitor data management practices covering data lifecycle, quality, architecture, and metadata.
• Collaborate with cross-functional teams (IT, analytics, business units) to drive data-driven decision-making.
• Maintain documentation, data dictionaries, and data lineage mapping.
• Evaluate and implement BI tools and technologies for enhanced business reporting and visualization.

Qualifications & Certifications (Mandatory)

Candidates must hold recognized certifications aligned with the Data Management and BI professional body of knowledge, including:

Data Modeling Certificate

Data Management Professional Certificate

Data Warehouse Certificate

Data Governance Certificate

Additional advantageous certifications:

• DAMA Certified Data Management Professional (CDMP)
• Microsoft Power BI, Tableau, or similar BI platform certifications
• ISO 8000 / ISO 38505 awareness

Experience & Skills

• Minimum 3–5 years of experience in data management, BI, or related domains.
• Proven experience in data modeling (conceptual, logical, physical).
• Strong knowledge of ETL processes, data warehousing, and reporting systems.
• Hands-on experience with SQL, Python, or data visualization tools.
• Understanding of data governance frameworks and metadata management.
• Excellent analytical, problem-solving, and communication skills.

Why Join Us

• Long-term career opportunity with a 5-year full-time position.
• Work with advanced enterprise data management tools and frameworks.
• Join a forward-thinking organization committed to data excellence and innovation.
• Opportunities for continuous professional development and certification advancement.

Application Process

Interested candidates should submit:

• A detailed CV
• Copies of relevant certifications

Job Types: Full-time, Permanent

Pay: From R40 000,00 per month

Work Location: In person

Our client is seeking an experienced Data Management Capability Lead to be based at their Centurion office. The role is responsible for leading the organisation's data capability centre, with a focus on SAP data migration, governance and ECC to S/4HANA transitions. The successful candidate will oversee end-to-end data programmes, build internal capability and ensure delivery across complex SAP landscapes.

Key Focus Areas:

• Strategic Leadership – Design and implement SAP data migration frameworks, manage ECC to S/4HANA transitions and set governance standards.
• Migration & Innovation – Deliver structured SAP data migrations using advanced tools (Data Services, LTMC, LSMW, ABAP, SQL) and implement SAP MDG.
• Capability Building – Develop and mentor consultants, data stewards and graduates into client-facing roles.
• Client Engagement – Facilitate readiness workshops, ensure compliance with POPIA and ISO27001 and advise stakeholders at the executive level.
• Delivery Management – Govern data project milestones, from mock loads to cutovers and reconciliations, ensuring business continuity and quality.

Minimum Requirements:

• Bachelor's degree in Information Systems, Computer Science, Data Science, or related field.
• SAP Activate Project Manager Certification.
• SAP MDG Certification and DAMA-DMBOK2 knowledge/certification.
• PMP or Agile/Scrum certification.
• 15+ years in SAP programmes, with at least 5 years leading large-scale data migration and governance portfolios.
• Proven experience across ECC, S/4HANA, IS-U, Real Estate, Retail, and Public Sector.

This is a senior leadership opportunity for a highly experienced professional with a strong track record in SAP data transformation. If you meet the requirements and are interested in this role, please apply.

Please note: Only shortlisted candidates will be contacted.

Company Description

Work with Us. Change the World.

At AECOM, we're delivering a better world. Whether improving your commute, keeping the lights on, providing access to clean water, or transforming skylines, our work helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the world's most complex challenges and build legacies for future generations.

There has never been a better time to be at AECOM. With accelerating infrastructure investment worldwide, our services are in great demand. We invite you to bring your bold ideas and big dreams and become part of a global team of over 50,000 planners, designers, engineers, scientists, digital innovators, program and construction managers and other professionals delivering projects that create a positive and tangible impact around the world.

We're one global team driven by our common purpose to deliver a better world. Join us.

Job Description

AECOM
is seeking a mid-level Environmental Data Manager to support our enterprise environmental data management system (EarthSoft EQuIS). The ideal candidate will have a strong understanding of environmental data and terminology, good communication skills, and the ability to collaborate with both technical and non-technical stakeholders. As a team Lead, you will be responsible for guiding a high-performing team toward achieving operational and strategic goals. You'll serve as a mentor, problem-solver, and project driver—ensuring that team efforts align with broader organizational objectives. This role demands a balance of technical expertise, emotional intelligence, and leadership acumen.

This position offers the flexibility of a hybrid work schedule including both in-office and remote work options and will be located in Pretoria, South Africa (Centurion office).

Management Responsibilities:

• Collaborate with cross-functional teams to ensure alignment with business goals.
• Lead, coach, and motivate a team of professionals to meet performance targets by fostering a culture of accountability, collaboration, and continuous improvement
• Conduct regular one-on-ones, performance reviews, and career development planning
• Set strategic direction and provide clear guidance to align team efforts with organizational goals.
• Cultivate a high-performance culture that values collaboration, inclusivity, and continuous improvement.
• Lead regular check-ins and performance reviews to support growth and accountability.
• Oversee financial planning, including budgeting, forecasting, and resource allocation.
• Drive talent development by identifying skill gaps and enabling upskills and career progression.
• Manage end-to-end project execution, ensuring timely delivery and quality outcomes.
• Contribute to business strategy, offering insights for growth and innovation.
• Mentor and coach team members for career growth
• Build and maintain strong relationships with internal and external stakeholders
• Support resource planning and allocation to ensure optimal team utilization.

Technical Responsibilities:

• The ideal candidate will be able to understand requests from environmental subject matter experts.
• Gather, organize, and validate field data from environmental monitoring activities such as air, soil, and water quality assessments.
• Manage field and analytical data in relational database systems like EarthSoft EQuIS, Locus EIM, or other environmental databases.
• Conducting QA/QC for electronic data deliverables with laboratory analytical data to ensure quality.
• Generating standardized report output through software tools like EQuIS or EIM.
• Prepare environmental data summaries and compliance reports for submission to regulatory agencies.
• Utilize statistical tools and predictive modeling techniques to assess environmental impacts and forecast trends.
• Design and create dashboards and visualizations using platforms such as Power BI or EQuIS Enterprise.
• Ensure data meets reporting standards and guidelines set by local, state, and federal authorities.
• Responsible for generating Borelogs through multiple platforms like gINT, open source, or EQuIS.
• Supporting legacy data migrations to EQuIS/EIM as needed, ensuring proper formatting and compatibility.
• Collaborate with stakeholders, project managers, and team members to achieve project objectives efficiently.
• Provide technical support to field sampling teams and act as a liaison between the project staff, analytical laboratory, data validator, and GIS analysts.
• Research state and federal regulations necessary to manage action levels or clean-up criteria.

Qualifications

Minimum Requirements:

• Bachelor's/master's degree in environmental engineering, Environmental Science, or a related discipline plus tem (10) years of relevant experience in environmental data management or consulting, with hands-on experience in EQuIS, LOCUS EIM, ESDAT, or similar databases or demonstrated equivalency of experience and/or education.
• Expertise and experience with environmental data and database systems (MS SQL Server, MS Access). Expertise with relational databases such as EarthSoft's Environmental Quality Information System (EQuIS) /EIM/ ESdat. Ability to continually analyze data at all stages for problems, logic, and consistency concerning field data collection, analytical reporting, and other expertise on EQUIS sub-tools (Collect, Edge, ArcGIS, highly desirable but not essential).
• Assist with projects globally and task delivery with high quality and within deadlines. Managing data (geological, Field data, chemical laboratory data) for technical report writing and interpretation as required by the team. Maintaining and updating various project dashboards using the web-based EQuIS Enterprise system; and preparing report-ready data tables, charts, and figures for internal review and external client reports.
• Project management duties include coordinating field events. Using the sample registry and Sample Planning module, create EDDs and load to EQuIS to track event closure.
• Proficient in EQuIS Collect Module, create and deploy different categories of forms and train field personnel on the same.
• Expert level understanding of Office 365, Excel, Power Query & Power Automate.
• Should be good in Documentation, including Data Management SOPs, Process Maps, Work instructions, Activity Analysis, and Task effort estimation
• Use of visualization tools like Power BI to help management make effective decisions for the environmental domain is desirable but not essential
• Knowledge of digital tools like Power BI, R Studio, KNIME Analytics, SQL, and Python is a plus but not mandatory.
• Strong organizational skills, with experience prioritizing, time management, and managing multiple tasks.
• Effective communication skills to interact with multidisciplinary teams and clients.
• Expertise in applying Six-Sigma techniques/analysis to the environmental data management process.
• Strong attention to detail with excellent analytical, judgment, and problem-solving capabilities.
• Comfortable running meetings and presentations

Additional Information

About AECOM

AECOM is proud to offer comprehensive benefits to meet the diverse needs of our employees. Depending on your employment status, AECOM benefits may include medical, dental, vision, life, AD&D, disability benefits, paid time off, leaves of absences, voluntary benefits, perks, flexible work options, well-being resources, employee assistance program, business travel insurance, service recognition awards, retirement savings plan, and employee stock purchase plan.

AECOM is the global infrastructure leader, committed to delivering a better world. As a trusted professional services firm powered by deep technical abilities, we solve our clients' complex challenges in water, environment, energy, transportation and buildings. Our teams partner with public- and private-sector clients to create innovative, sustainable and resilient solutions throughout the project lifecycle – from advisory, planning, design and engineering to program and construction management. AECOM is a Fortune 500 firm that had revenue of $16.1 billion in fiscal year 2024. Learn more at

What makes AECOM a great place to work

You will be part of a global team that champions your growth and career ambitions. Work on groundbreaking projects - both in your local community and on a global scale - that are transforming our industry and shaping the future. With cutting-edge technology and a network of experts, you'll have the resources to make a real impact. Our award-winning training and development programs are designed to expand your technical expertise and leadership skills, helping you build the career you've always envisioned. Here, you'll find a welcoming workplace built on respect, collaboration and community - where you have the freedom to grow in a world of opportunity.

As an Equal Opportunity Employer, we believe in your potential and are here to help you achieve it. All your information will be kept confidential according to EEO guidelines.

ReqID:
J

Business Line:
Geography OH

Business Group:
DCS

Strategic Business Unit:
Enterprise Capabilities

Career Area:
Engineering

Work Location Model:
Hybrid

Purpose of the role:

To support the business unit in improving data quality by documenting and applying data quality rules, identifying recurring data issues, and collaborating with stakeholders to ensure data is fit for purpose. This role acts as the quality assurance function for data flowing through business processes.

Key Responsibilities
:

• ·Collaborate with business users to understand how data is used and what quality means in context.
• Identify key data fields and define quality rules (e.g., format, completeness, allowed values).
• Document rules using standardized templates and maintain version control.
• Monitor data for errors and inconsistencies using profiling tools or reports.
• Conduct root cause analysis using lean tools (e.g., 5 Whys, Fishbone Diagram).
• · Escalate recurring issues to IT or system owners with clear documentation and impact statements.
• · Participate in continuous improvement efforts to refine data rules and processes.
• · Share findings and collaborate with other Data Stewards across departments.

Preferred Qualifications:

• Bachelor's degree in Information Systems, Data Management, Business Analytics, or related field.
• Exposure in Data Governance or Data quality concepts (advantageous)
• Experience working with business data (e.g., customer, product, pricing) is a plus)

JOB DUCK IS HIRING A DATA MANAGEMENT SPECIALIST

For ambitious, culturally diverse, curious minds seeking booming careers, Job Duck unlocks and nurtures your potential. We connect you with rewarding, remote job opportunities with US-based employers who recognize and appreciate your skills, allowing you to not just survive but thrive. As a lifestyle company we ensure that everybody working here has a fantastic time, which is why we've earned the Great Place to Work Certification for 4 years

Role Overview

If you enjoy working with data, love the clarity of spreadsheets, and take pride in getting things right the first time, the Data Management Specialist role is for you. You'll play a key part in helping teams track performance and uncover tax savings by managing property tax data with precision. Every day, your work will support smarter decisions and measurable results. This position is ideal for someone who thrives on consistency, values accountability, and finds satisfaction in completing tasks with care.

Schedule:

Monday to Friday from 8:00 AM to 5:00 PM CST (USA)

Your Responsibilities Will Include but Are Not Limited To:

• Inputting beginning and ending property tax values into pre-existing Excel spreadsheets
• Populating Excel templates to calculate tax savings and track KPIs
• Extracting public property tax data from appraisal district websites
• Transferring extracted data into client-provided formats
• Compiling reports from multiple data sources
• Supporting performance tracking such as consultant revenue per appeal season
• Ensuring data accuracy for meaningful KPI analysis

What We're Looking For:

• At least one year of experience in administrative support and data entry/management roles
• Strong experience with Excel and data entry required
• Ability to work independently with minimal supervision
• Comfortable with remote communication and task management
• Excel proficiency, including formulas and data manipulation
• Ability to create reports from multiple spreadsheets
• Strong data entry and reporting accuracy
• Comfortable retrieving and organizing public data
• Detail-oriented and committed to accuracy
• Self-motivated with a strong work ethic
• Reliable and accountable for daily task completion
• Tenacious with repetitive tasks and follow-through
• Proactive in asking questions and seeking clarity
• Ability to work both independently and in a team
• Reliable computer setup (laptop/PC, headset, stable internet min. 10 Mbps download / 5 Mbps upload)
• Quiet, distraction-free remote work environment

What's In It for You?

• Monthly compensation starting from
  1060 USD to 1150 USD
• Paid time off
  and holiday pay.
• Referral and annual
  bonuses.
• 100% Remote,
  Full-Time, Long-Term Career Opportunity.
• Parental leave.
• Opportunities for professional development and training
• Dedicated support from our team.
• A chance to work with clients who share our values.

Ready to dive in?
Apply now and make sure to follow all the instructions

DISCLAIMER: Every candidate must pass each step in our application process to become part of our team. Ensure you have all the required documentation ready to streamline your application process.

Job Overview

Business Segment: Personal & Private Banking

Location: ZA, GP, Johannesburg, 30 Baker Street

Job Type: Full-time

Job Ref ID: A-0002

Date Posted: 9/2/2025

Job Description

Provide infrastructure, tools and frameworks used to deliver end-to-end solutions to business problems. Build scalable infrastructure for supporting the delivery of clear business insights from raw data sources; with a focus on collecting, managing, analysing, visualising data and developing analytical solutions.Responsible for expanding and optimising Standard Bank's data and data pipeline architecture, whilst optimising data flow and collection to ultimately support data initiatives.

Qualifications

Type of Qualification: First Degree

Field of Study: Information Studies

Type of Qualification: First Degree

Field of Study: Information Technology

Experience Required

Data Monetisation

Data & Analytics

5-7 years

Experience with big data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

5-7 years

Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management. A successful history of manipulating, processing and extracting value from large disconnected datasets. Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.

5-7 years

Working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. Experience building and optimizing 'big data' data pipelines, architectures and data sets. Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

Additional Information

Behavioural Competencies:

Adopting Practical Approaches

Articulating Information

Challenging Ideas

Checking Things

Examining Information

Exploring Possibilities

Interpreting Data

Meeting Timescales

Producing Output

Providing Insights

Team Working

Upholding Standards

Technical Competencies:

Data Analysis

Database Administration

Data Integrity

Knowledge Classification

Please note: All our recruitment processes comply with the applicable local laws and regulations. We will never ask for money or any from of payment as part of our recruitment process. If you experience this, please contact our Fraud line on or

Company Description

INTEGRITY TRUCKING AND CONSTRUCTION SERVICES INC is a reputable construction company based in Balko, Oklahoma. The company is dedicated to delivering high-quality construction services to its clients. With a strong emphasis on integrity and excellence, the company aims to meet the diverse needs of its customers while maintaining high standards of safety and efficiency.

Role Description

This is a full-time hybrid role for an Executive: Data Management & Analytics, located in the Johannesburg Metropolitan Area with some work-from-home flexibility. The candidate will be responsible for overseeing data management processes, analyzing large datasets, developing data strategies, and providing actionable insights to drive business decisions. Daily tasks include managing data systems, conducting data quality assessments, ensuring data security, and collaborating with various departments to implement data solutions.

Qualifications

• Experience in Data Management and Governance
• Proficiency in Data Analysis, Data Visualization, and Business Intelligence tools
• Skills in Data Warehousing and Data Architecture
• Competence in Statistical Analysis and Predictive Analytics
• Strong problem-solving abilities and detail orientation
• Excellent communication and collaboration skills
• Ability to work independently and in a team in a hybrid environment
• Advanced degree in Data Science, Computer Science, Statistics, Information Systems, or related field
• Experience in the construction industry is a plus