| Description: |
Data Engineer Inside IR35 Rate: 500.00 per day
Duration: 6 Months
Location: UK Remote, occasional travel to London.
Clearance: Active SC Clearance The Opportunity We're looking for an experienced SC Cleared Data Engineer to join a large-scale data platform programme responsible for collecting, processing, and analysing vast volumes of customer interaction data. Working within a collaborative Agile team, you'll help build and maintain modern data products that support analytics, risk assessment, and operational decision-making. This role offers the opportunity to work with large-scale datasets, cloud-native technologies, and modern data engineering practices in a "you build it, you run it" environment. Key Responsibilities - Design, build, and maintain scalable data pipelines and data products.
- Process, validate, cleanse, and enrich large datasets from multiple sources.
- Develop and productionise data solutions using Python, Spark, and AWS technologies.
- Create and maintain Jupyter notebooks to support analysis and data engineering activities.
- Collaborate with multidisciplinary teams to deliver data-driven solutions.
- Support the publication, sharing, and consumption of data products.
- Maintain and improve automated data processing workflows.
- Champion testing, automation, reliability, and continuous improvement.
Essential Skills & Experience - Strong commercial experience with:
- Python
- Spark
- SQL
- Database technologies
- AWS
- Experience building ETL and data transformation solutions.
- Strong understanding of data engineering best practices.
- Experience working with large-scale datasets.
- Knowledge of data modelling, validation, and data quality processes.
- Experience working within Agile delivery environments.
- Familiarity with CI/CD and automated deployment practices.
- Experience supporting and maintaining production data platforms.
Desirable Skills - Scikit-Learn or Spark MLlib
- AWS EMR
- Apache Airflow
- Jenkins
- Hudi or other open table formats
- Kibana
- Grafana
- Understanding of Data Science principles
If you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website.  |
