Data Architect & Data Engineer, Databricks, Python, Spark, Software Development, BI, Machine Learning Engineering
Aktualisiert am 18.11.2024
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 02.12.2024
Verfügbar zu: 60%
davon vor Ort: 5%
Solution Architect
Python
Databricks
Data Lakehouse
BigQuery
Airflow
PostgreSQL
Hadoop
Elastic Search
Docker
Jenkins
GCP
Flask
Kafka
Azure
Azure Data Factory
Synapse
Kubernetes
OpenShift
AWS
Terraform
Apache Spark
Datenarchitektur
German
native
Serbo-Croatian
native
English
fluent

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

1 Jahr 5 Monate
2023-06 - 2024-10

Databricks Data Lakehouse solution

Data Architect Python SQL
Data Architect
  • Led end-to-end data architecture for a Swiss-German manufacturing company?s analytics solution, successfully delivered an Azure Databricks based Data Lakehouse (using Terraform, Azure Databricks, Azure DevOps, PySpark, Data Factory, Fivetran, Power BI)
  • Set up development standards, branching strategy, CI/CD and documentation for data engineering
Databricks Terraform Azure Devops Data Factory PySpark Fivetran Power BI
Python SQL
5 Monate
2023-01 - 2023-05

Monitoring Migration to OpenShift

SRE Engineer and Airflow Architect Python
SRE Engineer and Airflow Architect

  • Automating Kibana Monitor generation as well as generation of Grafana Dashboards
  • Setting up Airflow on OpenShift


Kubernetes OpenShift Elastic Search Airflow
Python
ECC AG / Deutsche Börse AG
1 Jahr
2022-01 - 2022-12

Solo founder

Solo Founder
Solo Founder
  • Built & released schedule generation web app based on user?s google calendar availabilities and todos (based on Flask, Docker, Redis, Celery, Postgres, genetic algorithm) 
1 Jahr 6 Monate
2020-06 - 2021-11

Data Engineer

Data Engineer
Data Engineer
Zattoo is one of the leading TV streaming providers in Europe and was acquired by TX Ventures. 

  • Designed and built a data mart for subscriber activity metrics as part of company wide effort to consolidate company success metrics (with Airflow and BigQuery). 
  • Contributed to GCP based data warehouse redesign: introducing Kafka, a data lake and BigQuery
  • Set up Airflow as the new main orchestration tool along with best practices as well as a Docker based development environment

Zattoo ? Berlin, Germany
1 Jahr 7 Monate
2018-04 - 2019-10

Big Data Engineer

Data Engineer
Data Engineer
Motionlogic was a Deutsche Telekom owned startup offering traffic & location reports. 

  • Designed and implemented a query engine using PySpark, HDFS, Redis & MongoDB to produce individually billable reports which largely expanded the product line 
  • Collaborated closely with Data Science team on quality and performance improvements of the core business algorithm for trip/activity extraction from movement chains and thereby ensured product satisfaction of some of Europe?s largest telecommunications companies. Implemented in PySpark to process more than 3TB per day for on-premise cluster with >1000 cores, 60 servers, >10 TB RAM.

Motionlogic ? Berlin, Germany
7 Monate
2015-04 - 2015-10

Implemented extensive integration

Quality Control Analyst Intern
Quality Control Analyst Intern
i4i is a VC-funded startup providing structured content apps for the life sciences industry to solve compliance.

  • Implemented extensive integration and regression testing in Python and maintained documentation

i4i ? Toronto, Canada

Aus- und Weiterbildung

Aus- und Weiterbildung

Kompetenzen

Kompetenzen

Top-Skills

Solution Architect Python Databricks Data Lakehouse BigQuery Airflow PostgreSQL Hadoop Elastic Search Docker Jenkins GCP Flask Kafka Azure Azure Data Factory Synapse Kubernetes OpenShift AWS Terraform Apache Spark Datenarchitektur

Programmiersprachen

Python
Experte
JavaScript
Fortgeschritten
Go
Basics
Scala
Fortgeschritten

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

1 Jahr 5 Monate
2023-06 - 2024-10

Databricks Data Lakehouse solution

Data Architect Python SQL
Data Architect
  • Led end-to-end data architecture for a Swiss-German manufacturing company?s analytics solution, successfully delivered an Azure Databricks based Data Lakehouse (using Terraform, Azure Databricks, Azure DevOps, PySpark, Data Factory, Fivetran, Power BI)
  • Set up development standards, branching strategy, CI/CD and documentation for data engineering
Databricks Terraform Azure Devops Data Factory PySpark Fivetran Power BI
Python SQL
5 Monate
2023-01 - 2023-05

Monitoring Migration to OpenShift

SRE Engineer and Airflow Architect Python
SRE Engineer and Airflow Architect

  • Automating Kibana Monitor generation as well as generation of Grafana Dashboards
  • Setting up Airflow on OpenShift


Kubernetes OpenShift Elastic Search Airflow
Python
ECC AG / Deutsche Börse AG
1 Jahr
2022-01 - 2022-12

Solo founder

Solo Founder
Solo Founder
  • Built & released schedule generation web app based on user?s google calendar availabilities and todos (based on Flask, Docker, Redis, Celery, Postgres, genetic algorithm) 
1 Jahr 6 Monate
2020-06 - 2021-11

Data Engineer

Data Engineer
Data Engineer
Zattoo is one of the leading TV streaming providers in Europe and was acquired by TX Ventures. 

  • Designed and built a data mart for subscriber activity metrics as part of company wide effort to consolidate company success metrics (with Airflow and BigQuery). 
  • Contributed to GCP based data warehouse redesign: introducing Kafka, a data lake and BigQuery
  • Set up Airflow as the new main orchestration tool along with best practices as well as a Docker based development environment

Zattoo ? Berlin, Germany
1 Jahr 7 Monate
2018-04 - 2019-10

Big Data Engineer

Data Engineer
Data Engineer
Motionlogic was a Deutsche Telekom owned startup offering traffic & location reports. 

  • Designed and implemented a query engine using PySpark, HDFS, Redis & MongoDB to produce individually billable reports which largely expanded the product line 
  • Collaborated closely with Data Science team on quality and performance improvements of the core business algorithm for trip/activity extraction from movement chains and thereby ensured product satisfaction of some of Europe?s largest telecommunications companies. Implemented in PySpark to process more than 3TB per day for on-premise cluster with >1000 cores, 60 servers, >10 TB RAM.

Motionlogic ? Berlin, Germany
7 Monate
2015-04 - 2015-10

Implemented extensive integration

Quality Control Analyst Intern
Quality Control Analyst Intern
i4i is a VC-funded startup providing structured content apps for the life sciences industry to solve compliance.

  • Implemented extensive integration and regression testing in Python and maintained documentation

i4i ? Toronto, Canada

Aus- und Weiterbildung

Aus- und Weiterbildung

Kompetenzen

Kompetenzen

Top-Skills

Solution Architect Python Databricks Data Lakehouse BigQuery Airflow PostgreSQL Hadoop Elastic Search Docker Jenkins GCP Flask Kafka Azure Azure Data Factory Synapse Kubernetes OpenShift AWS Terraform Apache Spark Datenarchitektur

Programmiersprachen

Python
Experte
JavaScript
Fortgeschritten
Go
Basics
Scala
Fortgeschritten

Vertrauen Sie auf Randstad

Im Bereich Freelancing
Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

Fragen?

Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Das Freelancer-Portal

Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.