← All jobs

Senior Python Streaming & Data Engineer (RabbitMQ / Snowflake / PostgreSQL)

Coopers Group AG

Employment type
Contract
Workload
100%
Location
Bern
Languages
German, German
First posted
Apply now
  • IT - Software Engineering
  • Bern
  • Contracting
  • Part-time
  • Snowflake, PostgreSQL, RabbitMQ

The two engineers, who together are responsible for the entire data processing chain, should ideally complement each other with their technical focuses as follows:

  • Streaming & Performance: Focus on RabbitMQ consumer development, Python Concurrency, performance optimization, and stable processing under high load
  • Data Integration & Recovery: Focus on Snowflake/PostgreSQL, reliable data ingestion as well as recovery, replay, and backfill processes

For both focuses, practical experience with RabbitMQ and demonstrable project experience in production environments are required.

Tasks:

  • Development and optimization of a scalable Python application for processing RabbitMQ data streams
  • Implementation and further development of RabbitMQ consumers for production streaming environments
  • Integration and configuration of data ingestion into Snowflake and Snowflake Postgres
  • Ensuring reliable message processing including acknowledgements, redelivery, backpressure, and deduplication
  • Conception and implementation of robust recovery, replay, and backfill processes
  • Optimization of latency, throughput, as well as CPU and memory consumption
  • Analysis and resolution of performance bottlenecks, load peaks, and error cases
  • Ensuring data completeness and data consistency for live and historical data
  • Integration of reload processes using Kestra and dlt (dltHub) in cooperation with the internal Data Ingestion Engineer
  • Development of automated tests as well as conducting performance and load tests
  • Integration of the solution into Azure DevOps CI/CD and implementation of basic monitoring functions
  • Close cooperation with internal platform and engineering teams as well as documentation and knowledge transfer

Profile:

  • Very good practical experience in Python development, especially with long-running production applications
  • Sound experience with RabbitMQ, especially consumer development, message processing, and error handling
  • Practical experience with streaming systems under real load as well as their performance optimization
  • Good knowledge of Snowflake and PostgreSQL, especially in the area of data ingestion and data integration
  • Experience with recovery, replay, backfill, deduplication, and data consistency
  • Very good understanding of acknowledgements, redelivery, backpressure, and message queuing
  • Knowledge of Python Concurrency, CPU/memory profiling, and performance optimization
  • Experience in the development and operation of production-ready data pipelines
  • Knowledge of automated tests, CI/CD, and monitoring
  • Experience with Azure DevOps as well as modern development and deployment processes
  • Knowledge of Azure, AKS, Kestra, dlt, Kafka, or Debezium is an advantage
  • Independent, structured, and solution-oriented way of working as well as strong teamwork skills
  • Experience in cooperating with internal platform, development, and operations teams

Automatically translated from the original.

Posted 6 days ago

Location

View on Google Maps

See this page on jobsswitzerland.ch