EndereçoSigma Software is looking for an experienced Data Engineer to join our growing engineering team. This opportunity is for you if you want to work with a tightly knit team of Data Engineers solving challenging problems using cutting-edge data collection, transformation, analysis, and monitoring tools in the cloud.
Sounds like you? We are waiting for you on our team!
CUSTOMER
Our client has superior end-to-end technology, a premium marketplace, and best-in-market advisory services that power the world's advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe
PROJECT
We invite you to work with a provider of comprehensive ad platforms for publishers, advertisers, and media buyers. We build and support high-quality data solutions to process terabytes of data on AWS's cloud-native data platform. Our project helps build graphs of graphs.
Job Description
- Extensively use several cutting-edge AWS services, 3rd party commercial software, and internally developed software to store, process, and deliver near real-time data
- Design, manage, improve, and build the next generation of data tools and frameworks to develop and maintain data products and infrastructure
- Support and troubleshoot data and/or system issues
- Collaborate with external teams to identify, troubleshoot, and permanently resolve data quality issues
- Develop in-house tools to automate common tasks
- Conduct code and design reviews to ensure the product quality remains high
- Mentor and guide colleagues and new team members
- Collaborate with Data Engineers and Product Managers to translate complex product requirements into working high-quality cloud-native data solutions
- Share learnings with wider engineering teams by doing technical demos
Qualifications
- 4+ years of hands-on experience in Software development and Big Data
- 4+ years of hands-on experience working with Python
- Hands-on experience building ETL pipelines using our internal framework
- Strong knowledge of algorithms and data structures
- Production experience with Spark
- Strong skills in SQL and proven experience working with major database platforms
WOULD BE A PLUS
- Knowledge of Databricks
Additional Information
PERSONAL PROFILE
- Excellent communication and writing skills
- A curious problem solver
- Versatile team player
