EndereçoEviden, part of the Atos Group, with an annual revenue of circa € 5 billion is a global leader in data-driven, trusted and sustainable digital transformation. As a next generation digital business with worldwide leading positions in digital, cloud, data, advanced computing and security, it brings deep expertise for all industries in more than 47 countries. By uniting unique high-end technologies across the full digital continuum with 47,000 world-class talents, Eviden expands the possibilities of data and technology, now and for generations to come.
Our team is building Eviden Computer Vision Platform, a real-time video analytics solution for different verticals. We use AI technologies to design and develop the product, as well as Big Data software components to manage the related data.
We are seeking a skilled and motivated Data Engineer to complete our team on end-to-end data pipeline implementation and data lake operation.
Responsibilities:
- Build and maintain robust data pipelines to ingest, transform, and load data from various sources. Ensure data quality, consistency, and reliability throughout the pipeline.
- Implement data transformation logic to clean, enrich, and transform raw data into structured formats suitable for analysis and reporting. Leverage ETL/ELT processes as needed.
- Manage the data platform infrastructure, including choosing appropriate storage technologies, optimizing storage utilization, and ensuring data accessibility.
- Implement and enforce data security measures, access controls, and compliance standards to maintain the integrity and privacy of the data.
- Develop mechanisms for efficient data search and retrieval, considering relevancy, query performance, and user experience.
- Monitor and optimize the performance of data pipelines and storage systems to ensure efficient data processing and retrieval.
- Maintain clear and comprehensive documentation of data pipeline designs, processes, and configurations to facilitate knowledge sharing and future maintenance.
- Implement workflows to automate the building, testing and deployment of data lake components following DevOps practices.
- Implement unit and integration tests following guidelines and propagate knowledge across the team.
- Manage AI assets (i.e., datasets, models) properly and securely.
- Integrate meta-data extraction components leveraging AI models and third-party tools such as labelling frameworks.
- Collaborate effectively with cross-functional teams including data scientists, Data Engineers, frontend and backend developers, and product owners to align on project goals and requirements.
Education
- Bachelor's, Master's, or PhD in Computer Science, Electrical Engineering, or a related field.
Essential knowledge and professional experience
- Proven experience (3+ years) in designing, building, and maintaining large-scale data pipelines and data lake infrastructure.
- Strong proficiency in programming languages such as Python.
- Hands-on experience in REST API development.
- Hands-on experience with Elasticsearch, including data ingestion, indexing, and search capabilities. Familiarity with Elasticsearch Query DSL and search relevancy concepts.
- Knowledge of data modelling, schema design, and ETL/ELT processes.
- Prepare software applications for their deployment using Docker and Kubernetes.
- Proficiency in the usage of Git and GitHub Actions.
- Practice of agile methodology.
- Proficient user of Linux environments (bash or shell).
- English level B2
Additional knowledge
- Experience in MLOps tools, e.g., MLFlow, Kubeflow.
- Experience with Google Cloud Platform (GCP).
- CPU vs GPU programming.
- General knowledge about clusters.
Competences
- Autonomy: capacity to seek and read documentation.
- Ability to collaborate: provide constructive comments and embrace best practices and guidelines.
- Fluency in English
- Good writing and presentation skills.
- Strong personal soft skills set communicative, enthusiastic, highly collaborative, proactive, and self-driven.
- Ability and enthusiasm to learn new technologies quickly.
Benefits
- Half-day Fridays.
- Intensive workday in summer.
- Personalised training and upskilling programme.
- Flexible working hours and ways of working.
- R&D environment. New ideas and technologies are welcome.
Let’s grow together.
