Role: Data Engineer - 2

Responsibilities:

• Designing, developing, and maintaining data architecture, data pipelines, and data workflows.

• Building and optimizing large-scale data processing systems for both structured and unstructured data.

• Collecting, integrating, and transforming data from various sources, including databases, APIs, and file systems.

• Collaborating with data scientists, analysts, and other stakeholders to understand data requirements and design effective data solutions.

• Developing and implementing data quality and data governance processes to ensure data accuracy, consistency, and reliability.

• Writing efficient and scalable code/scripts to automate data processing tasks and optimize data workflows.

• Monitoring and troubleshooting data pipelines to ensure their performance, availability, and reliability.

• Identifying and implementing improvements to enhance data infrastructure, including data storage, processing, and retrieval mechanisms.

• Keeping up to date with the latest technologies, tools, and trends in the field of data engineering.

• Mentoring and providing guidance to junior data engineers, as

needed.

Qualifications:

• Bachelor's degree in computer science, data science, or a related field. Advanced degree preferred.

• Proven experience as a data engineer or in a similar role, handling large-scale data processing and analytics projects.

• Strong programming skills in languages like Python, Java, or Scala, with experience in data manipulation, scripting, and automation.

• Proficiency in working with relational databases (SQL) and NoSQL databases.

• Experience with big data technologies and frameworks, such as Hadoop, Spark, Kafka, or Hive.

• Knowledge of data integration and ETL (Extract, Transform, Load) processes.

• Familiarity with cloud platforms, such as AWS or GCP, and their data-related services.

• Understanding of data modelling concepts and data warehousing= principles.

• Strong problem-solving and analytical skills, with the ability to troubleshoot and optimize data processing workflows.

• Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.