Data Engineer
Pune, IndiaFull-TimeMid-levelSoftware Engineering
Responsibilities will include
- Data Pipeline Development: Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
- ETL Processes: Develop robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources into our data ecosystem.
- Data Quality Assurance: Implement data validation and quality checks to ensure accuracy and consistency.
- Data Modeling: Design and maintain data models, schemas, and database structures to support analytical and operational use cases.
- Data Warehousing: Design and manage data warehouses or data lakes to support business intelligence and reporting.
- Performance Optimization: Continuously monitor and optimize data pipelines and queries for maximum efficiency and scalability.
- Data Integration: Build and maintain integrations with internal and external data sources and APIs.
- Infrastructure Management: Configure and manage data infrastructure components, including databases, data warehouses, and distributed computing frameworks.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver tailored solutions.
- Documentation: Document technical designs, workflows, and best practices for knowledge sharing.
Minimum Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in data engineering or a similar role.
- Proficiency in programming languages commonly used in data engineering (e.g., Python).
- Strong knowledge of database systems, data modeling techniques, and SQL proficiency.
- Experience with big data technologies (e.g., Hadoop, Spark) and cloud platforms (e.g., AWS, Azure).
- Excellent problem-solving skills with attention to detail.
- Familiarity with ETL tools (e.g., SSIS, Databricks).
- Effective communication skills for collaborating in a team-oriented environment.
- Ability to adapt to evolving technologies and business requirements.
