At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries. If your passion for data analytics solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering!

We are looking for someone to work from office location – Vikroli / Churchgate, Mumbai

Responsibilities:

• Design, develop, and maintain data pipelines and workflows using PySpark to process large volumes of data efficiently.
• Implement data ingestion, transformation, and storage solutions to support business requirements.
• Develop and optimize Python scripts and algorithms for data manipulation, analysis, and modeling tasks.
• Design and manage database systems, including schema design, indexing, and performance optimization.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.
• Perform data quality assessment, validation, and cleansing to ensure accuracy and integrity of data.
• Monitor and troubleshoot data pipeline issues, performance bottlenecks, and system failures.
• Stay up-to-date with emerging technologies and best practices in data engineering and analytics.

Qualification:

• Bachelor’s degree in Computer Science, Engineering, or related field.
• Should have a minimum of 3 years of experience in the field of Database or Data Analytics
• Minimum of 3 years of experience in data engineering, with expertise in Spark, Python, and database technologies.
• Experience with PySpark for data processing and analysis tasks.
• Proficiency in Python programming for scripting and automation.
• Solid understanding of database concepts and hands-on experience with SQL and NoSQL databases.
• Experience with data modeling, schema design, and ETL processes.
• Strong problem-solving skills and ability to work independently and collaboratively in a fast-paced environment.
• Excellent communication and interpersonal skills, with the ability to effectively communicate technical concepts to non-technical stakeholders.
• Good to have experience with a cloud platforms.