Job Description

Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark to ensure efficient data processing and analysis.
• Oversee the implementation and optimization of Databricks SQL queries to support data-driven decision-making.
• Provide expertise in Databricks Workflows to automate and streamline data workflows enhancing productivity.
• Collaborate with cross-functional teams to gather requirements and design data solutions that meet business needs.
• Ensure data quality and integrity by implementing robust data validation and testing procedures.
• Monitor and troubleshoot data pipelines and workflows to identify and resolve issues promptly.
• Optimize data storage and retrieval processes to improve performance and reduce costs.
• Lead the development of data models and schemas to support analytics and reporting.
• Implement best practices for data security and compliance to protect sensitive information.
• Provide technical guidance and mentorship to junior developers fostering a culture of continuous learning and improvement.
• Stay updated with the latest industry trends and technologies to drive innovation and improve existing processes.
• Document technical specifications and maintain comprehensive project documentation for future reference.
• Collaborate with stakeholders to ensure data solutions align with business objectives and deliver measurable impact.

Qualifications

• Possess a strong proficiency in Python programming for data processing and analysis.
• Demonstrate expertise in Databricks SQL for querying and managing large datasets.
• Have hands-on experience with Databricks Workflows to automate data pipelines.
• Show proficiency in PySpark for distributed data processing.
• Exhibit excellent problem-solving skills and attention to detail.
• Have strong communication and collaboration skills to work effectively with cross-functional teams.
• Display a commitment to continuous learning and staying updated with industry trends.
• Possess a bachelors degree in Computer Science Information Technology or a related field.
• Have a proven track record of delivering high-quality data solutions in a fast-paced environment.
• Show experience in implementing data security and compliance best practices.
• Demonstrate the ability to mentor and guide junior developers.
• Have strong documentation skills to maintain project records.
• Show a proactive approach to identifying and resolving technical issues.