Python Developer
Job Description
Job Description Developer 8 Yrs in Design and dveloping ETL Pipelines using on AWS EMR / SAS SQL Logic into Python coding using frame work such as Pyspark. We are seeking an experienced Python Developer with a strong background in designing and developing ETL (Extract, Transform, Load) pipelines on AWS EMR. The ideal candidate will be proficient in leveraging Python for complex data processing tasks and have hands-on experience with frameworks such as PySpark. The role requires an expert understanding of SAS SQL logic, along with the ability to convert it into Python code effectively. Key Responsibilities : – Design and Develop ETL Pipelines : Lead the development of efficient and scalable ETL pipelines on AWS EMR using Python and PySpark. – SAS to Python Translation : Convert and optimize complex SAS SQL logic into Python code, ensuring high performance and maintainability. – Data Transformation : Work with large data sets to transform raw data into meaningful insights and ensure the integrity and accuracy of the transformed data. – Cloud Infrastructure : Utilize AWS services, primarily AWS EMR, to manage and scale data processing tasks. – Performance Optimization : Improve the performance of existing ETL pipelines and processes to handle large-scale data volumes. – Collaboration : Work closely with cross-functional teams including data engineers, data scientists, and business analysts to understand requirements and deliver solutions. Required Skills & Experience : Experience : 8 years of hands-on experience in Python development, particularly in data engineering and building ETL pipelines. Technical Expertise : – Strong knowledge of AWS EMR, PySpark, and related cloud technologies. – Experience working with large datasets and data warehouses. – Proficiency in converting SAS SQL logic into Python code. – In-depth knowledge of SQL, including complex data queries and optimization. – Familiarity with distributed data processing and cloud-based data solutions. – Problem-Solving Skills: Ability to debug and optimize code, troubleshoot complex issues, and propose effective solutions. – Version Control & Collaboration Tools: Familiarity with Git and Agile methodologies. Preferred Skills : – AWS Certifications (e.g., AWS Certified Big Data – Specialty) is a plus. – Experience with other big data tools like Apache Hive, HBase, or Redshift. – Familiarity with data visualization tools or libraries (e.g., Matplotlib, Seaborn). Education : Bachelor’s degree in Computer Science, Information Technology, or related field. Advanced degrees are a plus. Soft Skills : – Excellent communication skills to collaborate with both technical and non-technical teams. – Strong analytical and troubleshooting abilities. – Ability to work in a fast-paced, dynamic environment. (ref:hirist.tech)