Design and implement scalable data processing solutions using Azure Databricks
- Develop and maintain data pipelines to ingest data from various sources
- Optimize data storage and retrieval processes for efficiency and performance
- Ensure data quality and integrity throughout the data lifecycle
- Troubleshoot and resolve data processing issues
- Deploy and manage big data solutions in a distributed computing environment
- Collaborate with data engineers and data scientists to facilitate data accessibility
- Document data processing solutions and best practices
Requirements:
- Bachelors degree in Computer Science, Information Technology, or a related field
- 4+ years of experience in data engineering or software development
- Proficiency with Azure Databricks and other Azure services
- Strong understanding of data warehousing concepts
- Experience with big data tools and frameworks (e.g., Hadoop, Spark)
- Excellent analytical and problem-solving skills.