Job Description
Overview of the role
As an Data engineer in our team, you will be responsible for governing solutions, assessing complex new data sources with various SAP and non-SAP source systems, and quickly turning these into business insights-ready data sets. You will participate in new scope & requirements, break-fix work, testing, and support activities to meet the business needs. Implementing appropriate processes to drive high-quality and sustainable solutions will be critical.
What you will do
- Partners with the product owner, scrum master and solution architect to deliver solutions.
- Understanding the business requirements.
- Responsible for reviewing and analyzing business requirements and designing technical solutions.
- Responsible for governance of the technology implementation of the solution designs
- Collaborate with internal and external implementation team members and partners to develop, test, and deploy solutions.
- Adheres to software development and maintenance standards by thoroughly understanding internal and external development best practices.
- Estimate and deliver multiple project assignments and system change requests with the required quality according to agreed-upon resourcing and timelines.
- You must analyze structured, semi-structured, and unstructured data sources in detail for quality, completeness, and business value.
- Responsible for governance and implementing robust data pipeline using Microsoft toolset and Databricks.
- Responsible for creating reusable and scalable data pipelines.
Skills
Required skills to be successful
- Expertise in data ingestion, data processing, and analytical pipelines for big data, relational / non-relational databases, and data warehouse solutions
- Experience in designing and hands-on development in cloud-based analytics solutions.
- Designing and developing warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services.
- Build workflows using data from various sources, including on-premises databases and external data sources with rest APIs and harvesting tools.
- Strong analytic skills related to working with unstructured datasets.
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.
- Hands-on experience with Agile software development
- Thorough understanding of Azure / AWS Cloud Infrastructure offerings.
- Expertise in Azure Stream Analytics is desirable.
- Knowledge of Scala is desirable.
What equips you for the role
- Relevant 8+ years of experience in Azure and Databricks.
- Minimum 5+ years of implementation experience from conceptualization to delivery of analytics capabilities on the Azure platform.
- Bachelor’s degree in computer science or an allied field of study.
- Knowledge of SAP BW/BO, Tableau, Power BI, and Scala will be considered as an added advantage.
- Certification on Azure platform toolsets will be considered an added advantage.