Data Engineer

Description

Job Summary: We are seeking a Data Engineer with experience in Databricks and cloud technologies to manage projects, teams, and communication within an agile environment. Key Highlights: 1. Key role in projects involving Databricks and cloud technologies. 2. Opportunity to work with agile methodologies and Azure DevOps. 3. Recognition of Databricks certifications and industrial experience. Minimum Requirements (Mandatory) * Minimum 2 years of hands-on experience in projects using Databricks and cloud technologies. * Proficiency in Databricks, SQL, and Python; PySpark for distributed data processing. * Delta Lake and Unity Catalog: layered modeling (bronze / silver / gold), permission management, and lineage tracking. * Creation, orchestration, and monitoring of production pipelines using Databricks Workflows, Delta Live Tables (DLT), or equivalents (at least 2 years). * Azure Data Factory and Azure Data Lake Storage (data flow integration, transformation, and management). * Databricks Asset Bundles (DAB) for deployment and productionization. * Strong software engineering practices (modular, maintainable code with testing and documentation) and version control **using Git:** branching, pull requests, and code reviews. * Agile methodologies and Azure DevOps; strong data quality practices (validations for completeness, duplication, valid ranges, and consistency). Certifications (Valued by the Client) * Preferred: Databricks Certified Data Engineer Associate (valued / differentiator in application). * Preferred: Databricks Certified Associate Developer for Apache Spark (valued / differentiator in application). * Preferred: DP\-750 — Azure Databricks Data Engineer Associate. Project, Team, and Communication Management * Time management: meets commitments and deadlines autonomously. * Requirements management: understands and translates technical requirements with support from the team lead. * Team/people management: coordinates effectively with own team and data-providing departments. * Communication: clear with the technical team and in documentation of deliverables. Desirable Requirements (Differentiators) * Experience with industrial data sources: process historians (PI System or equivalents), LIMS, dispatch systems, or ERP systems (adds differentiation value). * Prior experience in data projects within the mining industry (non-exclusive; no knowledge of mining processes required). Academic Qualifications Professional degree in Computer Science Engineering, Computing Engineering, Data Engineering, Mathematical Engineering, or a related field, with a solid foundation in databases, data architecture, and programming.