Data Lineage / Meta data Engineer
Location : Mississauga, Canada (onsite)
Role Summary
Developer focused on data lineage to build systems that parse and analyze large, multi-language codebases and automatically generate accurate data lineage. The role pairs strong Python engineering with hands-on Gen AI/LLM work and ownership of build, deployment, and delivery pipelines. The emphasis is on clean, efficient, well-architected design - systems that are correct, performant, and maintainable as they scale.
What You'll Do
- Design and build engines that parse multiple programming and query languages to extract accurate, column-level data lineage.
- Build Gen AI/LLM-powered retrieval and assistant capabilities on top of lineage and metadata.
- Develop backend services and the interfaces that expose them, from data processing through to user-facing views.
- Own the delivery pipeline - build, test, deployment, and production support across environments.
Design Principles We Value
- Simplicity first - favor the simplest design that solves the problem; avoid unnecessary complexity and premature abstraction.
- Separation of concerns - clear module boundaries, single-responsibility components, and well-defined interfaces so pieces can evolve independently.
- Correctness and determinism - prefer predictable, testable behavior; ground AI outputs in verifiable data rather than guesswork.
- Performance-aware design - think about time/space complexity, memory footprint, and efficient data structures and algorithms up front.
- Scalability and extensibility - design so new languages, sources, and capabilities can be added without rewrites.
- Robustness - graceful error handling, fault isolation, and defensive design so one bad input can't take the system down.
- Readability and maintainability - code and designs the next engineer can understand, extend, and debug.
- Reuse over duplication - shared models and common abstractions instead of repeated logic.
Required Skills
- Strong Python development with solid object-oriented and functional design skills.
- Software engineering fundamentals: data structures, algorithms, and complexity analysis.
- Data lineage / data flow analysis: parsing and analyzing source code (AST-based and pattern-based static analysis) and comfort reading multiple languages.
- Hands-on Gen AI/LLM experience: prompt design, tool/function calling, and agent orchestration.
- Understanding of retrieval-augmented generation and vector/semantic search concepts.
- Experience designing APIs and backend services, plus the front-end fundamentals to build usable interfaces.
- Pipeline and deployment experience: build/test automation, CI/CD, and releasing across environments.
- Data manipulation and transformation skills.
Nice to Have
- Graph algorithms and traversal techniques for dependency/lineage tracing.
- Data governance, metadata management, and data cataloging concepts.
- Experience designing systems for long-running or asynchronous workloads.
- Infrastructure automation and deployment tooling.
Key Responsibilities
- Translate ambiguous requirements into clean, efficient, well-structured designs.
- Extend parsing and analysis capabilities to new languages and frameworks without compromising architecture.
- Design and productionize Gen AI/LLM and retrieval features that are accurate and grounded in real data.
- Build and maintain deployment pipelines; support delivery across environments.
- Write tests and maintain quality across the development lifecycle.
- Collaborate on design reviews and raise the engineering bar for the team.
