Data Engineer
Description
Metric5 is currently seeking a Data Engineer to work within a team providing Data Warehouse and Business Intelligence services to our government customer using Agile processes. As a Data Engineer, you will work with varying huge data sources with different schemas and data elements to produce an effective and efficient Data Warehouse. You have an eye for spotting data correlations and a desire to dig into large datasets to find technical solutions and deliver business value.
Program Details
The program you will be supporting has a mission to provide development, security, and operations (DevSecOps) support to U.S. Citizenship and Immigration Services (USCIS) with a focus on development, operations, and modernization of the Agency’s Enterprise Data Warehouse/Data Lake. The team utilizes open-source, AWS Cloud, and Big Data technologies, agile project management practices, and modern DevSecOps delivery to provide the business intelligence support systems to meet the reporting, data analytics, and machine learning/artificial intelligence needs critical to USCIS leadership, data/business analysts, data scientists, and other decision-makers.
Requirements:
- Interact with designated product owners, system owners, and source system business owners to understand transactional system data models and elicit requirements and logic for ETLs
- Develop ETL workflows/data pipelines to ingest data using AWS Data Migration Service (DMS), Scala, Kafka, Restful APIs, and other technologies as determined by the client from multiple transactional systems to the target (including ODS, data marts, and data lake) according to documented logic and source-to-target mappings
- Troubleshoot data discrepancy and missing data issues resulting from daily ETL loads
- Work with operations team to deploy ETL jobs in integration and production environment and debug / troubleshoot critical production issues.
- Actively participate in Agile release development activities and ceremonies, including sprint planning, sprint grooming, artifact creation, sprint testing, demonstrations and retrospectives and solution releases.
- Document ETL logic, mappings, etc. in a concise and traceable manner to be used as a reference for future development and maintenance
- Execute other activities related to development work, such as participate in meetings, provide briefings, presentations and other support materials that will promote the program, assist in achieving user buy-in, and explain technical concepts to non-technical audiences.
Required Skills:
- 5+ years of experience with ETL development ingesting data from diverse and huge data sources
- 5+ years of experience with programming languages such as Java, Scala, Python, R, JSON Schema
- 2+ years of experience producing and consuming Rest APIs.
- 5+ years of experience with relational databases used to support BI analytics
- Experience with Databricks/Apache Spark structured streaming, and/or Kafka experience
- Demonstrated experience in a Data Warehouse/Data Lake and Business Intelligence environment
- Ability to write complex SQL queries and scripts
- Strong teamwork, co-ordination, planning and influencing skills
- Self-driven with the ability to adapt quickly, work in a challenging and fast paced environment within cross-functional teams, and to promote creative problem solving within their team
- Experience with Agile development practices, including Scrum and Kanban, and management tools (e.g., Jira, Confluence)
- Experience with GIT and branching strategies
- Experience with engineering/DevOps tools (i.e., Jenkins)
- Excellent analytical, communication and organizational skills
- Experience working in AWS Cloud environment
- Experience with Microsoft Office Suite including Excel, PowerPoint, and Visio
Desired Skills:
- Experience with AWS Database Migration Service (DMS)
- Experience with Postgres and Oracle
- Experience with Scaled Agile Framework SAFe
Education: Bachelor’s degree in a technical discipline preferred – Computer Science, Mathematics, or equivalent technical degree, or the equivalent combination of education, professional training, and work experience.
Location: Reston, VA – Currently fully remote due to COVID-19.
Clearance: Must be a US Citizen and be able to obtain a government agency Suitability Clearance. USCIS Entry on Duty (EOD) preferred.
