Responsible for delivering relevant technical solutions based on standard methodologies primarily for data collection, storing, processing and provisioning huge data sets. Successful candidate should be motivated, able to work with minimum supervision and have extensive experience in Big Data Technologies, Hadoop/Cloud Data Lake, ELT processing, Reporting and a background in finance. Candidate will be encouraged to work as an individual contributor as well as provide oversight and direction to other technical resources as required.
The Work Itself
Utilize multiple development languages/tools to build prototypes and evaluate results for efficiency and feasibility Design, develop, test, and implement data-driven solutions to meet business requirements, ability to quickly identify an opportunity and recommend possible technical solutions by working with third party vendors. Provide business analysis and develop ETL code to meet all technical specifications and business requirements according to the established architectural designs Extracting business data from multiple structured and unstructured data sources, utilizing data pipeline to ingest data in Enterprise Data Lake - hybrid environment Experience in Custom Data pipeline development (Cloud and in-house local) and migrate data from large-scale data environments -- Oracle, SQL server with experience in end-to-end design and build process of Near-Real Time and Batch Data Pipelines. Deploys application code using CI/CD tools and techniques and provides support for deployed data applications using Jenkins, GitHub Collaborating as part of a multi-functional Agile team to build and improve software for next generation Big Data and Cloud applications Performing unit tests and conducting reviews with other team members to ensure code is designed with high code coverage by code scans using Sonarqube, checkmarx Willing to take ownership of pipeline and can communicate concisely and persuasively to varied audience including data provider, engineering and analysts. Ability to research and assess open source technologies and components to recommend and integrate into the design and implementation Gaining business domain knowledge of the datasets and applying that knowledge to further enhance the project deliverables for the business.
The Skills You Bring
Bachelor's or Master's degree in related field or equivalent experience Minimum 3+ years of related technical experience, working with and understanding data Experience interpreted languages to achieve data engineering tasks. Expertise in Data technologies and tools like Spark, Hive, NiFi, Sqoop, Impala Experience in Cloud Technologies: AWS, Glue, Lambda, EMR and cloud database (Snowflake, Redshift, cloud RDBMS nice to have) Demonstrate ability to work with team members and clients to assess needs, provide assistance and resolve problems Excellent problem-solving skills, verbal/written communication, and the ability to explain technical concepts to business partners. Proven design and development background in ETL, DW, BI and data migration projects. Partner with Development teams to ensure Coding standards are in alignment with DevOps practices with respect to Tools, Standards and Security Ability to work in a fast-paced environment and handle multiple priorities and shifting priorities in parallel. Automation mind-set -- an innate aim to continuously look for ways to automate existing processes. Experience working on Agile Scrum teams. Willingness to learn and try new techniques and technologies.
KP RECRUITING GROUP
KP Recruiting Group was started to create a world-wide network of multi-level, highly skilled talent; that are either experts or working to become one, in their respective fields. Our process involves focusing on the talent, skills, and experience of an individual as a key indicator of the present and future value that they will bring to any organization.
In establishing a network, we are able to forge genuine relationships with talent and gain valuable insight that allows us to understand what motivates them. This data allows us to give strategic advice to our partners as they go through the process of managing their human capital.
See our jobs and apply here: https://kprecruitinggroup.catsone.com/careers/7781-General