Big Data/Data Engineer

2 - 6 Years

Job Description

Data Engineer will work on the collecting, storing, processing, and analysing of huge sets of data
The primary focus will be on implementing data ingestion and pipeline process, feature design and data streaming
Gather and process raw data at scale (including writing scripts, write SQL queries, etc.) to build features that will be used in modelling
Responsible for infrastructure that provides insight from raw dataand handles diverse sources of data seamlessly
Work closely with our Engineering and MachineLearning team to integrate amazing innovations and algorithms into our production systems
Process unstructured data into a form suitable for analysis and then do the analysis
Support business decisions with ad hoc analysis as needed
Create data catalog, data flow diagrams and interprets dataresults to business users

MongoDB, SparkML, Sklearn, python, Scala, Jupyter Notebook, ETL, Data Analysis, Data Mining, Data Modelling, Job Scheduling

Proficient understanding of distributed computing principles
Experience with integration of data from multiple data sources
Experience processing large amounts of structured and unstructured data
Knowledge of various ETL techniques and frameworks
Good understanding in datamining, machine learning, natural language processing, or information retrieval
Proficiency with Big Data / Spark Architecture
Experience with Spark programming in python or Scala, Jupyter Notebook
Experience with ML toolkits, such as SparkML, Sklearn or H2O
Experience with NoSQL databases, such as MongoDB
Experience with building stream-processing systems, using solutions such as Spark-Streaming, Flume, Kafka is a plus

Salary: Not Disclosed by Recruiter

Industry:IT-Software / Software Services

Functional Area:IT Software - Application Programming, Maintenance

Role Category:Programming & Design

Role:Software Developer


Desired Candidate Profile

Please refer to the Job description above

Company Profile

Netcore Solutions Pvt Ltd
View Contact Details+

Contact Company:Netcore Solutions Pvt Ltd