Description :
Job Location: Delhi
Experience: 5-10 years
Salary: Not Disclosed
Education: Graduate/Any Post Graduate
Job Description
Data Ingestion: Design and implement data ingestion processes to acquire data from various sources, including databases, sensors, logs, and external APIs.
Data Storage: Create and manage data storage solutions that can handle massive datasets, often using technologies like Hadoop Distributed File System (HDFS) and NoSQL databases.
Data Processing: Develop data processing pipelines and ETL (Extract, Transform, Load) workflows to clean, transform, and aggregate data for analysis.
Big Data Technologies: Work with big data technologies and frameworks like Apache Hadoop, Spark, Kafka, and others to process and analyze data at scale.
Data Modeling: Build data models and schemas to support efficient data retrieval and query performance.
Data Security: Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.
Scalability and Performance: Optimize data infrastructure for scalability, performance, and reliability, and troubleshoot issues as they arise.
Monitoring and Maintenance: Set up monitoring tools and systems to track the health and performance of data infrastructure components.
Data Integration: Integrate data from various sources, both structured and unstructured, to create a unified view of the data.
Documentation: Maintain documentation on data engineering processes, workflows, and infrastructure configurations.
Collaboration: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver the necessary data solutions.
Automation: Implement automation where possible to streamline data workflows and reduce manual intervention.
Qualifications and Skills:
Education: A bachelor's or master's degree in computer science, data engineering, or a related field is often required.
Programming Skills: Proficiency in programming languages such as Python, Java, Scala, and scripting languages for data processing.
Big Data Technologies: In-depth knowledge of big data technologies and frameworks,
Prakhar Softwares Solutions is a CMMI Level 3 , ISO 9001:2015, ISO 27001:2013 certified company dealing in multiple projects including software development, Staffing Management, Recruitment Process Outsourcing, E-governance. We have 10 offices across India and working for various e-governance projects of National Importance.