Big Data Developer

Key skills

What you’ll do

• The role involves designing and maintaining scalable distributed data processing systems and pipelines using Big Data technologies. You will collaborate with stakeholders to translate business requirements into technical solutions for handling massive volumes of data.

What they’re looking for

• Candidates must be proficient in Java, Scala, or Python and have a strong understanding of the Hadoop ecosystem and distributed computing frameworks. Experience with data ingestion tools, SQL/NoSQL databases, and data modeling principles is required.

Job description

As a Big Data Developer, your role is to design, develop, and maintain large-scale data processing systems and applications that handle massive volumes of structured and unstructured data. You will work with cutting-edge technologies to extract insights and provide valuable data-driven solutions to support business needs. Key Responsibilities: 1. Design, develop, and implement scalable and distributed data processing systems using Big Data technologies such as Hadoop, Apache Spark, or Apache Flink. 2. Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions. 3. Build and optimize data pipelines and workflows for efficient data ingestion, storage, processing, and analysis. 4. Perform data modeling and design to ensure data quality, consistency, and accuracy. 5. Write complex queries and transformations to extract, transform, and load data from various sources into the Big Data ecosystem. 6. Develop and maintain data integration solutions, including data connectors, APIs, and data ingestion frameworks. Required Skills and Qualifications: - Proficiency in programming languages such as Java, Scala, or Python. - Strong understanding of Big Data concepts, including Hadoop ecosystem components (HDFS, MapReduce, YARN) and distributed computing frameworks (Apache Spark, Apache Flink). - Experience with data ingestion, processing, and storage technologies like Apache Kafka, Apache NiFi, or cloud-based solutions (AWS S3, Azure Data Lake). - Knowledge of data modeling, data warehousing, and database design principles. - Familiarity with SQL and NoSQL databases. - Experience with data visualization tools and frameworks (e.g., Tableau, Power BI) is a plus. - Understanding of data security and privacy principles.

To apply and see any additional details, continue to the original posting on the employer’s site.