Job Responsibilities
- Design, develop, and maintain robust data platform infrastructure, databases, and data pipelines to ensure high performance and scalability.
- Design, develop, and maintain Extract, Transform, Load (ETL) processes to facilitate data movement and transformation across various systems.
- Ensure data availability, reliability, and scalability through best practices in data engineering and infrastructure management.
- Collaborate with other teams, including software engineering, data science, and product management, to integrate and optimize data systems.
- Build and maintain efficient, scalable, and stable data pipelines for large-scale data processing.
- Understand the evolving needs and challenges of platform users and develop strong tools and platform features to address these needs.
Minimum Qualifications
Job Requirements:
- 1-2 years of experience in data engineering or software development with a focus on data infrastructure.
- Proficiency in Java, Scala, and Python.
- Experience with version control systems like Git and build tools like Maven.
- Containerization and orchestration tools such as Docker and Kubernetes.
- Experience with big data technologies such as Apache Flink, Kafka, Flume, Spark, and Apache Airflow.
- Hands-on experience with relational and non-relational databases such as MySQL and MongoDB.
- Experience in handling big data and data warehousing in cloud environments.
- Strong problem-solving skills and the ability to manage multiple tasks and projects simultaneously.
- Excellent communication and teamwork skills, with the ability to work collaboratively in a cross-functional environment.
- Eagerness to learn new technologies and continuously improve technical skills.
- Familiarity with DevOps practices and automation tools is preferable.