Big Data Engineer

Spotflock is an AI products company to enable a new class of industry specific data interpretation and machine learning applications for businesses and consumers both. Spotflock is headquartered in Milpitas, California with its Product Engineering & Innovation in Hyderabad.

Job Description

Spotflock is hiring engineers to scale up its product development. You will need to have strong programming skills, a deep understanding of deep learning (algorithms & frameworks), modular design, distributed computing and specifically distributed deep learning; as well as excellent communication and planning skills. You and other engineers in this team will help advance Spotflock’s capacity to build and deploy leading solutions for a broad range of computer vision and machine learning related applications.


  • Write software to interact with HDFS and MapReduce & ingest data into Hadoop
  • Assess requirements and evaluate existing solutions
  • Build, operate, monitor, and troubleshoot Hadoop infrastructure
  • Develop tools and libraries, and maintain processes for other engineers to access data and write MapReduce programs
  • Evaluate and use hosted solutions on AWS / Google Cloud / Azure
  • Write scalable and maintainable ETLs
  • Understand Hadoop’s security mechanisms and implement Hadoop security


  • With a solid foundation in Computer Science and strong competencies in Mathematics or similar field; Master’s degree is a plus
  • Must have at least 4+ years of experience in building applications, solutions and products based on analytics.

Required Skills

  • Mandatory Core Java, SpringBoot expertise
  • Mandatory hands-on experience on Hadoop, HIVE & Spark
  • Must have hands-on experience on Hadoop distributions
  • Must have worked with one or more computation frameworks such as Spark
  • Experience with HBase, Kafka, ZooKeeper, Cassandra, Elasticsearch
  • Experience with non-relational & relational databases (SQL, MySQL, NoSQL, Hadoop, MongoDB, etc.)
  • Excellent programming & designing skills
  • Ability to design solutions independently based on the high-level architecture
  • A creative and innovative approach to problem-solving

Preferred Skills

  • Strong knowledge with Docker
  • Knowledge of version controls like GIT and SVN (preferred)

Soft Skills

  • Excellent verbal and written communication skills
  • Good time management and organizational skills
  • Strong ability to collaborate with a global development team

Candidate should be

  • Agile, Ambitious, Hardworking, Quick Learner, and a Critical Thinker
  • Salary should not be the main motive
  • Should be exited for a startup environment
  • Damn good with their skill set

Apply Today! Join the Flock!