Big Data Engineer
This job is no longer active.
View similar jobs.
POST DATE 9/12/2016
END DATE 10/9/2016
iSystems Group, inc.
JOB DESCRIPTIONJob Description:
iSystems Group has an exciting opportunity for a self-motivated and accomplished Big Data Engineer to join our team. Our customer uses their legacy data warehouse as the foundation for a variety of applications supporting intelligence analysis, threat tiering, targeting, entity resolution, and big data analytics. While this warehouse and the associated data pipeline adequately support current production applications, our customer is undertaking a major technology refresh initiative in order to support several new operational analytic priorities, and lay the foundation for next-generation analytics. In collaboration with the team, customer principals, and the solutions architect, the ideal candidate will use their knowledge of the Hadoop ecosystem and other distributed computing technologies to improve the data pipeline and data stores -- and exceed the customer's ingest, ETL, search, alerting, and data visualization goals. In this role you will:Provide design, integration, and operational support for distributed computing and storage technologies to ingest and process extremely large data sources using SQL, NoSQL, Hadoop, and Elastic ecosystem tools.Assist with the replacement of an existing commercial search system with a combined solution of Hadoop & Elasticsearch. Assist with the development and implementation of the solution using Elasticsearch 2.x and Hortonworks HDP 2.3 to improve data ingest, indexing, and search workflows.Provide day to day administrative of Hadoop, to include installation, maintenance, and configuration.Perform troubleshooting and issue resolution for the customer, users, and technical leadership.Participate as an engineer on an agile development team, writing, testing, and deploying code to implement changes to existing data pipelines or construct new ones. Provide ad-hoc data analysis to support end users. Basic Qualifications:Bachelor's degree with at least 2 years of experience building and operating big data processing pipelines and/or query solutions.Proficient understanding of distributed computing principles.Demonstrated ability to manage a Hadoop cluster, and associated services.Proficiency with Hadoop, MapReduce, and HDFS.Experience with big data ETL frameworks and techniques such as Flume, Oozie, or Sqoop.Experience with Big data interactive query technologies such as Spark, Impala, Hive, or Presto.Experience with the Elastic ecosystem - elasticsearch, kibana, marvel, watcher, and shield.Programming experience, ideally in Python or Java. Excellent written and oral communication skills. Desired Skills:Experience with loading and processing data on Hortonworks Hadoop clusters.Experience tuning Elasticsearch for ingest and query performance.Experience securing Elasticsearch and other big data technologies for the intelligence community.Experience with messaging/queuing technologies like RabbitMQ or Kafka.Experience with version control platforms (git, svn, etc.)Prior experience in support of law enforcement or intelligence related big data solution development. Security Clearance:Must currently possess an active US government Top Secret clearance with the ability to obtain and maintain SCI access within a reasonable, customer-mandated time frame. Must be willing and able to pass a counterintelligence (CI) polygraph examination
iSystems Group, Inc, is a Service-Disabled Veteran-Owned, PMI registered consultant company. We provide SharePoint Development and training, portfolio /program / project management services including risk management, earned value management, schedule management, training and project management office (PMO) support. Additionally, iSystems Group is experienced in systems engineering including COTS integration, big data analytics and enterprise architecture.