Streaming Architect/Data Engineer - Kafka

St. Louis, MO

Posted: 08/06/2019 Job Category: Software Dev. - Java and Open Source Job Number: 140124

Job Description


Are you looking for a new and challenging opportunity? Do you have experience with Big Data? Technology Partners can help you make your next big career move a reality.

Technology Partners is currently seeking a talented person for a Streaming Architect / Big Data Engineer.

Summary:
We are developing a next-generation analytics architecture to support our growing business. As part of the Data Architecture and Data Design team, you will be challenged daily to find innovative solutions to improve the speed and consumption of our information. This will also bring you into contact with a diverse array of stakeholders ranging from IT to our front line business organizations, and you will be working on defining an architecture platform that establishes the foundational layer for the copmany analytics for years to come.

What You Will Be Doing:
You will utilize your software engineering skills including Java, Spark, Python, Scala to Analyze disparate, complex systems and collaboratively design new products and services. You will architect, implement and deliver data pipelines using Big Data and Streaming Technologies like Kafka, Attunity, Nifi, Pentaho, Goldengate and other data transfer/logging/ETL technologies. You will also architect and develop scalable and reliable distributed data replication strategies to distribute data between on-premise and cloud repositories. You will leverage AWS or Azure to provide innovative solutions for data movement performance and scale, along with necessary storage and access methods. You will convert business and technical requirements to working prototypes and then deployable solutions in production and lower environments. You will also create and document architecture plans for near term and long term solution deployment. You will design and implement high-performance, scalable data solutions leveraging scheduling and automation, complete with metrics to identify issues. You will also provide best in class security in everything you do and ensure adherence to the company security tools and policies.

What We Are Looking For:
  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases structures (Hadoop, MongoDB, DataVault)
  • Experience building and optimizing data pipelines, architectures and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets and recommending transformations to optimal architectures.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • A successful history of manipulating, processing and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
  • Strong project management and organizational skills.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
  • Candidates should also have 3+ years of experience using the following software/tools:
  • Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
  • Experience with AWS/Azure cloud services
  • Experience with stream-processing systems: Kafka, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: Python, Java, etc.
If this position sounds like you, WE SHOULD TALK!

We realize our people are our most valuable asset, that is why we offer the following benefits:
  • Health, Dental, and Vision insurance
  • 401(k) retirement plan
  • Long and Short-Term disability
  • Life insurance
  • Direct deposit
  • Referral program
Your better future is ready, and we want to put the right tools in your hands to get you there. Let's go!
Looking for more opportunities with Technology Partners?

Check out jobs.Technologypartners.Net.
Technology Partners is an Equal Opportunity Employer. Technology Partners does not discriminate on the basis of race, color, religion, sex, national origin, age, disability or any other characteristic protected by applicable state or federal civil rights laws.

Meet Your Recruiter

Nicole Field

Apply Online
Apply with LinkedIn Apply with Facebook Apply with Twitter

Send an email reminder to:

Refer A Friend

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.

Technology Partners has amazed me. To this day, our partnership with them is one of the best business decisions we’ve made.

Alice Lawrence, CEO of Jordan Lawrence

Find your voice with our digital media and video solutions today.

Learn More