Streaming Data from the Universe with Apache Kafka
You might think that data collection in astronomy consists of a lone astronomer pointing a telescope at a single object in a static sky. While that may be true in […]
Deploying Kafka Streams and KSQL with Gradle – Part 2: Managing KSQL Implementations
In part 1, we discussed an event streaming architecture that we implemented for a customer using Apache Kafka®, KSQL from Confluent, and Kafka Streams. Now in part 2, we’ll discuss […]
Deploying Kafka Streams and KSQL with Gradle – Part 1: Overview and Motivation
Red Pill Analytics was recently engaged by a Fortune 500 e-commerce and wholesale company that is transforming the way they manage inventory. Traditionally, this company has used only a few […]
Analyse en profondeur de Kafka Connect – gestion des erreurs et files d'attente de lettre morte
Kafka Connect appartient à Apache Kafka® et c'est une structure performante qui permet de construire des pipelines de streaming en continu entre Kafka et d'autres technologies. Il peut être utilisé pour les données de transmission en continu dans Kafka [...]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 3
We saw in the earlier articles (part 1, part 2) in this series how to use the Kafka Connect API to build out a very simple, but powerful and scalable, streaming […]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 2
In the previous article in this blog series I showed how easy it is to stream data out of a database into Apache Kafka®, using the Kafka Connect API. I […]
The Simplest Useful Kafka Connect Data Pipeline in the World…or Thereabouts – Part 1
This short series of articles is going to show you how to stream data from a database (MySQL) into Apache Kafka® and from Kafka into both a text file and Elasticsearch—all […]
Building a Real-Time Streaming ETL Pipeline in 20 Minutes
There has been a lot of talk recently that traditional ETL is dead. In the traditional ETL paradigm, data warehouses were king, ETL jobs were batch-driven, everything talked to everything […]
Creating a Data Pipeline with the Kafka Connect API – from Architecture to Operations
This is a guest blog from Alexandra Wang, Software Engineer, Pandora Media. You can connect with her on LinkedIn. Also special thanks to Lawrence Weikum and Stu Thompson as contributors of the work in this post. […]
Streaming data from Oracle using Oracle GoldenGate and the Connect API in Kafka
This is a guest blog from Robin Moffatt. Robin Moffatt is Head of R&D (Europe) at Rittman Mead, and an Oracle ACE. His particular interests are analytics, systems architecture, administration, and […]