Project Metamorphosis: Unveiling the next-gen event streaming platform.Learn More
Déploiement d'Apache Kafka

How the New Confluent Hub Makes Finding Connectors Easier Than Ever

Confluent Hub launched in 2018 as a place to discover and share Apache Kafka® and Confluent Platform plugins. Users have found the site a much better place to discover useful code than seeking out individual git repos. These plugins include connectors, Single Message Transforms (SMTs), and converters for Kafka Connect. At launch, we had a humble inventory of about a dozen plugins. Now, there are over 140 available, including 100+ that are fully supported by Confluent and our partners!

Since its launch, we’ve looked for ways to improve Confluent Hub, and today we’re happy to introduce a complete frontend redesign, based on user feedback. In particular, we’ve tried to make it much easier to parse plugins across dimensions that matter most to developers.

Finding what you need

Confluent Hub

The most notable update to Confluent Hub is the addition of a detailed plugin taxonomy that lets you filter options across a number of key plugin dimensions.

Type de plugin

Determining the plugin type you need is likely your first concern as a developer. The plugin types on Confluent Hub are as follows:

  • Sink connector: sink connectors consume messages from Kafka topics and export them to external systems (e.g., Amazon S3 or MySQL).
  • Source connector: source connectors consume data from external systems (e.g., Amazon S3 or MySQL) and write them to Kafka topics.
  • Transform: SMTs are applied to messages as they flow through Connect. SMTs transform inbound messages after a source connector has produced them but before they are written to Kafka. Alternatively, SMTs transform outbound messages before they are sent to a sink connector.
  • Converter: converters change the format of data from bytes to a Connect internal data format, and vice versa. They are necessary in order for a Kafka Connect deployment to support a particular data format when writing to or reading from Kafka.

Assistance aux entreprises

Depending on where you’re doing your development, you may or may not need access to enterprise-level support. You might even be quite adventurous with your own projects. Lots of companies we talk to, however, aren’t comfortable putting something into production without the ability to get expert support on the phone. To help both types of users navigate the available plugins, Confluent Hub lets you filter by enterprise support type:

  • Confluent supported: Confluent enterprise support for these plugins is included as part of a Confluent Platform subscription. (You can contact us to learn more.)
  • Partner supported: enterprise support is available from the partner who maintains the plugin.
  • None: Many open source plugins fall in this category. No enterprise support is available, although varying levels of community support may be available. An open source Git repo and the Confluent Community Slack are often good places to look.

Vérification

Similar to enterprise support, you may or may not be developing in an environment that requires everything to be ready for “enterprise scale” from day zero. It’s impossible to point to a single benchmark that quantifies this, but these verification levels are our attempt to identify at least those plugins that are known to be built for high performance and stability. These categories include:

  • Confluent Built: built, tested, and supported by Confluent.
  • Confluent Tested: community plugins, which are tested and supported by Confluent.
  • Verified Gold: partner built and supported integrations that meet the highest level of functional compatibility with the Confluent Platform ecosystem. (See the Confluent Verified Integration FAQs for further details.)
  • Verified Standard: partner built and supported integrations that meet a standard level of functional compatibility with the Confluent Platform ecosystem. (See the Confluent Verified Integration FAQs for further details.)
  • None: community contributions that have not been verified or tested by Confluent. This doesn’t mean they’re not good—we just haven’t had a chance to rigorously test them.

Licence

This one is pretty straightforward:

  • Commercial: requires a Confluent Platform or partner subscription. Confluent supported plugins can be used for free on a single broker cluster (Developer License) or free for 30 days on an unlimited number of brokers (Evaluation License).
  • Free: available for free through an open source or community license.

Disponible dans Confluent Cloud

Our fully managed Apache Kafka as a service product, Confluent Cloud, offers some of these connectors, fully managed, on leading public clouds like Google Cloud Platform (GCP), Microsoft Azure, and Amazon Web Services (AWS).

Let us know what you think or submit a plugin

That’s a quick overview of what’s changed in the Confluent Hub frontend. We’re always looking for ways to improve the product and serve the community better, and one of the key ways we do this is by adding new open source and partner connectors to Confluent Hub. If you would like to get your code in front of more developers, or if you’d just like to give some feedback, send us an email at confluent-hub@confluent.io. We look forward to hearing from you!

Ethan Ruhe is a product manager at Confluent, where he leads the development of the Confluent CLI and formerly the Confluent Hub.

Did you like this blog post? Share it now

Subscribe to the Confluent blog

More Articles Like This

Learning All About Wi-Fi Data with Apache Kafka and Friends

Recently, I’ve been looking at what’s possible with streams of Wi-Fi packet capture (pcap) data. I was prompted after initially setting up my Raspberry Pi to capture pcap data and […]

Building a Clickstream Dashboard Application with ksqlDB and Elasticsearch

Using a powerful, event-driven application can help you unlock insights contained in the event streams of your business. Before we get into the technology, let’s go over some questions you […]

Webify Event Streams Using the Kafka Connect HTTP Sink Connector

The goal of this post is to illustrate PUSH to web from Apache Kafka® with a hands-on example. Our business users are always wanting their data faster so they can […]

Sign Up Now

Recevez jusqu'à 50 $ US de réduction sur votre facture chaque mois calendaire pour le premier trimestre.

Nouvelles inscriptions uniquement.

By clicking “sign up” above you understand we will process your personal information in accordance with our Politique de confidentialité.

En cliquant sur « Inscription » ci-dessus, vous acceptez les termes du/de la Conditions d'utilisation et de recevoir occasionnellement des e-mails publicitaires de la part de Confluent. Vous comprenez également que nous traiterons vos informations personnelles conformément à notre Politique de confidentialité.

Gratuit à vie sur un seul broker Kafka
i

Le logiciel permettra une utilisation illimitée dans le temps de fonctionnalités commerciales sur un seul broker Kafka. Après l'ajout d'un second broker, un compteur de 30 jours démarrera automatiquement sur les fonctionnalités commerciales. Celui-ci ne pourra pas être réinitialisé en revenant à un seul broker.

Sélectionnez un type de déploiement
Manual Deployment
  • tar
  • zip
  • deb
  • rpm
  • docker
ou
Déploiement automatique
  • kubernetes
  • ansible

By clicking "download free" above you understand we will process your personal information in accordance with our Politique de confidentialité.

En cliquant sur « Téléchargement gratuit » ci-dessus, vous acceptez la Contrat de licence Confluent et de recevoir occasionnellement des e-mails publicitaires de la part de Confluent. Vous acceptez également que vos renseignements personnels soient traitées conformément à notre Politique de confidentialité.

Ce site Web utilise des cookies afin d'améliorer l'expérience utilisateur et analyser les performances et le trafic sur notre site Web. Nous partageons également des informations concernant votre utilisation de notre site avec nos partenaires publicitaires, analytiques et de réseaux sociaux.