Project Metamorphosis: Unveiling the next-gen event streaming platformLearn More

Partitioning

Changing and Optimizing Partition Count

KSQL automatically creates Kafka topics to hold its results, but you may still have to think about topic sizing. In cases where you don’t want to use the cluster-wide default number of partitions, you can specify the number of partitions you want to use.

Directions

In any KSQL query that will create a new Kafka topic, simply add the PARTITIONS parameter to the WITH clause. The new topic will now have that number of partitions.

1. Consider a Kafka topic called clickstream-raw. We don’t even need to know how many partitions it has, but we do know we need a new topic that has five partitions.

2. Create a STREAM from that partition:

CREATE STREAM clickstream-raw WITH (KAFKA_TOPIC=’clickstream-raw’,VALUE_FORMAT=’JSON’);


3. Create a new stream with the desired number of partitions. The newly created topic will be partitioned the way we specify.

CREATE STREAM clickstream_new WITH (PARTITIONS=5) AS SELECT * from clickstream_raw;
< Back to the Stream Processing Cookbook

Ce site Web utilise des cookies afin d'améliorer l'expérience utilisateur et analyser les performances et le trafic sur notre site Web. Nous partageons également des informations concernant votre utilisation de notre site avec nos partenaires publicitaires, analytiques et de réseaux sociaux.