[MSA]Kafka Connect Start Jdbc Connect Example
Kafka Connect Source - 데이터를 보내는쪽
Kafka Connect Sink - 데이터를 받는쪽
Kafka Connect 설치
Confluent: Data in Motion
Confluent is building the foundational platform for data in motion so any organization can innovate and win in a digital-first world.
www.confluent.io
- curl -O http://packages.confluent.io/archive/6.1/confluent-community-6.1.0.tar.gz
- tar xvf confluent-community-6.1.0.tar.gz
Kafka JDBC Connect 설치
https://docs.confluent.io/kafka-connectors/jdbc/current/sink-connector/overview.html
JDBC Sink Connector for Confluent Platform | Confluent Documentation
The Kafka Connect JDBC Sink connector allows you to export data from Apache Kafka® topics to any relational database with a JDBC driver. This connector can support a wide variety of databases. The connector polls data from Kafka to write to the database b
docs.confluent.io
Kafka Connect에 Jdbc 설정 정보 추가
- /etc/kafka/connect-distributed.properties 에 kafka jdbc connect path 추가
- plugin.pah=[Kafka JDBC connect 파일 위치]
plugin.path=/Users/puk0806/Kafka/kafka_jdbc_connect/confluentinc-kafka-connect-jdbc-10.7.1/lib- ./share/java/kafka 폴더에 사용 드라이버 복사
- 사용 jdbc 드라이버 복사 ex) mariadb-java-client-3.0.10.jar
Kafka Connect 서버 실행
- ./bin/connect-distributed ./etc/kafka/connect-distributed.properties
$KAFKA_CONNECT_HOME ./bin/connect-distributed ./etc/kafka/connect-distributed.properties
Kafka Connect 서버가 정상적으로 실행되면 Kafka서버에 topic이 등록된다.
- connect-ocnfigs
- connect-offsets
- connect-status
