We have kafka-connect on actual environments, passing data from jdbc to kafka. I need to test some of our code locally, I'd like to setup kafka-connect locally. But I cannot find anyone who can setup kafka-connect from scratch. I managed to cheat some docker compose file(below), and setup plugins directory with kafka-connect-jdbc-10.9.0.jar and resolve all dependency hell, and created some source connector, but it does not read. It constantly says it scanned few thousands tables, picked one to continue but is seems it never selects any data. Any my configuration files are mostly copied from working environment.

Nothing is sent to kafka, valid db table, valid timestamp and incrementing column data, nothing in connect-offsets, no errors because almost NO logs.

To begin with: does anyone know how to force kafka-connect to print what does it actually do? Almost all I can see is:

INFO Filtered tables: [my table]

So it is connected, it seems my table but for unbeknownst reason it does nothing.

docker compose:

    services:
  kafka-connect:
    network_mode: host
#    image: quay.io/strimzi/kafka:0.36.1-kafka-3.5.1
    image: confluentinc/cp-kafka-connect:latest

    container_name: kafka-connect

    environment:
      - CONNECT_LOG4J_DEBUG=true
      - CONNECT_LOG4J_LOGGERS=io.confluent.connect.jdbc.source=DEBUG,org.apache.kafka.connect.runtime.WorkerSourceTask=DEBUG,io.confluent.connect.jdbc.source.JdbcSourceTask=DEBUG
      - CONNECT_BOOTSTRAP_SERVERS=localhost:9092
      - CONNECT_REST_PORT=8083
      - CONNECT_GROUP_ID=connect-cluster
      - CONNECT_REST_ADVERTISED_HOST_NAME=localhost
      - CONNECT_CONFIG_STORAGE_TOPIC=connect-configs
      - CONNECT_OFFSET_STORAGE_TOPIC=connect-offsets
      - CONNECT_STATUS_STORAGE_TOPIC=connect-status
      - CONNECT_KEY_CONVERTER=org.apache.kafka.connect.storage.StringConverter
      - CONNECT_VALUE_CONVERTER=org.apache.kafka.connect.converters.ByteArrayConverter
      - CONNECT_PLUGIN_PATH=/opt/kafka/plugins
    ports:
      - "8083:8083"
    volumes:
      - ./plugins:/opt/kafka/plugins
      - /tmp/kafka-connect-logs:/opt/kafka/logs  

connector:

{

"name": "oracle-jdbc-source", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector", "connection.url": "jdbc:oracle:thin:@localhost:1521/XEPDB1", "connection.user": "...", "connection.password": "...", "table.include.list": "...", "table.types": "VIEW", "mode": "timestamp+incrementing", "timestamp.columns.mapping": "regex:[...]", "incrementing.column.mapping": "regex:[...]", "timestamp.delay.interval.ms": 300,

"transforms": "...


"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"validate.non.null": "true",
"poll.interval.ms": "2000",
"table.poll.interval.ms": "200000",
"batch.max.rows": "20000"

} }