Shell script to monitor kafka lag value

Posted Jun 28, 20201 min read

Hand-written a shell script to monitor kafka lag value. I used to write it in python before, and it feels more troublesome. A shell version is written here, you can use it directly.

cd /usr/share/kafka/kafka_2.11-2.4.1/|| exit 1
lag=$(./bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my_group 2>/dev/null|grep -v GROUP|awk'NR>1{num+= $6}END{print num}')
echo "$lag"
if ["$lag" -gt 10]; then
  echo "lag value is too large"
  #Or mail or send_ding_msg, set yourself
fi

The following is the implementation.

Finally, this script can be added to the crontab scheduled task. I currently execute it every 10 minutes, and I have not encountered the accumulation of messages.