bash

Remote Tail

fjavierm, , Leave a comment

This article is just a quick code snipped for bash shell to allow us to tail log files from a remote endpoint.

Nowadays, we are building plenty of microservices and the common pattern is to aggregate them using tools like Kibana to store and search them with the help of some correlation ids. Despite, this is a very good solution, sometimes we do not need anything that fancy.

In Spring Actuator, we can find the endpoint “/logfile” that it has proven a lot of times to be pretty useful. This endpoint allows us to recover the log file from the server. We can download it or just check it on the browser. The problem is when this logfile reach a size that our browser can not manage properly. We can use tools like “wget” to download the log file and analyse it locally but it seems absurd to download the whole file every time we want an update.

I have a different proposal. With a few lines of shell scripting, we can write a snipped to tail the log file into a local file, and we can use “less” to monitor it or perform searches on the file.

#!/bin/bash

#
# Check if the given server support HTTP range header
# param 1: url
#
function check_ranges_support() {
  ret=`curl -s -I -X HEAD $1 | grep "Accept-Ranges: bytes"`

  if [ -z "$ret" ]; then
    echo "Ranges are nor supported by the server"
    exit 1
  fi
}

#
# Recovers the total length of the given file
# param 1: url
#
function get_length() {
  ret=`curl -s -I -X HEAD $1 | awk '/Content-Length:/ {print $2}'`
  echo $ret | sed 's/[^0-9]*//g'
}

#
# Print the requested part of the remote file
# param 1: url
# param 2: off
# param 3: len
# param 4: output file
#
function print_to_logfile() {
  curl --header "Range: bytes=$2-$3" -s $1 >> $4
}

#
# Clean the previous log file
#
function clean_logfile() {
  rm -f $1
}

# call validation
if [ $# -lt 1 ]; then
  echo "Syntax: remote-tail.sh <URL> [<logfile name>]"
  exit 1
fi

url=$1
offset=0
logfile=tmplog

if [ $# -eq 2 ]; then
  logfile=$2
fi

check_ranges_support $url
clean_logfile $logfile

len=`get_length $url`
off=$((len - offset))

until [ "$off" -gt "$len" ]; do
  len=`get_length $url`

  if [ "$off" -eq "$len" ]; then
    sleep 5 # we refresh every 5 seconds if no changes
  else
    sleep 1 # we refresh every second to not hammer too much the server
    print_to_logfile $url $off $len $logfile
  fi

  off=$len
done

We can use it with:

./remote-tail.sh https://server/logfile

I hope it helps.

Remote Tail