wiki‎ > ‎

logs

Extract from 5GB log file the logs 



# Get the first line of the logs for a given date
grep -n 'at 2009-11-25' production.web01.log | head -n 1 && tellme

# TODO : check optimization with 'time' command: Stop  reading  a  file  after  NUM matching lines.

# Get the last line  / TODO How to start grepping from a specific line?
grep -n 'at 2009-11-25' production.web01.log | tail -n 1 && tellme

# Extract the log file lines for the given date. File is a about 45 Mb
sed -n '64332368,65003422p' production.web01.log > 2009_11_25_web01.log && tellme

sed -n '56465993,57141087p' production.web02.log > 2009_11_25_web02.log && tellme

It takes ages!

Suggestion to make it faster

  1. Write a small shell script which extract the log files for a given date & merge logs from web01 & web02 ?
  2. Import production log files into a database on dev01 and use a nice web interface to search. question do you know a tool for that job?
  3. http://blog.philburrows.com/articles/2009/09/28/rails-logging-with-mongodb/ + http://github.com/tobi/clarity?
Comments