logs
Extract from 5GB log file the logs
# Get the first line of the logs for a given date grep -n 'at 2009-11-25' production.web01.log | head -n 1 && tellme # TODO : check optimization with 'time' command: Stop reading a file after NUM matching lines. # Get the last line / TODO How to start grepping from a specific line? grep -n 'at 2009-11-25' production.web01.log | tail -n 1 && tellme # Extract the log file lines for the given date. File is a about 45 Mb sed -n '64332368,65003422p' production.web01.log > 2009_11_25_web01.log && tellme sed -n '56465993,57141087p' production.web02.log > 2009_11_25_web02.log && tellme
It takes ages!
Suggestion to make it faster
Write a small shell script which extract the log files for a given date & merge logs from web01 & web02 ?
Import production log files into a database on dev01 and use a nice web interface to search. question do you know a tool for that job?
http://blog.philburrows.com/articles/2009/09/28/rails-logging-with-mongodb/ + http://github.com/tobi/clarity?