Monday, March 17, 2014

A bash script to restart Apache if the free memory is too low

"Self, I'm having terrible problems with Apache eating up all of my memory and crashing MySQL or the entire server itself, and I know reloading Apache frees up the memory every time. I know that the right thing to do is to properly troubleshoot what's causing Apache to consume all this memory, but I'm in a pinch and out of time and need something quick and dirty to keep things running. Why not write a bash script to check how much memory is free and reload Apache if it gets below a certain threshold? And have it log what it does each time it runs?"

As a sysadmin, we strongly discourage things like this when other people do them. This is a terrible script. You shouldn't be running this script. But sometimes you need to do terrible things and run terrible scripts like this. So here it is:

#!/bin/bash

FREE_MEMORY=$(free -m | grep Mem | cut -d' ' -f 28)

THRESHOLD=150

LOG=low-memory-fixer.log

APACHE_STATUS=$(service httpd status)

if [ "$FREE_MEMORY" -lt "$THRESHOLD" ]
        then
                echo -e "Time to reload Apache. Current memory value is $FREE_MEMORY, lower than the current threshold of $THRESHOLD. Reloading Apache. Date is $(date)." >> $LOG
                service httpd reload
                echo -e "Apache status is $APACHE_STATUS. Date is $(date)." >> $LOG

        else
                echo -e "Memory is still good. Current memory value is $FREE_MEMORY, higher than the current threshold of $THRESHOLD. Nothing to do here! Date is $(date)." >> $LOG
fi


You'll want to create the log file in the same directory the script is running in - "touch low-memory-fixer.log" is sufficient. If you want to set up a cronjob that runs this script every 10 minutes, run "crontab -e" as the user this being run by and add a line like this: "*/10 * * * * bash /path/to/the/script/goes/here/low-memory-fixer.sh".

This script was written in five minutes. This script is one of the worst solutions I've seen in my professional career. Don't run this script. All humor aside, if you're running this script, you really need to dig into Apache and find out what's going on, or to get someone who can do that to look at your server.

Friday, March 7, 2014

How do I find the creation date for an AMI?

So, you have a bunch of AMIs on your AWS cloud that are just taking up space and you want to clear out the older ones. You go to sort the AMIs by date and find that there's nothing in the AWS console that lets you do that. So you look at the API and the command line client, going through a bunch of JSON and realize that there's nothing that gives a creation date for an AMI. "I just want to know when my AMI was created!" you say.

At the time of this posting, there's no way to find that. However, what you can find is a list of all your snapshots, the date they were made and the AMI they were based on. So if you've made at least one snapshot of the AMI at some point, you logically know that the AMI was created on or before that date.

Now how do we get a list of our snapshots, sorted by date or AMI? Like this:

aws ec2 describe-snapshots --owner-ids MyAWSaccountIDgoesHere --query 'Snapshots[*].[Description,StartTime]' --output text | sort -k8n,8 | cut -d' ' -f5,7 > ami-snapshot-dates_sorted-by-date.txt


If you want to get a list sorted by AMI, you'd do this:
aws ec2 describe-snapshots --owner-ids MyAWSaccountIDgoesHere --query 'Snapshots[*].[Description,StartTime]' --output text | sort -k5n,5 | cut -d' ' -f5,7 > ami-snapshot-dates_sorted-by-ami.txt


Hint: If you're feeling lazy and don't want to look up your AWS account ID, (presuming that you're the root user), you can use "--owner self" instead of "--owner-ids MyAWSaccountIDgoesHere"