cordump.com

Ever have that need to change one little thing in a bunch of files? You know after you accidentally misspelled your Boss’s name as “Marvin Blowhard” on every page of the Company website.
Now you could open up each file in vi and find every reference to Mr. Blowhard to Joe Smith, even using search & replace
s/Marvin Blowhard/Joe Smith/ inside vi get’s kind of tedious after a while. That joke doesn’t seem so funny now, does it?
Now consider a little BASH and sed foo

for i in `ls *.html`; do sed "s:Marvin Blowhard:Joe Smith:g" $i >temp;mv temp $i; done

All nice and done so you can get back to planning your next BOFH moment, like sending him in the into the server room with a flash light to look for breaks in the fiber runs.

As a follow up to the previous tip on getting notified. What do you do when you have an already running process that you would like have notify you when it completes?
So you kicked off this update that was only supposed to take a few minutes but now an hour later it is still running and you are tired of having to check up on it, wouldn’t it be nice if you could just get a nice little e-mail letting you know when its done. Well, this little one liner just might make your day, ok maybe your hour…

while [ $(ps auxww | grep -i SOMEPROCESS | grep -v grep > /dev/null; echo $?) -eq 0 ]; do echo -n “.”; sleep 5; done; echo `/bin/date` | mail -s “SOMEPROCESS has completed” me@example.com

This little ditty will print a “.” every 5 seconds while the process is running, then when it completes will shoot you a nice little e-mail with the time it completed in the body. Now just don’t forget to the processes in screen so they can keep running even if you get disconnected.

Sometime you want to run a command or a series of commands in a loop to say shutdown all the nfs daemons via their init scripts. Now we could do them one at a time like:

$ /etc/init.d/nfs stop
Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down NFS services: [ OK ]
$ /etc/init.d/nfslock stop
Stopping NFS locking: [ OK ]
Stopping NFS statd: [ OK ]

Or we could use a bash for loop with a little copy & paste

$ for i in /etc/init.d/nfs /etc/init.d/nfslock
> do
> $i stop
>done
Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down NFS services: [ OK ]
Stopping NFS statd: [ OK ]

Or just completely automate it and get it down quick and easy

$ for i in `ls /etc/init.d/nf*`
> do
> $1 stop
>done
Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down NFS services: [ OK ]
Stopping NFS statd: [ OK ]

It’s that simple and you saved your self a bunch of typing.

In Linux you can look in /proc/$PID , where $PID is your Process ID of course, to find out a great deal about currently running processes. It is laid out in a hierarchical format with file names that are fairly intuitive, Read on for an example from a running Nagios instance.

(continue reading…)

Just edit the appropriate lines in the script and run it from the crontab on your Mysql Master Server.
This will optimize every table in every database on the local machine then send an email detailing which database/tables it optimized with timestamps so you can gauge how long each table took
A word of warning when using optimize it will write-lock the table it is currently working on and it could take several hours for a large table to be optimized. This should only be run when DOWNTIME can be tolerated as most applications will hang waiting for the write lock to disappear.
Read more about optimize Here to determine if the risk/rewards is worth it for you

(continue reading…)

Cool vision for Data Center Automation using Virtual Machine’s to automagically adjust services to properly distribute load ensuring a minimal waste of CPU resourceshttp://www.oreillynet.com/sysadmin/blog/2007/04/data_center_automation.html.
Sounds like they have some pretty smart people over there at Novell work on some pretty cool stuff.

I have been using Cfengine for a couple a years now and while it is a quantum leap above the old method of manual work custom scripts it still leaves alot to be desired. So I figured I would jot down my reasons for for looking for a replacement.

1. The syntax is complex and not intuitive
   
2. There is no way to extend Cfengine to handle new cases
   
3. No easy way to ensure that X is in place before doing Y
   
4. The answer for most configuration changes is use editfiles or copy to copy a static file in place. Both options have scaling issues and make minor differences in configurations more painful than the should be.
   
5. Not easy to reuse previous configurations to create something new without excessive duplication.
   
6. Stagnant, Cfengine is maintained by one person who does not generally except patches. This make the kind of vibrant and community driven advancement represented in most Open Source solutions noticeably missing.
   
7. Sloooooow transferring files over the network To be fair Cfengine is not really designed for this
   

I have been testing out puppet and trying to recreate some of the configs that we have cfengine maintaining for us currently and I have run into a what I think is well not a bug, but a documentation issue probably. When I set a variable inside a node then attempt to use that variable inside an imported manifest it does not exists unless I wrap it in a class within the imported file then include the class in the node.
Since that explanation even confuses me a little let me show an example.

node jim {
$splat = ’splat’
include splat_test
include splat_file_test
}
class splat_test {
file{”/tmp/destory_all_$splat”:
ensure => present
}
}
class splat_file_test {
import “splat_file.pp”
}

site.pp

file{”/tmp/file_$splat”:
ensure => present
}

splat_file.pp
This generates the files /tmp/destroy_all_splat and /tmp/file_, what we want is /tmp/file_splat not /tmp/file_ . Now with the addition of 3 simple lines I can make it work.

node jim {
$splat = ’splat’
include splat_test
include splat_file_test
}
class splat_test {
file{”/tmp/destory_all_$splat”:
ensure => present
}
}
class splat_file_test {
import “splat_file.pp”
include splatter
}

site.pp

class splatter {
file{”/tmp/file_$splat”:
ensure => present
}
}

splat_file.pp
This works but I still think the first example should work as well since I am importing the splat_file.pp into the splat_file_test class there shouldn’t be any functional difference right?
Perhaps I am just not wrapping my head around this right and the correct method is to always have everything in a manifest wrapped in a class and not expect the act of importing the manifest into a class to convey the “classyness” to it, but I spent 2 days trying to figure this out and using the imports into a class just seemed to be the intuitive way to do it.

After setting up the initial yum repository we decide that we wanted a 2 more repositories to reflect third-party RPM’s we received from vendors and RPM’s we created internally. These are steps we took to add these 2 repositories for CentOS both i386 and x86_64 like before.

1. Create the new directories for the Vendor repository

   % mkdir -p /var/yum/centos/4/vendor/i386
   % mkdir -p /var/yum/centos/4/vendor/x86_64

2. Create the new directories for the Local repository

   % mkdir -p /var/yum/centos/4/local/i386
   % mkdir -p /var/yum/centos/4/local/x86_64

3. Now add the following to your CentOS-Base.repo and push it out using your ConfigurationManagement System of choice.

   [vendor]
   name=CentOS-$releasever - Vendor baseurl=http://yum.example.com/centos/$releasever/vendor/$basearch
   gpgcheck=0
   enabled=1
   [local]
   name=CentOS-$releasever - Local
   baseurl=http://yum.example.com/centos/$releasever/local/$basearch/
   gpgcheck=0
   enabled=1

4. And for the impatient, flush the yum cache on the clients to make it see the new repositories a little quicker.

   % yum clean all

Setting up a local yum repository is actually fairly simple, here is a simple plan for setting up a yum repository for CentOS 4.4 i386(32bit) and x86_64(64bit).

1. Create the repository directory structure on a machine with plenty of disk space and a web server

   % mkdir -p /var/yum/centos/4/os/i386/base
   % mkdir -p /var/yum/centos/4/os/x86_64/base
   % mkdir -p /var/yum/centos/4/updates/i386
   % mkdir -p /var/yum/centos/4/updates/x86_64

2. Add a VirtualHost entry to you web server pointing to /var/yum as the DocumentRoot

   &lt VirtualHost XXX.XXX.XXX.XXX:80 &gt
   ServerName yum.example.com
   ServerAlias yum
   ServerAlias XXX.XXX.XXX.XXX
   DocumentRoot /var/yum
   ErrorLog /var/httpd/yum.log
   IndexOptions FancyIndexing VersionSort FoldersFirst NameWidth=*
   &lt /VirtualHost &gt
   &lt Directory /var/yum &gt
   Options +Indexes
   order deny,allow
   allow from all
   &lt /Directory &gt

   Example using Apache 2.0.X
   

3. Locate a CentOS mirror to download from the file from.
   
4. Download the Centos Base files

   % wget -r -nH –cut-dirs=4 -P /var/yum/centos/4/os/i386/base http://isoredirect.centos.org/centos/4/os/i386/
   % wget -r -nH –cut-dirs=4 -P /var/yum/centos/4/os/x86_64/base http://isoredirect.centos.org/centos/4/os/x86_64/

5. Download the Centos Updates files

   % wget -r -nH –cut-dirs=4 -P /var/yum/centos/4/updates/i386 http://isoredirect.centos.org/centos/4/updates/i386/
   % wget -r -nH –cut-dirs=4 -P /var/yum/centos/4/updates/ix86_64 http://isoredirect.centos.org/centos/4/updates//x86_64/

6. Ensure the createrepo scripts are installed on the yum server

   % yum install createrepo

7. Use createrepo to create the required repository manifests

   % createrepo /var/yum/centos/4/os/i386/base
   % createrepo /var/yum/centos/4/os/x86_64/base
   % createrepo /var/yum/centos/4/updates/i386
   % createrepo /var/yum/centos/4/updates/x86_64

8. And finally replace the /etc/yum.repos.d/CentOS-Base.repo on the client machines with the following.

   [base]
   name=CentOS-$releasever - Base
   baseurl=http://yum.example.com/centos/$releasever/os/$basearch/base
   gpgcheck=1
   gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-centos4
   enabled=1
   #released updates
   [update]
   name=CentOS-$releasever - Updates
   baseurl=http://yum.example.com/centos/$releasever/updates/$basearch/
   gpgcheck=1
   gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-centos4
   enabled=1