RsaProtectedConfigurationProvider: Not recommended for children under 5

So just an intro before getting into the nitty gritty, these are my experiences personally when dealing with the RSAProtectedConfigurationProvider combination of tools in order to get sections of my web.config for an enterprise web application encrypted and some of the hassles along the way. As with anything cryptosecurity-related, there is some cursing involved.

Create an RSA Key

The first step to getting RSA encryption into your web application’s web.config, you need to create a common key that can be shared across servers and/or development environments. To do this, open up a Visual Studio command prompt (or navigate a regular command prompt over to your %WINDIR%\Microsoft.NET\Framework(64)\{insert .net version}\ directory) as an administrator and type in

aspnet_regiis -pc "ProjectKeyNameHere" -exp

This will create a key for you in the %PROGRAMDATA%\Microsoft\Crypto\RSA\MachineKeys directory

MachineKeys folder

Each key is given its own unique, seemingly nonsensical filename. However, the naming convention is actually the calculated key hash based off of the keyname itself, followed by the machine key GUID, and separated by an underscore. The key hash is calculated by an interesting algorithm when it gets imported into the keystore, and the machine GUID can simply be found in the registry under HKLM:\SOFTWARE\Microsoft\Cryptography.

Quick Trivia Fact: The key “c2319c42033a5ca7f44e731bfd3fa2b5_…” is the key that is used by IIS to encrypt the MetaBase. Delete or mess with this key and the IIS server on the machine will cease to function.

So even though you can modify permissions to keys and even delete them directly from this directory, DO NOT MANUALLY EDIT PERMISSIONS OR DELETE KEYS FROM THIS DIRECTORY OR YOU WILL SCREW THINGS UP. Use the aspnet_regiis tool to manage your keys instead. Trust me on this (or take the risk and have fun troubleshooting later).

Exporting the key

Now, in order to share the key to other developers and/or machines, you will need to export the key into an XML file. This can be done by running the following command:

aspnet_regiis -px "ProjectKeyNameHere" "C:\location\for\key\KeyName.xml" -pri

This will give you a plain text XML file which describes the private key. As with anything that has “private” in its name, treat this file with respect and proper security procedures (aka, don’t throw this up on a public share, make sure you keep track of where it goes, etc).

Installing existing (XML) key to machine(s)

Once you send off the XML file in a secure fashion to another developer or machine, you can then run the following command on that machine in order to import the key into that machine’s RSA key store:

aspnet_regiis -pi "ProjectKeyNameHere" "C:\location\of\key\KeyName.xml"

Keep in mind, the “ProjectKeyNameHere” name has to be the same throughout the entire process, as this is the name of the key that will be identified in the web.config later on.

At this point, any administrative user on the box can now use the key to encrypt/decrypt information. No other (regular/non-admin) user will have access to read the key — this includes the user that IIS AppPools run under. The names or groups of the users that have this permission change based on environment, but is usually “NT AUTHORITY\NETWORK SERVICE”. In order to grant permissions for other users to read the key, run the following command:

aspnet_regiis -pa "ProjectKeyNameHere" "UsersOrGroups"

If you’re using the impersonation setting in IIS for your web application, read the “Impersonation permission issues” section below.

Adding key to web.config

In order for IIS to know which key to use for encryption/decryption, we first have to turn off the default RSAConfigurationProvider in the web.config and then add in our own key that we created. To do this, simply add the following to your web.config (directly under the root configuration node):

<configProtectedData>
    <providers>
      <remove name="RSAProtectedConfigurationProvider" />
      <add keyContainerName="ProjectKeyNameHere" useMachineContainer="true" description="Uses RsaCryptoServiceProvider to encrypt and decrypt" name="RsaProtectedConfigurationProvider" type="System.Configuration.RsaProtectedConfigurationProvider, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </providers>
</configProtectedData>

Change the assembly version number for System.Configuration.RsaProtectedConfigurationProvider to whichever version of .NET you are using for your application (e.g. 2.0.0.0 for 2.0/3.5).

After saving the file, you will now be able to use aspnet_regiis to encrypt and decrypt sections of your web.config file as you see fit.

To encrypt (in this example, the connectionStrings node):

aspnet_regiis -pef "connectionStrings" "c:\path\to\web.config\"

To decrypt

aspnet_regiis -pdf "connectionStrings" "c:\path\to\web.config\"

Note that I supply the path to the config file, but I don’t use the filename itself in the parameter (e.g. Don’t use c:\path\to\web.config\web.config). Also note that you cannot encrypt the entire config file, only nodes under the root configuration node (it would be a bad idea to encrypt the entire file as the part to know how to decrypt the file would also be encrypted).

You will also be able to use tools such as the Enterprise Library Config Tool in order to do this for you automatically should you so desire.

Reference: ASP.NET IIS Registration Tool http://msdn.microsoft.com/en-us/library/k6h9cz8h(v=vs.80).aspx

Running publish commands to encrypt config

Personally, I find it easiest when I don’t encrypt any data in the web.config file until I’m ready to publish (so if I have to change a connectionString, I won’t have to manually decrypt, change, and then re-encrypt each time). This process will vary based on which method you use to publish your web application. I will only be covering the file system publish method, but make sure to check out the reference links below for compiling your own methods for publishing.

For MSBuild functionality, you can either store these commands in the main .csproj file if you want the action to occur globally across all the publish profiles that you have, or you can set it individually in each .publxml file (recommended).

Now, what I ended up doing for the file system publish was capturing an event after the build and web.config transform but before the files were moved to the filesystem location that was specified. Under the root Project node in the .pubxml file, I added the following:

<Target Name="BeforePublish" AfterTargets="CopyAllFilesToSingleFolderForPackage">
    <Exec Command="$(MSBuildBinPath)\aspnet_regiis.exe -pef &quot;connectionStrings&quot; &quot;$(MSBuildProjectDirectory)\$(IntermediateOutputPath)Package\PackageTmp&quot;" />
</Target>

A few things to note here. The AfterTargets variable was a pain in the butt to find (and apparently only works well with msbuild 4.0). It is literally not documented anywhere officially (or nowhere I can find on the internet at the moment). But this will capture the moment before the files are moved to the destination file system location. The variables in the Exec Command node (such as $(MSBuildBinPath)) are all listed under the MSBuild documentation, which is what I ended up using. If you use quotes inside of the command, make sure to use “&quot” as it is an XML file, after all. If you have multiple sections that you want to encrypt, you will have to create an Exec Command node for each one.

Reference: How to: Extend the Visual Studio Build Process http://msdn.microsoft.com/en-us/library/ms366724.aspx
Reference: MSBuild Concepts http://msdn.microsoft.com/en-us/library/vstudio/dd637714.aspx

Impersonation permission issues

One of the first problems I ran across was the fact that when IIS has the impersonation flag set to true for the web application, it will use the impersonated account to access the RSA key in order to decrypt the configuration file. Unfortunately, there is no way around this if you MUST have impersonation: you will have to grant access to the key for each user who is using the application. An easy way to do this is to grant permission to the group needing access to the application. Unfortunately, if you have something like Windows Authentication turned on, that group may end up being “Domain Users” or goodness forbid “Everyone”:

aspnet_regiis -pa "ProjectKeyNameHere" "Everyone"

This sort of sucks in terms of security, I know. Right now, there really isn’t a better way of dealing with it without rewriting your application and turning off impersonation.

Cannot read key error

Check to make sure that permissions on the key are setup correctly. If it helps to debug, give permissions to everyone to see if the error goes away. If it does, find out which user/group actually needs read access to the key. If it doesn’t, here’s Google :).

Safe Handle error

“Failed to decrypt using provider ‘RsaProtectedConfigurationProvider’. Error message from the provider: Safe handle has been closed”

So this one is a pain, but again, it has everything to do with permissions. If you did everything right, and read the note above about not manually changing the permissions on the MachineKeys folder and using the aspnet_regiis tool instead, you shouldn’t have to worry about this.

On the other hand, if one of your co-workers decided that it was a good idea to change permissions on that folder without realizing the consequences it may produce, well, you have this cryptic error message to deal with.

First and foremost, check the permissions on the MachineKeys folder (%ProgramData%\Microsoft\Crypto\RSA\MachineKeys). You should see this little lock icon near the folder:

11-6-2013 3-31-10 PM

That means that general users can’t read the contents of the folder. This is a good thing. Now check your permissions for the “Everyone” group:

11-6-2013 2-59-56 PM

The !!!ONLY!!! permissions that the “Everyone” group should have checked are the “Special Permissions”. Digging a little deeper, we find out what those special permissions are:

11-6-2013 2-59-22 PM

Specifically, we have the following from the list provided by Microsoft:

  • List Folder/Read Data
  • Read Attributes
  • Read Extended Attributes
  • Create Files/Write Data
  • Create Folders/Append Data
  • Write Attributes
  • Write Extended Attributes
  • Read Permissions

and the following quote: “The default permissions on the folder may be misleading when you attempt to determine the minimum permissions that are necessary for proper installation and the accessing of certificates.”

If you grant any further permissions to the “Everyone” group, such as general read access, write access, or full control, you will see that not only will you break the functionality of the existing keys in the folder, any additional key you install to the folder will be improperly installed, as aspnet_regiis will complain about the safe handle. The former will start working once you fix the permissions on the folder (for the most part). The latter will not be fixed as they were improperly installed and will be stuck in a state of limbo. The aspnet_regiis tool will not even be able to delete those keys (it will complain about…the safe handle being closed). The only way you will be able to delete them is by manually going into the MachineKeys folder, finding out which key you have to delete (either by last modified time or by the key hash – this you can get by installing the key to another machine and comparing the first part of the filename for both). Then, once manually deleted, you can use the aspnet_regiis tool to reinstall the key.

Again, I don’t have to remind you to be very meticulous and careful when rooting around and modifying things in this folder as it can break much more than just your web application.

Reference: Default permissions for the MachineKeys folders http://support.microsoft.com/kb/278381

Posted in Microsoft, Programming, Security | No Comments »

Adding MVC4 to a project: The operation could not be completed. Not implemented

While experimenting around with adding the MVC framework onto an existing C# Webforms project, I came across an interesting error when trying to add the MVC3/MVC4 project type GUID into the existing csproj file for the project (I tried both, seeing if there was something wrong with that). I came up with the following error when first opening the solution with the project (this was using Visual Studio 2012):

Visual Studio 2012 MVC4 Error

“The operation could not be completed. Not implemented”

That was very…descriptive. I went ahead and repaired the MVC4 package thinking that might fix it, but it was no good. Made sure Visual Studio was up to date – it was. I was pretty stumped. I took another look at the csproj file. Before adding MVC4 to the project, I had the following:

<ProjectTypeGuids>
{349c5851-65df-11da-9384-00065b846f21};
{fae04ec0-301f-11d3-bf4b-00c04f79efbc}
</ProjectTypeGuids>

So it’s your basic web application project {349c5851-65df-11da-9384-00065b846f21} combined with a C# project {fae04ec0-301f-11d3-bf4b-00c04f79efbc}. So, I went ahead and added in the MVC4 type {E53F8FEA-EAE0-44A6-8774-FFD645390401} to get the following code:

<ProjectTypeGuids>
{349c5851-65df-11da-9384-00065b846f21};
{fae04ec0-301f-11d3-bf4b-00c04f79efbc};
{E53F8FEA-EAE0-44A6-8774-FFD645390401}
</ProjectTypeGuids>

Seems like it should work. I compared the above to a newly created MVC project and apparently, the problem exists with the order that the project type GUIDs are listed in. I had added the MVC4 project type GUID after everything, whereas it needs to go before the others, like so:

<ProjectTypeGuids>
{E53F8FEA-EAE0-44A6-8774-FFD645390401};
{349c5851-65df-11da-9384-00065b846f21};
{fae04ec0-301f-11d3-bf4b-00c04f79efbc}
</ProjectTypeGuids>

Problem is, I can’t seem to find any sort of documentation for this behavior. It really shouldn’t matter what order the project type GUIDs are in, right? I can understand from a developer standpoint that MVC depends on the Web application framework which depends on the C# project – and in that way, the project can correctly identify the build order – but I thought that this was something that Visual Studio would have been able to figure out more easily (without having to list the project type GUIDs in the proper order). Anyways, after figuring that out, MVC worked with the existing Webforms application like a champ.

Posted in Microsoft, Programming | 3 Comments »

System Center Configuration Manager (SCCM) 2012: Client PKI and Subordinate CA woes

EDIT 2013-12-30 Before you remove your Root CA from configuration, as suggested in this guide, note that it will most likely break OSD / PXE booting for SCCM, potentially along with other things. According to this technet post, the bug is happening for people who seem to use a 3rd party cert which might not be compatible with SCCM? I put a question mark there since I’m still not sure what the cause might be and Microsoft seems hesitant on fixing this bug. I am using a root CA that was created with OpenSSL and use the root CA to sign my subordinate CA which is being used in my enterprise CA (in other words, my Windows Enterprise CA is subordinate to the one I created using OpenSSL). If it helps you get running, and you’re not using things like PXE and OSD, then this guide might provide a workaround. Otherwise, the workaround is not to use PKI, which sucks.


When trying to setup SCCM on my network, I came upon some trouble getting the secure communication working between the server and the client (PKI settings and HTTPS communication). I finally ended up figuring out the issue after a few good hours of debugging and log hunting, and so hopefully this information might help someone else out.

I had set up SCCM on one of my servers, configured it to use PKI communication with the clients. I had a root CA and also a subordinate CA (the subordinate was issuing certificates to the computers). All that being set up, I decided to deploy the client using the automatic deployment method through SCCM. Looked at the client and realized that it was failing to communicate with the server correctly. Alright then, so lets go ahead and check the logs in C:\Windows\CCM\Logs :

In CcmMessaging.log:

<![LOG[Begin searching client certificates based on Certificate Issuers]LOG]!><time="18:39:06.247+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="3868" file="ccmcert.cpp:3759">
<![LOG[Certificate Issuer 1 [E=austrianalex@gmail.com; CN=AustrianAlex CA; OU=Certificate Authority; O=AustrianAlex; L=Spokane; S=Washington; C=US]]LOG]!><time="18:39:06.247+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="3868" file="ccmcert.cpp:3775">
<![LOG[Finding certificate by issuer chain returned error 80092004]LOG]!><time="18:39:06.247+420" date="10-11-2012" component="CcmMessaging" context="" type="2" thread="3868" file="ccmcert.cpp:3884">
<![LOG[Completed searching client certificates based on Certificate Issuers]LOG]!><time="18:39:06.247+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="3868" file="ccmcert.cpp:3918">
<![LOG[Unable to find any Certificate based on Certificate Issuers]LOG]!><time="18:39:06.247+420" date="10-11-2012" component="CcmMessaging" context="" type="2" thread="3868" file="ccmcert.cpp:3995">
<![LOG[Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
DateTime = "20121012013906.253000+000";
HRESULT = "0x87d00215";
ProcessID = 684;
ThreadID = 3868;
};
]LOG]!><time="18:39:06.253+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="3868" file="event.cpp:729">
<![LOG[Post to https://MYCCMSERVER/ccm_system/request failed with 0x87d00231.]LOG]!><time="18:39:06.255+420" date="10-11-2012" component="CcmMessaging" context="" type="2" thread="3868" file="messagequeueproc_outgoing.cpp:430">

Finding certificate by issuer chain returned error 80092004. I had specified my root CA in the SCCM console, and I thought that was enough. But certificate chain? My subordinate CA is issuing these certs, but if it trusts the root, it should also trust the subordinate, right? I decided to go and add the subordinate CA certificate into the site settings as well.

Client Settings for SCCM

So now I had the Root CA and subordinate CA specified in the settings. Pushed out the client again. This time I got a different error.

In CcmMessaging.log:

<![LOG[Begin searching client certificates based on Certificate Issuers]LOG]!><time="19:09:24.100+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="ccmcert.cpp:3759">
<![LOG[Certificate Issuer 1 [E=austrianalex@gmail.com; CN=AustrianAlex CA; OU=Certificate Authority; O=AustrianAlex; L=Spokane; S=Washington; C=US]]LOG]!><time="19:09:24.100+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="ccmcert.cpp:3775">
<![LOG[Certificate Issuer 2 [CN=AustrianAlex Windows CA]]LOG]!><time="19:09:24.100+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="ccmcert.cpp:3775">
<![LOG[Skipping Certificate [Thumbprint 9F5CAC8D5572421BA2EEAB2BDC2AAFB8A41365FC] issued to 'TheClient' as root is 'AustrianAlex CA']LOG]!><time="19:09:24.101+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="ccmcert.cpp:3877">
<![LOG[Completed searching client certificates based on Certificate Issuers]LOG]!><time="19:09:24.101+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="ccmcert.cpp:3918">
<![LOG[Unable to find any Certificate based on Certificate Issuers]LOG]!><time="19:09:24.101+420" date="10-11-2012" component="CcmMessaging" context="" type="2" thread="5832" file="ccmcert.cpp:3995">
<![LOG[Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
DateTime = "20121012020924.107000+000";
HRESULT = "0x87d00215";
ProcessID = 1224;
ThreadID = 5832;
};
]LOG]!><time="19:09:24.107+420" date="10-11-2012" component="CcmMessaging" context="" type="1" thread="5832" file="event.cpp:729">
<![LOG[Post to https://MYCCMSERVER/ccm_system/request failed with 0x87d00231.]LOG]!><time="19:09:24.109+420" date="10-11-2012" component="CcmMessaging" context="" type="2" thread="5832" file="messagequeueproc_outgoing.cpp:430">

Skipping Certificate [Thumbprint 9F5CAC8D5572421BA2EEAB2BDC2AAFB8A41365FC] issued to ‘TheClient’ as root is ‘AustrianAlex CA’. Apparently, the chain issue went away and now it is ignoring the certificate entirely. Double checked and made sure that the root CA matched the issued certificate and it did. This is when I realized that since the CA certificates were already installed as Trusted Root Certificates (through Group Policy in my case) on the client machine, there might be a conflict with those I was specifying in the SCCM site settings. They were both the same Root CA, however, the SCCM client’s logic decided to…skip any certificates that matched the CA…even though they were the same? I still don’t get the behavior, but whatever, the solution turned out to be to remove the specified CA’s from the SCCM site settings:
SCCM Site Settings Client Connection

Also, I had to make sure to uninstall the client from the computer before doing an automatic push install to purge the settings (otherwise, doing another automatic push install will continue to use the specified certificates, getting the same message as above). I did this by executing the following on the client computer:

cd c:\Windows\ccmsetup
ccmsetup.exe /uninstall

ccmsetup.exe /uninstall

After that, a quick push install and the client started communicating using PKI.
Client install SCCM 2012

TL;DR:

If you have the root CA already installed on a client computer, do not specify a root CA in the SCCM console site settings, or you’re going to have a bad time.

Posted in Microsoft | 3 Comments »

Effective Social Engineering: Another DEFCON 20 skytalk

The way a lot of security pen-testers do social engineering calls by phone – “Hey, I’m <so and so> from IT. You have a virus. I need your password.” Rinse, repeat, and occasionally, you will get one or two people who will actually give you their password. The other 99% of the time, employees have been trained not to give out their password and rightfully so. “Well, we got a username and password, so it still counts…” But you also wasted time calling 50 other people before you got the credentials, which most likely put the company on high alert.

At the DEFCON 20 Skytalk, the presenter with the alias “Tanks4U” talked about the inefficiencies and the problems with using the commonly employed methods of social engineering, and the companies who hire security guys to call <x> amount of people for statistical purposes – specifically if it’s an all or nothing goal like getting credentials. You didn’t get them to give you the password? Hang up. They challenged or questioned you? Hang up. Try again with some other person. And it totally compromises the whole point to social engineering.

“You’re not supposed to tell them to jump in a river. You are supposed to take them to the river and convince them that they are on fire.” Social engineering isn’t just about a cold call to a person, it involves an interaction with the person on the other side and observation of person to get the reaction you need. In effective social engineering, you need to know the following:

  • Know your goals – what details do you want to obtain? Username and password could be one, but what if you want the building layout? When does your cleaning staff come in? Who are the managers? Any details on the business that you can obtain can help you further in obtaining access to their business. Write them out on a piece of paper in a checklist. Make a scoring system. Ask them for things like the last four digits of their phone number, how their badges look like, last four digits of their SSN, etc.
  • Know your target – it helps to observe how people react when faced with certain situations for this one. Go to a coffee shop before hand and sit there for a few hours. Watch how people interact with one another. You don’t have to be good at knowing how to social engineer people, just good enough to figure out how to small talk with others. Ask them questions about the weather or talk about hardships (ever had a flat tire or kids?), or engage in other commonalities that people share. Make a connection.
  • Know yourself – sure you might know how to talk to others, but can you actually do it when it comes time? How will you react? Most people might get nervous and that is simply something you get over after you have some practice with social engineering. Public speaking also helps (try performing in a play and get your acting skills in shape). Or just sit in front of a mirror and practice your technique. Ultimately, though, it will come down to analyzing how you react in a live situation with other people. Remember the coffee shop? Go make small talk with those people that you were just observing.

In terms of the actual call or contact, performing the following steps will be key to establishing a good baseline for social engineering (or not! depending on the situation. Adjust accordingly with what you know above):

  • Initiation – “Hi, my name is <so and so>. Is this <target>?”
  • Pleasantries – “How are you doing? The weather sucks right now. How’s work? Busy today?” Establish a connection to your target through commonalities.
  • Establish a pretext – who are you pretending to be? What is your purpose? Establish authority.
  • Emotional injection – this is the start of the RIP phase (reactionary identity preservation). Ask them universal questions like, “Did you use Google today?” “Are you aware that we have an employee handbook? Did you read it?” Note that most of these are yes/no questions and that all of them should have the same response, no matter who you call (thus universal or almost rhetorical). This gets your target into an almost defensive stage and gives them a warm fuzzy feeling when they answer correctly – reassure them that they won the argument and play with their emotions. Get their responses (make them angry, worried, or comfortable).
  • Team up – once you get them to emotionally connect with you on the problem that you have (oh you have a virus infection), work towards getting your goal by teaming up and fixing the problem. “Hey, so let’s just take care of this virus infection quickly” or “let’s figure something out”. Prepare to get your goal.
  • Request an action – have them go to a website, give them their credentials, or open up a command shell and start typing things – whatever your goal is, ask them to do it now. If they refuse, don’t give up and try to obtain your goal some other way (Plan B, C, D…).
  • Closing – Once you accomplish your goal, make them feel like the accomplished something as well. Give them reinforcement saying something like, “You did everything right, good job” or, depending on other emotional responses, give them other feedback like warnings, “Don’t open bad websites” or “let’s just keep this between us; I’d rather not have to tell your boss about this so that you don’t get in trouble.” This is a good phase to make sure that they don’t go and tell others about the incident so that you have more opportunities to contact others at the business.
Posted in Security | No Comments »

VMware Workstation Automatic Suspend/Start and Backup Scripts for Ubuntu

I had previously listed some scripts that I use for VMware Workstation management on the Intrinium blog, but have found a few annoyances with running them. So, I’m re-releasing the scripts in their modified form.

The objective performed by these scripts are as follow:

  • They provide a simple way to suspend all virtual machines on host shutdown and resume only a select few on host startup.
  • They provide a way to check if certain virtual machines are running. If they are not running, the script will send an alert (and attempt to start the machine).
  • Backups will occur on a per-machine basis (that can be set to be different than the running list) and will only suspend one machine at a time for backups, starting them back up once the backup is finished. The backup will be sent to a samba share in my example, but this can be configured to be anything you can cp/scp to.

I’ve also modified the scripts to be a little more flexible in terms of modularity (user set directories) and included some error checks. Other than that, you know the routine – test it before you run it and don’t blame me if you delete everything.

First, the startup script I’ve dropped as /etc/init.d/vmwaresuspend (and of course, run “update-rc.d vmwaresuspend defaults”). This script runs on startup and shutdown of the host, making sure to suspend all running virtual machines and start up only machines on the machine list when the host starts back up:

#!/bin/bash

### BEGIN INIT INFO
# Provides: vmwaresuspend
# Required-Start:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Required-Stop: 0 1 6
### END INIT INFO
USER="media" #the user to run VMware as
MLIST="/media/raid/vmware/machine.list" #the machine list for machines to START on startup (see example below)

case "$1" in

start)
if [ $# -gt 1 ]
then
        su ${USER} -c "vmrun start '$2' nogui"
else
        while read VM;
        do
                echo "Starting $VM"
                su ${USER} -c "vmrun start '$VM' nogui"
        done < $MLIST
fi
exit 0
;;

stop)
if [ $# -gt 1 ]
then
        su ${USER} -c "vmrun suspend '$2'"
else
        vmrun list|grep vmx$|while read VM
        do
                echo "Suspending $VM"
                su ${USER} -c "vmrun suspend '$VM'"
        done
fi
exit 0
;;

restart)
# Nothing to be done for restart
exit 0
;;
esac
exit 0

The machine list is simply the full path to the vmx file for the virtual machine, one per line. Example:

/home/vmware/vm1/vm1.vmx
/home/vmware/vm2/vm2.vmx

Next, the script that checks to see if VMs are running. I set the list to be the same one as the startup script above, but you can change it to a different one if you feel like it. I saved this in /root/cron/checkVMs.sh and updated cron to have it run this script every 5 minutes:

#!/bin/bash
MLIST="/media/raid/vmware/machine.list"
LOCKFILE="/media/raid/vmware/backup.lock"

if [ -f ${LOCKFILE} ]; then #checks for the lock file made by backup script
        exit 0
fi
# vmrun list | tail -n+2 | awk -F/ '{print $NF}' | rev | cut -d\. -f2- | rev ...or we can just ps grep :)
while read VM;
do
        if ! ps ax | grep -v grep | grep "$VM" > /dev/null
        then
                echo "$VM is down!"
                #include mail -s here if you don't receive output for cron jobs.
                #include "vmrun start $VM" if you want to start the VM automatically if down
                # - it might not work due to other factors (rebuilding vmware modules, etc)
        fi
done < $MLIST

Lastly, the backup script. Like with the checking script, you can set a different machine list for it if you are wanting to backup other machines. Do note, this script does try to suspend the machine before backup and then start it after the backup completes – if your machine was stopped and you prefer it to remain stopped after the backup, you will have to change the logic of the script. Otherwise, it will start up. I saved this as /root/cron/backupVMs.sh and set cron to run it every week on Sunday night:

#!/bin/bash
LOCKFILE="/media/raid/vmware/backup.lock"
BACKUP_DIR="/home/media/vmwarebackup"
SMB_DIR="//yourShareServer/vmwarebackup"
MLIST="/media/raid/vmware/machine.list"
SUSPEND="/etc/init.d/vmwaresuspend"
CREDFILE="/root/cron/smbcred"

if [ -f $LOCKFILE ]; then
        echo "A backup is currently in progress"
        exit 1
fi
touch $LOCKFILE

mount -t smbfs -o credentials=$CREDFILE $SMB_DIR $BACKUP_DIR

if mountpoint -q $BACKUP_DIR #did it mount? If not, bad things could happen to your tiny 60GB SSD drive
then
        find $BACKUP_DIR -mtime +30 | xargs rm -rf #remove backups over 30 days old
        datetime=$(date '+%d_%m_%y_%H_%M')
        mkdir $BACKUP_DIR/${datetime}
        cd $BACKUP_DIR/${datetime}
        while read VM;
        do
                mkdir $(basename ${VM})
                ${SUSPEND} stop $VM
                cp -R $(dirname ${VM})/* ./$(basename ${VM})/
                ${SUSPEND} start $VM
        done < $MLIST
        umount -l $BACKUP_DIR
else
        echo "Samba share failed to mount: $BACKUP_DIR"
fi
rm $LOCKFILE

Modify to your hearts content.

Posted in Linux, Programming | No Comments »