Last week Karl Fosaaen described in his blog the various trials and tribulations we went through at a hardware level in building a dedicated GPU cracking server. This week I will be doing a complete walkthrough for installing all the software that we use on our box. This includes the operating system, video drivers, oclHashcat-plus, and John the Ripper. Because we have AMD video cards, the driver installation and compiling John the Ripper sections will be tailored for AMD, sorry Nvidia users.
Installing the OS:
For an operating system, Linux and Windows are going to be the way to go. For a headless server however, Linux is the best way to go. The only downside with Linux is that driver support among video cards, especially AMD, is somewhat lacking to its Windows counterpart. However, the good news is that both AMD and Nvidia have been increasing their support for Linux drivers in recent years.
Any Linux distribution will do, but for our server, we opted for Ubuntu 12.10 64-Bit server edition to do the most minimal setup. Much of the information for the next few sections is from the hashcat wiki.
To start off, download the Ubuntu 12.10 server edition ISO from Ubuntu. We don’t have a cd drive on our server, so we had to copy the ISO to a flash drive. YUMI and UNetbootin make this process painless on Windows and Linux, respectfully. Otherwise, the ISO can be burned to a disc.
Boot up the Ubuntu image, choose your language, and select Install Ubuntu Server.
Navigate through the installation options and select your preferences. For most people, the defaults should be sufficient. Then create your user when the dialog comes up. When the installation reaches the “Partition Disks” section, either manually set them up (if you know what you’re doing) or just use the “Guided – use entire disk” option. We choose not to use LVM on our box, but the option is up to you.
After you are done partitioning your hard drive, write the changes to the disc. If you have an HTTP proxy, enter the information when the dialog appears. If not, then just continue. Next, select if you would like to have automatic updates enabled. We opted not to, but it’s entirely up to you. When the software selection appears, select OpenSSH server by navigating to it with the arrow keys and pressing spacebar to select the option.
None of the other packages are required unless you need them. Press enter to install the software. When the installation is finished, install GRUB to the master boot record and reboot. You should now be booted into your new Ubuntu server!
Setting Up Ubuntu:
Before we install the video drivers, we have to setup our Ubuntu server with X11. This is because the AMD drivers require X11 to interact with video cards to obtain fan speeds and GPU temps, which are very important to know when cracking away.
To begin, ssh into your server and update Ubuntu with the following command:
sudo apt-get update && sudo apt-get upgrade
After Ubuntu has updated, we will need to install a minimal X11 environment that our user can automatically login to when the server is rebooted. This is to ensure that the xserver will always be running and in turn allow continuous cracking without any hiccups.
To keep it simple, a light weight window manager is recommend. Openbox, fluxbox, and blackbox are three simple light weight window managers that we can use. You are by no means restricted to a window manager. If you want gnome, xfce, or kde, those can be installed too. For this installation, we will install fluxbox with lightdm as the display manager. To install these, run the following command:
This should install all the necessary packages for an X11 environment to run. Now that we have an X11 environment installed, we need to let applications from the console know which display we are using. To do this, we set the DISPLAY variable to our current display. The format for the DISPLAY variable is hostname:display. For a local instance, the hostname can be omitted. The default display is usually going to be 0. Run the command below to set your current display to 0.
export DISPLAY=:0
Add the above command to your bashrc to make it persistent whenever your user logs in. I have run into many issues because I did not have this set. So make sure your bashrc is setup with your correct display location.
Now that our X11 environment is setup, we can install the AMD drivers.
Installing AMD Drivers:
To begin installing the AMD drivers, we need to install some prerequisites. First install unzip with the following command
sudo apt-get install unzip
Next, we need to install the dependencies for fglrx, which is the proprietary Linux driver for AMD on Ubuntu’s repositories. The only difference between fglrx and AMD’s Catalyst drivers is that the latter is newer, but they both require the same dependencies. Run the following command to install the fglrx dependencies:
sudo apt-get build-dep fglrx
If the fglrx dependencies are not installed, the AMD driver installation will fail with this fglrx error:
The latest version at this time is 13.1. It should also be noted that oclHashCat-plus requires a specific version of Catalyst installed, which at this point is 13.1. So we’re good there.
oclHashcat-plus comes in a 7z format. So we need to install p7zip to extract it.
sudo apt-get install p7zip
Run p7zip with the –d flag to extract a 7z file.
p7zip -d oclHashcat-plus-0.14.7z
Navigate to the newly extracted ocl directory and run one of the Example.sh scripts to test run the cracking process.
If all goes well you should see your cards loading up and the hash getting cracked! If you do not see all your cards being recognized, make sure that your xorg.conf was created properly. Try running the amdconfig command above again to regenerate an xorg.conf.
Next we will install John the Ripper with OpenCL support
Installing John the Ripper
Like oclHashcat-plus, John also supports cracking hashes on GPUs, but it must be compiled with the options to do so. Much of the information here is taken from the john GPU wiki (https://openwall.info/wiki/john/GPU).
Next, install the libssl-dev package from apt-get so that John compiles correctly.
sudo apt-get install libssl-dev
Navigate to the john src directory. Compile john with OpenCL for either 32 bit or 64 bit with
make linux-x86-opencl
and
make linux-x86-64-opencl
respectfully. John can also be compiled with CUDA support if you have Nvidia cards. The information on how to do that is located on their wiki.
If you get openssl headers not found during compilation, install the libssl-dev package.
Navigate back to the run directory and your newly compiled john binary should be there. You can test that John can use your GPUs by running a test command.
Conclusion
This is guide details one of many possible setups for a GPU cracking server. When all is done, our cracking server built with these specifications works very well. In Karl’s blog here, he describes common ways to obtain hashes to crack on Windows, Linux, and web applications.
Necessary cookies help make a website usable by enabling basic functions like page navigation and access to secure areas of the website. The website cannot function properly without these cookies.
Name
Domain
Purpose
Expiry
Type
YSC
youtube.com
YouTube session cookie.
52 years
HTTP
Marketing cookies are used to track visitors across websites. The intention is to display ads that are relevant and engaging for the individual user and thereby more valuable for publishers and third party advertisers.
Name
Domain
Purpose
Expiry
Type
VISITOR_INFO1_LIVE
youtube.com
YouTube cookie.
6 months
HTTP
Analytics cookies help website owners to understand how visitors interact with websites by collecting and reporting information anonymously.
We do not use cookies of this type.
Preference cookies enable a website to remember information that changes the way the website behaves or looks, like your preferred language or the region that you are in.
We do not use cookies of this type.
Unclassified cookies are cookies that we are in the process of classifying, together with the providers of individual cookies.
We do not use cookies of this type.
Cookies are small text files that can be used by websites to make a user's experience more efficient. The law states that we can store cookies on your device if they are strictly necessary for the operation of this site. For all other types of cookies we need your permission. This site uses different types of cookies. Some cookies are placed by third party services that appear on our pages.
Cookie Settings
Discover why security operations teams choose NetSPI.