Silencing my Dell T340 – Part 2

For those of you who read my Part 1 of this project you’ll remember I said I’d try installing a better CPU heat sink once I had some cash.

Well I did and it didn’t work. This post will hopefully save someone 1.5 hours of their life.

The mission was to get a Noctua NH-D15 installed in my Dell T340

The first step was to remove the existing heat sink and left over thermal compound:

First thing I wanted to do was make sure the heat sink would fit before trying to figure out the bracket.

I tried it in the recommended configuration and it does fit but I’d lose access to my bottom PCIe 8x slot.

I then rotated the heat sink 90 degrees to a less than ideal airflow path which made it possible to JUST BARELY use the bottom PCIe 8x slot.

If I were keeping the Noctua installed I’d put a strip of electrical tape long the edge of the PCIe card in the bottom slot so if the heat sink ever made contact with the card it wouldn’t blow up.

Ok, so, the heat sink appears to barely fit. Now to mount it. This is where everything went wrong.

The pre-installed CPU socket back plate is not compatible with the Noctua heat sink. I removed the motherboard and then the OEM CPU back plate which also required removing the CPU locking assembly (the arm and bracket that hold the CPU to the motherboard socket) because it was screwed into the OEM CPU back plate.

Once I had removed the OEM CPU back plate I installed the Noctua provided one and quickly realized it wasn’t going to work. The Noctua plate does not have threaded holes so I was unable to re-attach the CPU locking assembly.

At this point there was nothing more I could do, I had to re-install the original OEM CPU back plate and OEM cooler and then put everything back in the case. In theory the motherboard is raised a fair distance from the back of the T340’s case. I could have gotten some washers and nuts and attempted to re-attach the CPU locking assembly using them and the Noctua back plate but this ultimately seemed like a bad idea.

I reached out to Noctua support just to see if they had a compatible back plate. They said “no” and that their products are not compatible with this motherboard.

I’ve also reached out to Arctic, BeQuite!, CoolMaster, Corsair, Thermaltake and Zalman to see if any of them sell a compatible cooler. So far the results are:

  • Arctic – No compatible products
  • BeQuite! – No compatible products
  • CoolMaster – Didn’t read my request properly and didn’t actually answer my question
  • Corsair – No compatible products
  • Thermaltake – No response yet
  • Zalman – No response yet

I will update this post when/if the remaining companies respond to me.

I’m just going to have to live with my current solution which does an adequate job.

How to perform an offline audit of your Active Directory NTLM hashes

It’s read-only Friday so I decided to perform a offline audit of our Active Directory passwords.

I found this great tool: https://gitlab.com/chelmzy/five-minute-password-audit which in turn is a fork of this tool: https://github.com/DGG-IT/Match-ADHashes

What I’m going to write here is mostly a repeat of these two Gitrepos with a few tweaks and corrections.

To perform this procedure you will need to be able to login to a Domain Controller. You’re also going to want a secure location to perform all of this work so the dumped list of usernames and hashes doesn’t escape your control.

The secure location should be a workstation or server running the same or a newer version of Windows than your Domain Controller. For example if you’re running AD 2012R2 you can’t complete this on a 2008R2 box. You’re secure workstation or server will need to be running PowerShell 5.0 or newer.

Step 1 – Export NTDS.dit and the SYSTEM hive

  1. Login to a domain controller
  2. Open a Command Prompt window
  3. Type “ntdsutil”
  4. Click ‘Yes’ if the UAC prompts you
  5. Run the following commands:
    ifm
    activate instance ntds
    
    # Replace <DOMAINNAME> with your domains name
    create full c:\temp\<DOMAINNAME>-audit
    
    # Wait for command to complete
    quit
    quit
  6. Transfer “C:\Temp\<DOMAINNAME>-audit” to the secure location you’ll work on it. I do not recommend performing the rest of these steps on your Domain Controllers

Step 2 – Download the latest Have I Been Pwned Offline NTLM password list

  1. Go to https://haveibeenpwned.com/Passwords
  2. Scroll to the bottom and download the “ordered by prevalence” NTLM link
  3. Once downloaded, transfer the password list to your secure location in the audit directory and extract it

Step 3 – Covert the hashes in the NTDS.dit file to Hashcat formatting

  1. On your secure workstation/server launch PowerShell as an administrator (right click, run as administrator on the PowerShell shortcut)
  2. Install the DSInternals tools by running
    Install-Module -Name DSInternals -Force
  3. Go into the audit directory
    cd c:\temp\<DOMAINNAME>-audit
  4. Convert the hashes
    $key = Get-BootKey -SystemHivePath .\registry\SYSTEM
    
    # Change <DOMAINNAME> to your domains name
    Get-ADDBAccount -All -DBPath '.\Active Directory\ntds.dit' -BootKey $key | Format-Custom -View HashcatNT | Out-File <DOMAINNAME>-hashes.txt -Encoding ASCII

Step 4 – Compare your hashes to HIBP

The code in the Git Repos I linked at the beginning of the article are written as functions. For myself I just wanted a script I could execute with the appropriate parameters instead of futzing around with importing the function.

I also tweaked the original script for formatting (I like a bit more white space personally), added CSV headers, removed the spaces between commas, had the script append it’s execution time to the end of the CSV file and allowed for relative filenames as parameters instead of requiring absolute paths.

Here is my version of the script:

<#
This is a slightly altered version of https://gitlab.com/chelmzy/five-minute-password-audit/blob/master/Match-ADHashes.ps1 which is a slightly alter version of https://github.com/DGG-IT/Match-ADHashes/ for no nonsense output. All credit to them.

.NAME
    Match-ADHashes

.SYNOPSIS
    Matches AD NTLM Hashes against other list of hashes

.DESCRIPTION
    Builds a hashmap of AD NTLM hashes/usernames and iterates through a second list of hashes checking for the existence of each entry in the AD NTLM hashmap
        -Outputs results as object including username, hash, and frequency in database
        -Frequency is included in output to provide additional context on the password. A high frequency (> 5) may indicate password is commonly used and not necessarily linked to specific user's password re-use.

.PARAMETER ADNTHashes
    File Path to 'Hashcat' formatted .txt file (username:hash)

.PARAMETER HashDictionary
    File Path to 'Troy Hunt Pwned Passwords' formatted .txt file (HASH:frequencycount)

.PARAMETER Verbose
    Provide run-time of function in Verbose output

.EXAMPLE
    $results = Match-ADHashes -ADNTHashes C:\temp\adnthashes.txt -HashDictionary -C:\temp\Hashlist.txt 

.OUTPUTS
    Array of HashTables with properties "User", "Frequency", "Hash"
    User                            Frequency Hash                            
    ----                            --------- ----                            
    {TestUser2, TestUser3} 			20129     H1H1H1H1H1H1H1H1H1H1H1H1H1H1H1H1
    {TestUser1}                     1         H2H2H2H2H2H2H2H2H2H2H2H2H2H2H2H2

.NOTES
    If you are seeing results for User truncated as {user1, user2, user3...} consider modifying the Preference variable $FormatEnumerationLimit (set to -1 for unlimited)
    
    =INSPIRATION / SOURCES / RELATED WORK
        -DSInternal Project https://www.dsinternals.com
        -Checkpot Project https://github.com/ryhanson/checkpot/

    =FUTURE WORK
        -Performance Testing, optimization
        -Other Languages (golang?)

.LINK
    https://github.com/DGG-IT/Match-ADHashes/

#>

param(
    [Parameter(Mandatory = $true)]
    [System.IO.FileInfo] $ADNTHashes,

    [Parameter(Mandatory = $true)]
    [System.IO.FileInfo] $HashDictionary
)

process {

    $stopwatch = [System.Diagnostics.Stopwatch]::StartNew()

    # Set the current location so .NET will be nice and accept relative paths
    [Environment]::CurrentDirectory = Get-Location

    #Declare and fill new hashtable with ADNThashes. Converts to upper case to 
    $htADNTHashes = @{} 
    Import-Csv -Delimiter ":" -Path $ADNTHashes -Header "User","Hash" | % {$htADNTHashes[$_.Hash.toUpper()] += @($_.User)}

    #Create empty output object
    $mrMatchedResults = @()
       
    #Create Filestream reader 
    $fsHashDictionary = New-Object IO.Filestream $HashDictionary,'Open','Read','Read'
    $frHashDictionary = New-Object System.IO.StreamReader($fsHashDictionary) 


    # Output CSV headers
    Write-Output "Username,Frequency,Hash"

    #Iterate through HashDictionary checking each hash against ADNTHashes
    while (($lineHashDictionary = $frHashDictionary.ReadLine()) -ne $null) {

        if($htADNTHashes.ContainsKey($lineHashDictionary.Split(":")[0].ToUpper())) {

                $user = $htADNTHashes[$lineHashDictionary.Split(":")[0].ToUpper()]
                $frequency = $lineHashDictionary.Split(":")[1]
                $hash = $linehashDictionary.Split(":")[0].ToUpper()
                Write-Output "$user,$frequency,$hash"

            }

        }

    $stopwatch.Stop()
    Write-Output "Function Match-ADHashes completed in $($stopwatch.Elapsed.TotalSeconds) Seconds"

}
    
end {
}

To execute it, copy/paste it into notepad and save it as ‘myAudit.ps1’ or what ever file name you’d like.

Now perform your audit:

# Replace <DOMAINNAME> with your domain name
.\myAudit.ps1 -ADNTHashes <DOMAINNAME>-hashes.txt -HashDictionary <HIBP TEXT FILE> | Out-File <DOMAINNAME>-PasswordAudit.csv

# Example
.\myAudit.ps1 -ADNTHashes myDomain-hashes.txt -HashDictionary pwned-passwords-ntlm-ordered-by-count-v5.txt | Out-File myDomain-PasswordAudit.csv

The final result will be a CSV file you can dig through.

Step 6 – Clean it all up

The output may or may not surprise you but what ever the outcome, when you’re done you want to get rid of the <DOMAINNAME>-hashes.txt and the NTDIR.dis file as soon as possible. If someone snags a copy of that you’ll likely get in some serious trouble.

Head on over to SysInternals and grab SDelete

.\sdelete.exe -p 7 -r -s <DIRECTORY OR FILE>

 

List/Audit all folder delegate permissions on an Exchange mailbox

We recently needed a way to see what delegate permissions a client had given across the vastness that is their mailbox and it’s folder structure.

Digging around online I found this script from John Hopkins which got me 90% of the way there.

Their script was missing three things for my use case:

  • Delegate permissions on the root folder of the mailbox
  • Exclude the actual user from the report
  • Little tidier formatting of the output

This script has been tested against Exchange 2016 only.

 

# Use displayname
$mailbox = "<DISPLAY NAME>"

$permissions = @()
$folders = Get-Mailboxfolderstatistics $mailbox | % {$_.folderpath} | % {$_.replace(“/”,”\”)}

# Get the root folder of the mailbox
$folderKey = $mailbox + ":" + "\"
$permissions += Get-MailboxFolderPermission -identity $folderKey -ErrorAction SilentlyContinue | Where-Object {$_.User -notlike “Default” -and $_.User -notlike “Anonymous” -and $_.AccessRights -notlike “None” -and $_.User -notlike $mailbox}

# Get the rest of the folders
$list = ForEach ($folder in $folders)
   {
    $folderKey = $mailbox + ":" + $folder
    $permissions += Get-MailboxFolderPermission -identity $folderKey -ErrorAction SilentlyContinue | Where-Object {$_.User -notlike “Default” -and $_.User -notlike “Anonymous” -and $_.AccessRights -notlike “None” -and $_.User -notlike $mailbox}
   }

$permissions | Format-Table -AutoSize

 

Accessing a Pi-Hole behind an Apache reverse proxy

Update 2019-09-15: Finally got around to looking into this and it turns out all I had to change was “ProxyPreserveHost Off” to “ProxyPreserveHost  On” to get things working. I’ve updated the original post to reflect the changes. I also didn’t note in my original host that I purposely restricted access to the apache virtual host to 10.0.0.0/24 and 192.168.0.0/24 (my internal networks). You’ll want to update the “Allow from” lines to reflect your internal networks OR remove the “<Location /></Location>” all together to make it accessible from anywhere (not recommended).

Update 2019-08-19: I just recently found out that this proxy configuration only allows read-only access to the Pi-Hole UI. I was attempting to white-list a domain and it was failing when accessing my Pi-Hole via the proxy. I had to go directly to the box’s FQDN to white-list a domain. I will leave this post for reference and update it when I figure out a fix to this problem.

Update 2019-09-29: My first Lets Encrypt certificate came due for auto-renewal and failed because of my original configuration. I’ve updated the apache configuration below so Lets Encrypt can access the non-SSL /.well-known directory to automatically renew certificates.

 

Original Post

Today I got tired of accessing my Pi-Hole over HTTP, having to remember to put /admin/ in the URL and having to load up a browse that wasn’t Vivaldi or Firefox because they don’t have an easy way to ignore Strict-Transport-Security for my domain.

I checked out some documentation about adding SSL to the Pi-Hole directly but have concerns that future updates will wipe out all the custom configuration to lighttpd. According to this you also have to be careful when enabling SSL on your Pi-Hole as it could interfere with blocking.

I already have an Apache webserver running so configuring it to reverse-proxy seemed like an easier task, plus if for some reason I wanted to access my Pi-Hole from the general internet (without VPN) it would be simple to enable that.

Here is the reverse proxy configuration I used with a restriction to my two internal networks and a redirect from HTTP to HTTPS:

<VirtualHost *:80>
       ServerName pihole.mydomain.com
       DocumentRoot /var/www/html
       CustomLog logs/pihole.mydomain.com.log combined
       ErrorLog logs/pihole.mydomain.com-error.log

	<Location /.well-known>
		Order allow,deny
		Allow from all
	</Location>

	<Location />
		Order deny,allow
		Deny from all
		Allow from 127.0.0.1
		Allow from 192.168.0.0/24
		Allow from 10.0.0.0/24
	</Location>

       RewriteEngine On
       RewriteRule ^(.well-known)($|/) - [L]
       RewriteCond %{SERVER_PORT} 80
       RewriteRule ^(.*)$ https://pihole.mydomain.com/ [R,L]
</VirtualHost>

<VirtualHost *:443>
        ServerName pihole.mydomain.com
        DocumentRoot /var/www/html
        CustomLog logs/pihole.mydomain.com.log combined
        ErrorLog logs/pihole.mydomain.com-error.log

	<Location />
		Order deny,allow
		Deny from all
		Allow from 127.0.0.1
		Allow from 192.168.0.0/24
		Allow from 10.0.0.0/24
	</Location>

        RewriteEngine On
        RewriteRule ^/$ /admin [R]

        ProxyRequests On
        ProxyPass /  http://pihole-internal-hostname.mydomain.com/
        ProxyPassReverse / http://pihole-internal-hostname.mydomain.com/
        ProxyPreserveHost On

        SSLEngine on
        SSLHonorCipherOrder off
        Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"

        SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
        SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

        SSLCertificateFile /etc/letsencrypt/live/pihole.mydomain.com/cert.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/pihole.mydomain.com/privkey.pem
        SSLCertificateChainFile /etc/letsencrypt/live/pihole.mydomain.com/chain.pem
</VirtualHost>

I am aware that my SSL configuration is not the best. I’m waiting for CentOS 8 to come out before migrating off my existing CentOS 6 server.

To find the best SSL configuration for your OS and Web Server I recommend checking out Mozilla’s SSL Configuration Generator: https://ssl-config.mozilla.org/

Some Microsoft Storage Spaces Benchmarks

My backup server has a ASRock H370M-ITX/AC motherboard in it and at the time of these benchmarks 3x6TB Seagate Ironwolf SATA disks.

I run Veeam and used the SATA disks as my backup repository.

My original configuration was 2x6TB Ironwolfs in a RAID0 (using Microsoft Storage Spaces) with Bitlocker enabled. This worked perfectly fine and I had no performance issues.

A project I was working on required me to add some redundancy to my backup storage so I purchased the 3rd disk and re-configured the Microsoft Storage Space as a RAID5 and re-enabled Bitlocker. Since then I had nothing but performance issues. When two backup jobs ran at the same time the server became nearly unresponsive. The jobs still ran and completed but it was very difficult to use the server and jobs took longer with the RAID5 configuration than the RAID0 configuration. A performance difference makes sense but this amount seemed abnormal.

I ended up with a spare LSI MegaRAID 9270-8i I couldn’t sell so I decided to throw it into my backup server and try running the above configuration with hardware RAID but before I did that, I ran some benchmarks.

As you can see BitLocker has a huge negative impact on this configuration even though the server is running a Intel Core i3-8100 which has hardware acceleration built in for encryption.

You can probably guess how this is all going to end now.

First a 3 disk RAID0:

Last but not least, a 3 disk RAID5:

It’ still a ~70% hit on sequential writes but the server is completely usable and backup jobs run at the speeds I would expect over 1GBe.