Event ID 20292 from DHCP-Server

Checking over our DHCP server we were seeing quite a few of these errors appearing in the ‘Microsoft-Windows-DHCP Server Events/Admin’ event log:

Log Name:      DhcpAdminEvents
Source:        Microsoft-Windows-DHCP-Server
Date:          1/28/2019 10:23:49 PM
Event ID:      20292
Task Category: DHCP Failover
Level:         Error
Keywords:      
User:          NETWORK SERVICE
Computer:      dc2.mydomain.com
Description:
A BINDING-ACK message with transaction id: 943568 was received for IP address: 10.253.166.162 with reject reason:  (Reject Reason Unknown ) from partner server: dc1.mydomain.com for failover relationship: dc1.mydomain.com-dc2.mydomain.com.

Researching this error I came across this forum post: https://social.technet.microsoft.com/Forums/en-US/15d00412-3dfc-4520-a74e-1f32fe1329ef/windows-server-2012-dhcp-event-id-20291?forum=winserveripamdhcpdns

Which lead me to this KB article: https://support.microsoft.com/en-ca/help/2955135/event-id-20291-is-logged-in-the-system-log-when-a-client-computer-is-m

The hotfix that Microsoft mentions is from November 2014 and has been installed on our server for a very long time. We never noticed this error back in 2014 when the hotfix was installed so we were not able to “first remove the failover relationship, install the update to both DHCP nodes and restart them, and then reestablish the failover relationship” per Microsoft’s article.

The article leads me to believe you have to deconfigure failover on all subnets, destroy the failover relationship, re-create the failover relationship and then re-configure failover on each subnet.

Turns out you can just right click ‘Deconfigure failover’ and then right click ‘Configure failover’ on the specific subnets having the issue and re-use the existing failover relationship to resolve this issue assuming you’ve installed the November 2014 hotfix.

DHCP stops serving IPs when audit log is full

We run two DHCP servers in a HA configuration. The HA is configured to split the scopes in half. Depending on how high up the scope your IP is will determine which DHCP server you get your IP from. We have DHCP audit logging enabled.

DHCP1 handles 0-127 and DHCP2 handles 128-254 (we mostly use /24’s right now).

We started getting reports of random devices on the network not being able to connect or login to the domain. By the time a technician got to the PC to check it the issue was resolved magically.

We dug into the DHCP servers and found the DHCP audit log on DHCP1 was full (36MB in size). The log on DHCP2 was not full (yet, only 34MB in size).

Stopping DHCP on DHCP1, renaming the audit log and then starting DHCP on DHCP1 again appeared to resolve the issue.

The thing that had us scratching our heads is we’ve had this problem before and we had re-configured DHCP on these servers to allow the log files to grow to 250MB but things had stopped at 36MB.

We used this PowerShell to make the change a long while ago and restarted the DHCP service: https://docs.microsoft.com/en-us/powershell/module/dhcpserver/set-dhcpserverauditlog?view=win10-ps

Set-DhcpServerAuditLog -MaxMBFileSize 250

Per the above link it states “-MaxMBFileSize Specifies the maximum size of the audit log, in megabytes (MB).”

It turns out this PowerShell command simply changes the registry value for HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DHCPServer\Parameters\DhcpLogFilesMaxSize which you can just do manually if you’d prefer.

I have no idea how I found it but after some digging I found this article for Server 2008 (we’re using 2012R2): https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc726869(v=ws.10)

It states:

 

Dynamic Host Configuration Protocol (DHCP) servers include several logging features and server parameters that provide enhanced auditing capabilities. You can specify the following features:

  • The file path in which the DHCP server stores audit log files. DHCP audit logs are located by default at %windir%\System32\Dhcp.
  • A maximum size restriction (in megabytes) for the total amount of disk space available for all audit log files created and stored by the DHCP service.
  • An interval for disk checking that is used to determine how many times the DHCP server writes audit log events to the log file before checking for available disk space on the server.
  • A minimum size requirement (in megabytes) for server disk space that is used during disk checking to determine if sufficient space exists for the server to continue audit logging.

 

I’ve bolded and italicized the relevant line. The article also specifically references the registry key the PowerShell command changes.

This leads me to believe the PowerShell documentation is incorrect and “-MaxMBFileSize” specifies the maximum size of all audit logs added together. Not a maximum size per individual audit log.

I checked the directory size of “%windir%\system32\dhcp” on both servers and they were very close to 250MB.

We’ve since made the following change:

Set-DhcpServerAuditLog -MaxMBFileSize 4096

I will update this article if this does not resolve the issue for us.

 

Update 2019-01-10: I can confirm this resolved the issue for us. The log file for the following day reached 54MB with no issue.

Migrating DHCP from 2003 to 2012 R2

This post will likely fall on deaf ears since no one out there is still running Windows Server 2003 right?

Well we are for our oldest DHCP server. Better yet it’s 2003 Standard (non R2) which means I can’t installed Active Directory Management Gateway Service which would allow me to remotely access DHCP on the 2003 server via PowerShell 5.1 running on a different box. Newer versions of PowerShell include support for working with DHCP. PowerShell 2.0 (latest available for Server 2003) does not support these commands.

Googling around found me the standard recommended way of migrating DHCP subnets.

On the source machine run:

netsh dhcp server export 192.168.0.0.txt 192.168.0.0

This will export all of the current leases and reservations in the 192.168.0.0 scope into a text file, you can then transfer the text file over to your new DHCP server and run the following to import it:

netsh dhcp server import 192.168.0.0.txt 192.168.0.0

The downside to this method is that it causes a temporary outage of your DHCP server during the import/export. I just migrated 80 odd subnets during the day and the outages were so short no one noticed.

Alright so that was the easy part.

In addition to 80 standard DHCP scopes we have 4 superscopes that also need to be migrated.

Attempting to migrate the superscopes using the above method failed. When I attempted to import the scope into the destination DHCP server I got the error “TLS supported but not configured”.

The first post I found for this error in Google links to a Technet discussion where someone states “No, you cannot direct migrate windows server 2003 DHCP/DNS to windows server 2012 DHCP/DNS.”. Clearly not an accurate statement since I’d just migrated 80 standard scopes.

So here I am. I cannot move 4 super scopes using the netsh method, I cannot use PowerShell because the 2003 server is to old and I do not want to upgrade it to 2003 R2 for multiple reasons (did I mention it’s a Domain Controller to?).

What I ended up doing was building a new Windows 2012 R2 Standard box, joined it to our domain so I could remotely access it and have the benefit of domain logins for accessing resources across our network and then installed DHCP on it. Immediately after the DHCP Server installation completed I went into the Windows Firewall and blocked DHCP just in case.

Then on the old DHCP server I ran this command:

netsh dhcp server export all.txt all

I then transferred the “`all.txt“` file over to the DHCP server I just built and ran this:

netsh dhcp server import all.txt all

and ended up with a complete copy of my old DHCP server on my temporary DHCP server including my superscopes, no errors.

Now I can use PowerShell to finish this up. On the temporary DHCP server I ran this:

Export-DhcpServer -ComputerName "localhost" -File "C:\temp\SUPERSCOPE1.xml" -ScopeId 192.168.48.0,192.168.49.0 -Leases

transferred “`SUPERSCOPE1.xml“` to the new DHCP server and ran:

Import-DhcpServer -ComputerName "localhost" -File "C:\temp\SUPERSCOPE1.xml" -BackupPath "C:\temp\" -ScopeId 192.168.48.0,192.168.49.0 -Leases

And bingo. Superscope successfully migrated from 2003 to 2012R2 with a small middle step.

If you screw up or need to do this in batches over time you can quickly and easily wipe everything out on the temporary DHCP server by doing the following:

  1. Stop the DHCP service
  2. Delete the contents of C:\Windows\system32\dhcp
  3. Start DHCP servoce

You’ll end up with a blank DHCP server that you can re-import a fresh copy of your old DHCP server into.