We use SnapManager for Exchange and NetApp Single Mailbox Recovery to backup our Exchange 2010 environment.
We’ve noticed that randomly our backups will fail with a verification error. The backups are still good and the failure usually happens 2-3 times in a row and then doesn’t happen again for a month.
NetApp has a KB article (login required) for this which basically tells you to verify that a set of applicable Exchange DLL and EXE files are all the same version on all of our Exchange servers that SME runs on.
We did this, they are and in our case we actually only involve a single server when it comes to SME so the versions on our other Exchange servers are irrelevant (I think).
Here is a snippet of what our log looks like:
[02:47:01.595] ErrCheckLogs: failed with ERROR -1811 [02:47:01.595] Operation terminated with error -1811 ChecUnknown, in 0.0 seconds. [02:47:01.595] WARNING: Database/log consistency checking returned with error code 0xFFFFF8ED. [02:47:01.611] Error Code: 0xc004146f Transaction log verification failed. [02:47:01.611] Error Code: 0xc004146f Transaction log verification failed. [02:47:01.611] Error Code: 0xc004146f Transaction log verification failed. [02:47:01.611] Updating SnapInfo file at C:\Exchange\Logs\lun_Exchange_Logs_AE\sme_snapinfo\EXCH__MYSERVER\SG__AE\08-31-2015_22.00.12__Daily\SnapInfo__08-31-2015_22.00.12.sme with LogConsistentCheck=Failure [02:47:02.126] Verification failed. Error code: 0xc004146f, LUN: C:\Exchange\Databases\lun_Exchange_AE\ [02:47:02.126] An autosupport message is sent on failure to the storage system of LUN [C:\Exchange\Databases\lun_Exchange_AE\]. [02:47:02.126] ***LOG VERIFICATION STATUS: AE [02:47:02.126] Failed to verify physical integrity of the transaction logs. [02:47:02.126] ***LOG CONSISTENT VERIFICATION: Public Folders (MYSERVER) [02:47:02.126] Start to verify the logs in SnapInfo directory...
I’ve opened a case with NetApp and it appears they have an internal bug for this issue which you can learn nothing about here: http://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=132153
Update 2015-09-18
Heard back from NetApp and they recommend the following:
- Disable circular logging (this is enabled in our case)
- There are log files with bad generation numbers and you can fix the issue by running the SnapManager for Exchange backup without verification which will commit and truncate the transaction logs, removing the log files with the bad generation number then you can Re-run the job again with verification enabled.
Update 2015-09-22
After finally digging into the circular logging configuration on our Exchange system it turns out we only had it enabled for 2 of our 5 mail stores. I then went back through all of our failure logs and the only databases that ever generated a verification error were the 2 configured for circular logging.
I’ve disabled circular logging on both of those mail stores and we’ll see if that solves this problem.