Tag: FIPFS

Happy New Year from Microsoft to Exchange on-premise users!

Due to a pretty bad MSExchange Antimailware update that was sent out, As a result of bad code, the MSExchange Antimalware engine broke, and Exchange will not process email.

The following errors are found in the event viewer of the affected systems:

Event 5801:

The anti-malware agent encountered an error while scanning. MessageId: <guid@domain.net> Message sent: 1/1/2022 5:05:21 PM From: <> Size: 12503 Bytes Error: Microsoft.Filtering.ScanAbortedException: Exception of type 'Microsoft.Filtering.ScanAbortedException' was thrown.
   at Microsoft.Filtering.InteropUtils.ThrowPostScanErrorAsFilteringException(WSM_ReturnCode code, String message)
   at Microsoft.Filtering.FilteringService.EndScan(IAsyncResult ar)
   at Microsoft.Exchange.Transport.Agent.Malware.MalwareAgent.OnScanCompleted(IAsyncResult ar)

Event 5300:

The FIP-FS "Microsoft" Scan Engine failed to load. PID: 19120, Error Code: 0x80004005. Error Description: Can't convert "2201010009" to long.

Event 1106:

The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error

The current fix is to disable the anti-malware scan module with the following Exchange system script:

Disable-AntimailwareScanning.ps1

https://docs.microsoft.com/en-us/Exchange/antispam-and-antimalware/antimalware-protection/antimalware-protection?view=exchserver-2019

1/3/2022 Update:
After several days the issue came back. Apparently, disabling the anti-mailware scanning does not actually resolve it, possibly the mail processing still goes through the bad engine! Microsoft has now released an article with the corrected procedure of resetting the engine with the following article: https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-exchange-on-premises-transport-queues/ba-p/3049447

To reset the engine, run the following script: https://aka.ms/ResetScanEngineVersion

If unavailable, here is the code:

[CmdletBinding()]
param (
    [switch]$Force
)

begin {
    #region Remoting Scriptblock
    $scriptBlock = {
        #region Functions
        function Get-ExchangeInstallPath {
            return (Get-ItemProperty -Path Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Setup -ErrorAction SilentlyContinue).MsiInstallPath
        }

        function StopServicesAndProcesses {
            Write-Host "$($env:COMPUTERNAME) Stopping MSExchangeTransport, FMS, and updateservice..."
            Stop-Service FMS -Force
            $updateservice = Get-Process updateservice -ErrorAction SilentlyContinue
            if ($null -ne $updateservice) {
                $updateservice | Stop-Process -Force
                Start-Sleep -Seconds 2
                $updateservice = Get-Process updateservice -ErrorAction SilentlyContinue
                if ($null -ne $updateservice) {
                    Write-Warning "$($env:COMPUTERNAME) Could not end process updateservice.exe. Please end this process and rerun the script."
                    return $false
                }
            }

            return $true
        }

        function RemoveMicrosoftFolder {
            Write-Host "$($env:COMPUTERNAME) Removing Microsoft engine folder..."
            $installPath = Get-ExchangeInstallPath
            if ($null -ne $installPath) {
                $microsoftFolder = Join-Path $installPath "FIP-FS\Data\Engines\amd64\Microsoft"
                Remove-Item -Recurse -Force $microsoftFolder
            }
        }

        function EmptyMetadataFolder {
            Write-Host "$($env:COMPUTERNAME) Emptying metadata folder..."
            $installPath = Get-ExchangeInstallPath
            if ($null -ne $installPath) {
                $metadataFolder = Join-Path $installPath "FIP-FS\Data\Engines\metadata"
                Get-ChildItem $metadataFolder | Remove-Item -Recurse -Force
            }
        }

        function StartServices {
            Write-Host "$($env:COMPUTERNAME) Starting services..."
            Start-Service FMS
            Start-Service MSExchangeTransport
        }

        function StartEngineUpdate {
            Write-Host "$($env:COMPUTERNAME) Starting engine update..."
            $installPath = Get-ExchangeInstallPath
            $updateScriptPath = Join-Path $installPath "Scripts\Update-MalwareFilteringServer.ps1"
            $fqdn = [System.Net.Dns]::GetHostEntry([string]"localhost").HostName
            & $updateScriptPath $fqdn
        }

        function WaitForDownload {
            $percentComplete = 0
            do {
                Start-Sleep -Seconds 1
                $transfer = Get-BitsTransfer -AllUsers | Where-Object { $_.DisplayName -like "Forefront_FPS*" }
                if ($null -ne $transfer) {
                    if ($null -ne $transfer.BytesTotal -and
                        $null -ne $transfer.BytesTransferred -and
                        $transfer.BytesTotal.GetType() -eq [Int64] -and
                        $transfer.BytesTransferred.GetType() -eq [Int64] -and
                        $transfer.BytesTotal -gt 0) {
                        $percentComplete = ($transfer.BytesTransferred * 100 / $transfer.BytesTotal)
                    }

                    Write-Progress -Activity "$($env:COMPUTERNAME) Downloading scan engines" -Status "$($transfer.BytesTransferred) / $($transfer.BytesTotal)" -PercentComplete $percentComplete
                }
            } while ($null -ne $transfer)
        }
        #endregion Functions

        Add-PSSnapin -Name Microsoft.Exchange.Management.Powershell.E2010
        $hasMailboxRole = (Get-ExchangeServer ($env:COMPUTERNAME)).ServerRole -like "*Mailbox*"
        if ((-not $Force) -and (-not $hasMailboxRole)) {
            Write-Host "$($env:COMPUTERNAME) This server does not have the Mailbox role. Add -Force to proceed anyway."
            return
        }

        Add-PSSnapin -Name Microsoft.Forefront.Filtering.Management.PowerShell
        $engineInfo = Get-EngineUpdateInformation
        Write-Host "$($env:COMPUTERNAME) UpdateVersion: $($engineInfo.UpdateVersion)"
        $isImpacted = $engineInfo.UpdateVersion -like "22*"
        if ((-not $Force) -and (-not $isImpacted)) {
            Write-Host "$($env:COMPUTERNAME) This server is not impacted. Add -Force to proceed anyway."
            return
        }

        $succeeded = StopServicesAndProcesses
        if (-not $succeeded) {
            return
        }

        RemoveMicrosoftFolder
        EmptyMetadataFolder
        StartServices
        StartEngineUpdate
        WaitForDownload
    }
    #endregion Remoting Scriptblock
}
process {
    Invoke-Command -ScriptBlock $scriptBlock
}