SRM Actual failover fails with the error message “Unable to access file ds:///vmfs/volumes/xxxx-xxxx-xxxxx/”

Recently I encountered a very interesting issue under SRM where one of my customer was trying to perform a planned migration and right after virtual machine were powered down on the production site and preparing for migration I ran through the error message given below

“Unable to access file ds:///vmfs/volumes/xxxx-xxxx-xxxxx/”

Out of 25 VM’s in the protection group we encountered this error message on 7 of them.

Being a dedicated SRM resource first thing that pops up in to our brains would be to analyze the SRM logs on the production site

So I browsed through C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\VMware-dr.log

Found below stack traces being triggered during the error occurred.

VMware-DR.log

03:00 [92668 verbose 'LocalVcServer-PCM' opID=279bbc83 remoteTask=task-488793] [PCM] Recording new filter 'sessionId[522b6]52cd053a-2d2e-ed16-580c-b972dda651df' for token '509' and connection version '1'
 2017-10-26T20:23:38.343+03:00 [92668 error 'RemoteTaskMonitor' remoteTask=task-488793] Dr::Internal::RemoteTaskBase::HandleTaskError: The remote task failed, error=
 --> (vim.fault.CannotAccessFile) {
 -->    faultCause = (vmodl.MethodFault) null,
 -->    faultMessage = (vmodl.LocalizableMessage) [
 -->       (vmodl.LocalizableMessage) {
 -->          key = "com.vmware.vim.vpx.file.creation.Error",
 -->          arg = (vmodl.KeyAnyValue) [
 -->             (vmodl.KeyAnyValue) {
 -->                key = "filename",
 -->                value = "/vmfs/volumes/54465721-86f264ec-c4d9-f8db889e2639"
 -->             }
 -->          ],
 -->          message = "/vmfs/volumes/54465721-86f264ec-c4d9-f8db889e2639 could not be created"
 -->       }
 -->    ],
 -->    file = "ds:///vmfs/volumes/54465721-86f264ec-c4d9-f8db889e2639/"
  -->    msg = "Unable to access file ds:///vmfs/volumes/54465721-86f264ec-c4d9-f8db889e2639/"
 --> }
 03:00 [92668 error 'RemoteTaskMonitor' remoteTask=task-488793] Dr::Internal::RemoteTask<class Dr::TypedMoRef<class Vim::VirtualMachine,void> const >::Fail: The remote task failed

Now the SRM log was not pretty straight forward on providing me the exact cause of the issue but looking at the task & events tab under thick client I could see the error message encountered during create virtual machine task.

Now when do you see a create virtual machine task initiated during planned migration??

The answer would be while creating a place holder VM in the production site. So in order to create a placeholder VM you need a accessible placeholder datastore.

Bingo! So cause was lying at the placeholder datastore as the datastore assigned under placeholder datastore tab was not accessible to all the ESXI host. So any VM’s registered to the ESXI host which does not have the placeholder datastore accessible ran through the same error message.

So you have two choices in order to fix this issue.

• Either map this datastore to all the ESXI host so that it could be accessible to each and every host.
• Or add a new datastore which is accessible from all the ESXI host whose registered VM’s are part of the protection group.

Once we rectified the issue we had one happy customer.

Hope this article was informative. Watch out for more.