Authentication error from http requests to SharePoint web services initiated by SharePoint timer job

Solution

All Gimmal Solutions

Product

All Gimmal Products

Product Version

(PH/FA)

Issue

An authentication error occurs when http requests to SharePoint web services initiated from a Sharepoint timer job are serviced by the SharePoint web application on the same server that the timer job executes on.


Example Scenario:
In this example scenario Gimmal Provisioning is deployed in a two server SharePoint farm, with the server A configured as the web front end server and server B configured as the app server.  A provisioning definition is created to provision sites from the provisioning template.  A site is successfully provisioned, with the provisioning job running on the app server.  A file resources in the provisioning definition is added or updated, and the 'Update Sites' button is clicked in the provisioning definition to queue an update in the provisioning list.

If the provisioning timer job runs on the app server the provisioning job runs successfully and the file is provisioned to the target site.  If the provisioning timer job runs on the web front end server the provisioning job fails and the file is not provisioned.

The following exception is found in the SharePoint ULS logs:

Copying files from the resource gallery to the provisioned site.
Failed to copy files from the resource gallery to the provisioned site.

Exception
TYPE: Gimmal.Common.CommonException
MESSAGE: Failed to upload file '/sites/testsite/_catalogs/masterpage/yourpage.aspx'.

STACK: at Gimmal.Common.SharePoint.Client.CopyManager.UploadFileViaRest(HttpClient client, String webUrl, String folderUrl, String destinationFilename, Stream content, Boolean overwrite)
at Gimmal.Common.SharePoint.Client.CopyManager.UploadFileViaRest(HttpClient client, String webUrl, String folderUrl, String destinationFilename, Byte[] content, Boolean overwrite)
at Gimmal.Common.SharePoint.Client.CopyManager.UploadDestinationFileViaRest(HttpClient client, String destinationFilename, Boolean overwrite)
at Gimmal.Common.SharePoint.Client.CopyManager.UploadDestinationFile(String destinationFilename, Boolean overwrite, Boolean useAppToken, Boolean loadItem, String username, String password, Expression`1[] retrievals)
at Gimmal.Common.SharePoint.Client.CopyManager.CopyFilesInFolder(CopyManager copyManager, List sourceList, Folder sourceFolder, String targetFolderFieldName, String targetFolderPath, Boolean recursive, String checkInComment, String publishComment, String approvalComment, Int32 batchSize)
at Gimmal.Common.SharePoint.Client.CopyManager.CopyFilesInFolder(ClientContext sourceContext, String sourceFolderPath, String targetFolderFieldName, ClientContext targetContext, String targetFolderPath, Boolean recursive, String checkInComment, String publishComment, String approvalComment, Int32 batchSize)
at Gimmal.Provisioning.ProvisioningJobHelper.TryCopyResourcesToSite(ClientContext govHubContext, Uri newSiteUri)

Inner Exception
TYPE: System.AggregateException
MESSAGE: One or more errors occurred.

STACK: at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at System.Net.Http.HttpClientExtensions.ExecutePostWithRetry(HttpClient client, String url, HttpContent content, Int32 retryCount, Int32 delay)
at Gimmal.Common.SharePoint.Client.CopyManager.UploadFileViaRest(HttpClient client, String webUrl, String folderUrl, String destinationFilename, Stream content, Boolean overwrite)

Inner Exception
TYPE: System.NotSupportedException
MESSAGE: The stream does not support concurrent IO read or write operations.

STACK: at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
at System.Net.ConnectStream.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
at System.Net.Http.StreamToStreamCopy.TryStartWriteSync(Int32 bytesRead)
at System.Net.Http.StreamToStreamCopy.StartRead()

The inner exception 'The stream does not support concurrent IO read or write operations' is an invalid exception.  This exception does not occur when the provisioning timer job runs on the app server.


Multiple 401 errors are found in the IIS logs on the web front end server

GET /sites/govhub/_api/web/getfilebyserverrelativeurl(@u)/openbinarystream @u='/sites/govhub/Lists/GimmalResources/Gimmal.Provisioning/Page%20Layouts/yourpage.aspx' 443 - 401 0 0 15
GET /sites/govhub/_api/web/getfilebyserverrelativeurl(@u)/openbinarystream @u='/sites/govhub/Lists/GimmalResources/Gimmal.Provisioning/Page%20Layouts/yourpage.aspx' 443 - 401 0 0 15
GET /sites/govhub/_api/web/getfilebyserverrelativeurl(@u)/openbinarystream @u='/sites/govhub/Lists/GimmalResources/Gimmal.Provisioning/Page%20Layouts/yourpage.aspx' 443 - 401 0 0 15
GET /sites/govhub/_api/web/getfilebyserverrelativeurl(@u)/openbinarystream @u='/sites/govhub/Lists/GimmalResources/Gimmal.Provisioning/Page%20Layouts/yourpage.aspx' 443 - 401 0 0 15
POST /sites/simpletest/_api/contextinfo - 443 - 401 0 0 16
POST /_vti_bin/client.svc/contextinfo - 443 0#.w|am\sernamename - 200 0 0 15
POST /sites/simpletest/_api/web/getfolderbyserverrelativeurl(@f)/files/add(overwrite=true,url=@u) @f='/sites/simpletest/_catalogs/masterpage'&@u='yourpage.aspx' 443 - 401 0 0 15
POST /sites/simpletest/_api/web/getfolderbyserverrelativeurl(@f)/files/add(overwrite=true,url=@u) @f='/sites/simpletest/_catalogs/masterpage'&@u='yourpage.aspx' 443 - 401 1 2148074254 0
POST /sites/simpletest/_api/web/getfolderbyserverrelativeurl(@f)/files/add(overwrite=true,url=@u) @f='/sites/simpletest/_catalogs/masterpage'&@u='yourpage.aspx' 443 - 401 1 2148074248 15


Testing revealed that this issue occurred when host entries were configured to point the web application host names to the local static IP addresses hosting these web applications – ensuring that requests created from the server would be routed to itself.  Provisioning failed during the Rest upload web service call with a 401.  With the host entries pointed to the other server there was no 401 errors and provisioning completed successfully  So the issue only occurred on server requests to itself, on both the wfe and app server.

The Loopback check was disabled by setting the DisableLoopbackCheck registry key, which bypasses the security check to check for the request domain name against the server host name for all requests, following Microsoft support documentation - https://support.microsoft.com/en-us/help/896861/you-receive-error-401-1-when-you-browse-a-web-site-that-uses-integrate..

To set the DisableLoopbackCheck registry key, follow these steps:
Set the DisableStrictNameChecking registry entry to 1.

Click Start, click Run, type regedit, and then click OK.
In Registry Editor, locate and then click the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
Right-click Lsa, point to New, and then click DWORD Value.
Type DisableLoopbackCheck, and then press ENTER.
Right-click DisableLoopbackCheck, and then click Modify.
In the Value data box, type 1, and then click OK.
Quit Registry Editor, and then restart your computer.

With the DisableLoopbackCheck registry key disabled, the provisioning test of provisioning an updated file to a target site were re-executed, with the same results.

Root Cause

 All Microsoft documentation that has been referenced on this issue states that DisabledLoopBackCheck bypasses the security check to check for the request domain name against the server host name for ALL requests.  When BackConnectionHostNames is configured, this check is bypassed only for hostnames in the BackConnectionHostNames registry key. BackConnectionHostNames implements this exclusion differently than DisableLoopBackCheck in specific scenarios.

Resolution

The Microsoft documentation provides two methods for disabling the loopback check, setting the DisableLoopbackCheck registry key, which bypasses the security check to check for the request domain name against the server host name for all requests, and configuring the BackConnectionHostNames registry key, where the loopback check is disabled only for hostnames in the BackConnectionHostNames registry key.   BackConnectionHostNames is preferred as it decreases the attack surface for attacks that try to reflect, but is a simply limiting this exclusion list to specific hostnames, rather than for all requests.

In the example scenario, when the web application host names were added to the BackConnectionHostNames registry key the web service call to SharePoint was successful, the provisioning job was successful and the file was uploaded to the target site. This scenario was tested on both the SharePoint web front end server and the app server by removing the BackConnectionHostNames entries, resetting IIS, confirming the 401 error, re-adding the BackConnectionHostNames again, and reconfirming that the web service upload call was successful, and Provisioning completed successfully.

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk