Implementing Multi-Threading with MaxDegreeOfParallelism in SharePoint using CSOM – An Insight

In the last article we saw how to use Multi Threading in SharePoint using CSOM. But, I felt some what we don’t have control over the thread in the last implementation. i.e., If we have 1000 files in the source and by the previous code, all the 1000 files may try to upload simultaneously. That may cause definitely an impact over the tenant. (Yes, we faced the request time out, forbidden exceptions etc., )

So, I want the batches to be splitted and then upload the files. i.e., very simple. There should be multi threading. But, I will be deciding my thread count.

 namespace Console.Office365
 {
     using Microsoft.SharePoint.Client;
     using Newtonsoft.Json.Linq;
     using System;
     using System.Collections.Generic;
     using System.IO;
     using System.Linq;
     using System.Threading.Tasks;
 
     class Program
     {
         static void Main(string[] args)
         {
             UploadFilesMultiThread();
         }
 
 
 
         public static void UploadFilesMultiThread()
         {
             string[] filePaths = System.IO.Directory.GetFiles("D:\Temp\UploadFiles");
 
             OfficeDevPnP.Core.AuthenticationManager authMgr = new OfficeDevPnP.Core.AuthenticationManager();
 
             string siteUrl = "https://******.sharepoint.com/sites/CommunitySite/";
             string userName = "Sathish@******.onmicrosoft.com";
             string password = "**********";
 
 
 
             List<Action> actionsArray = new List<Action>();
 
             foreach (var filePath in filePaths)
             {
                 actionsArray.Add(new Action(() => UpLoadFiles(authMgr, siteUrl, userName, password, filePath)));
             }
 
 
 
 
             Action[] array = actionsArray.ToArray();
 
             System.Threading.Tasks.Parallel.Invoke(new System.Threading.Tasks.ParallelOptions { MaxDegreeOfParallelism = 2 }, array);
 
 
             System.Console.WriteLine("Process Completed");
             System.Console.ReadLine();
         }
 
         private static void UpLoadFiles(OfficeDevPnP.Core.AuthenticationManager authMgr, string siteUrl, string userName, string password, string filePath)
         {
             using (var ctx = authMgr.GetSharePointOnlineAuthenticatedContextTenant(siteUrl, userName, password))
             {
                 Web web = ctx.Web;
                 ctx.Load(web);
 
                 List list = web.Lists.GetByTitle("MyDocumentsLibrary");
                 ctx.Load(list);
                 ctx.Load(list.RootFolder);
 
                 ctx.ExecuteQueryRetry();
 
                 if (System.IO.File.Exists(filePath))
                 {
                     System.Console.ForegroundColor = ConsoleColor.Green;
                     System.Console.WriteLine("Entered into : " + Path.GetFileName(filePath));
                     
                     Folder folder = web.GetFolderByServerRelativeUrl(list.RootFolder.ServerRelativeUrl);
                     folder.UploadFile(Path.GetFileName(filePath), filePath, true);
 
                     folder.Update();
 
                     ctx.Load(folder);
                     ctx.ExecuteQueryRetry();
                 }
             }
         }
 
 
          
 
     }
 
 }
 

On the below line, the MaxDegreeOfParallelism Parameter is the one, which decides how many items in the array to be taken at a time simultaneously.

System.Threading.Tasks.Parallel.Invoke(new System.Threading.Tasks.ParallelOptions { MaxDegreeOfParallelism = 2 }, array);

Happy Coding,

Sathish Nadarajan.