SharePoint Online - Inventory

GitHub Repo

Background

The idea for this started when working on a SharePoint on-prem (2013 specifically) to SharePoint Online migration project as I needed to crawl every document library within the tenant and get an extract of all the fields to reconcile that they had been created and set properly.

After that thought it would be good to do the same thing for other SharePoint objects so it then became a pet project that i'm still working on at the moment inspired from James Hammonds SharePoint On-Prem farm inventory script.

What it covers

This script essentially exports SharePoint objects (Site, Web, List, Item) properties that while (mostly) accessible via the UI just isn’t viable to scan through at scale to review. Once the script has completed you can import the data into Excel, Power BI, SQL, etc so you can review the data to spotlight areas to look in to.

Data Model

Open in a new tab to see full size

What use-cases does this have

This script is mainly for checking configuration settings for sites. Here are some things you could check for:

• Versioning settings Lists / Libraries
• Sharing setting for sites
• Locale is set correctly
• Time Zones are set correctly
• Libraries with a certain name has a certain column
• Sites, lists, and libraries that are not crawled by search
• If a site breaks a pattern (eg each site with prefix "dept-" must have an certain list / document library)
• A specific field choice values are consistent
• Extract of all documents and items for a site

Report Examples

Below are some (redacted) screenshots of some simple pivot tables that can be created from this data that can be looked further in to or simply get an understanding of the SharePoint object configuration.

Lists that have broken permission inheritance

Document Libraries that are not crawled for search

Make sure to check sites as well if you replicate this

Site Language Setting

Site Sharing Setting

Site Timezone Setting

How do you run it

PowerShell

#1. Load Inventory Script
$ModulePath = "C:\Users\Kris\Documents\GitHub\SharePointOnlineInventory";
. ($ModulePath + "\Module-SPOnlineInventory.ps1") -ModulePath $ModulePath;

#2. Set tenant Url
$TenantUrl = "https://{TENANT}.sharepoint.com";

#3. Get Credentials to run the inventory scripts with
$Tenant = "{TENANT}.onmicrosoft.com";
$AppCreds = (Get-SPOnlineHelperAppCredential -StoredCredentialName "{STORED CRED NAME}");

#Certificate Authentication
$PnPCredentialParams = @{
    ClientID = ($AppCreds.ClientID);
    Thumbprint = ($AppCreds.ClientSecret);
    Tenant = $Tenant;
};

#Client ID and Secret
<#
$PnPCredentialParams = @{
    ClientID = ($AppCreds.ClientID);
    ClientSecret = ($AppCreds.ClientSecret);
};

#Interactive
$PnPCredentialParams = @{
    Interactive = $true;
};
#>


#3. Inventory Settings
$InventorySettingsParams = @{
    PnPCredentialParams = $PnPCredentialParams;
    LogFilePrefix = "{LOG PREFIX}";
    ExportFolder = "C:\Users\Kris\Documents\Blogging\Online Inventory\Example";
    Delimiter = "{#]";
};

$InventorySettings = (Get-SPOnlineInventorySettings @InventorySettingsParams);

#4. Remove existing CSV Extracts
Get-ChildItem -LiteralPath ($InventorySettings.ExportFolder) -Filter "*.csv" | Remove-Item;

#5. Run Inventory Command
Run-SPOnlineFullInventorySitesWebsLists -InventorySettings $InventorySettings -TenantUrl $TenantUrl;

1. Load the Inventory script
2. Set the Tenant Url if scanning entire tenant (otherwise the specific site url)
3. Get the credential you are going to use and store the values in a Hash. These parameters are needed (excluding url) for Connect-PnPOnline as they will be splatted
4. As the inventory appends to the CSV clear the existing ones out (make sure to export to a currently empty directory!)
5. Run the inventory command you want processed with its required parameters

Check out the example script on GitHub

What commands does this have

All of these commands are GETs, this script does not perform any updates to SharePoint.

Get-SPOnlineInventorySettings

Gets the settings object for the inventory

Review this to:

• Remove or Add any fields to be excluded
• Change the Date format

Run-SPOnlineInventoryFullInventory

Runs the entire inventory over the whole tenant.

Run-SPOnlineInventoryFullInventoryForSite

Runs the entire inventory over the one particular site

Run-SPOnlineInventoryPartialInventoryForSite

Runs the inventory for the specified elements for the particular site. Edit this function to suit what elements you want to extract.

Run-SPOnlineInventoryFullInventorySitesWebsLists

Runs the inventory for Site Collections, Webs, and Lists for the entire tenant

Inventory-SPOnlineSiteCollection

Extract of Site Collection Properties

Inventory-SPOnlineWeb

Extract of Web Properties

Inventory-SPOnlineLists

Extract of List (including document libraries) Properties

Inventory-SPOnlineListFields

Extract of List (including document libraries) Field Properties

Inventory-SPOnlineContentTypes

Extract of Content Types Properties

Inventory-SPOnlinePermissions

Extract of Permissions Properties. This is used be other methods in the script (List, and Item permissions). Web will be developed soon

Inventory-SPOnlineGroups

Extract of Site Group Properties

Inventory-SPOnlineFeatures

Extract of Site Collection Feature Properties

Inventory-SPOnlineListItems

Extract of List Item (including documents) Properties

Inventory-SPOnlineListItemPermissions

Extract of List Item (including documents) Permission Properties

Inventory-SPOnlineListViews

Extract of List (including document libraries) View Properties

Inventory-SPOnlineWebParts (incomplete)

Extract of Web Parts Properties

I’ll write a few further posts in the future going over some of these.

Eventually I’ll also write a post on how to thread this (likely write another method within the inventory itself)

Notes before running

• Credentials being used needs to have access to all the SharePoint sites to inventory
• When the script completes make sure to check that there were no errors
• Throttling is a concern for this script. There are some elements in the script (eg Get-SPOnlineHelperPnPProperty) to help mitigate this however make sure to review the output to see if there are any errors thrown
• If you run this over an entire tenant I would recommend excluding the permissions as it is a massive bottleneck for the script. Threading this helps. Going to look into how to speed this up. Scanning items also takes a while but isn’t quite as time consuming as permissions.
• Change the following if required
  • The Date Format is hardcoded at the moment (dd-MMM-yyyy HH:mm:ss.fff), change this if required.
  • As the dates are returned in UTC so are converted into the time zone of the machine running the script (regardless of what the site is set to). This is set in the inventory settings. You can hard code this to your preferred time zone.
• Detecting if the SharePoint list items have unique permissions is done via the REST API. The reason for this is it’s faster as this property can be retrieved in bulk, PnP has to load this per-item.
• Schema XML most of the time can be excluded unless you have a requirement for it. Having this will vastly increase the size of the CSV file extracts.
• As this script generates CSV files line breaks are replaced. If you need them you can have the function have them replaced with “{LineBreak}” then re-add the line break when processing it for the import (SQL, Power BI, etc).
• Using the “WaitBetweenSites” will force a 1 minute wait between sites to help further prevent throttling.
• Review the excluded system fields (this list is incomplete and there are some system fields I left in (eg Created, Modified)

Eventual Changes

• Threaded Method – will have to create X (number of threads) CSVs then after completion merge them.
• Investigate if there is a better way of handling permissions as calling Get-PnPProperty for each item that has unique permissions is a massive bottleneck on this script.
• Power BI report improvements
• Adding column to the sites or webs extract to indicate if the site is also a Team Site
• Extending the following:
  • Web Parts (currently not well tested)
  • Site Collection and Web Features
• Adding methods to inventory the following
  • Property Bag
  • Site Permissions
  • Site Collection Administrators
  • Team Channels
• Let me know of other properties that would be worth including

Script is free to use (and extend) and is provided as-is.

Thanks for reading this far, let me know if you have any questions, if there are any issues, or if you’ve found the script useful 😃