Effortless document location updates and deletions
Version 2.6 March 2024
Introduction
This guide is for users who want to update the Document Searchability database without using Document Searchability UI, either directly or through a background script. You will need to call Aquaforest.Searchlight.DatabaseManager.exe (located in the bin folder where Document Searchability is installed) with the appropriate parameters based on the operation being performed.
Operations
A list of supported operations is described in the following sections.
Update locations
This operation allows you to:
add new locations and optionally delete existing locations before adding
update the authentication of existing locations
Usage
Below is a simplified usage example of all the allowed parameters. Optional parameters are in square brackets.
Aquaforest.Searchlight.DatabaseManager.exe /Operation="UpdateLibraryLocations" /DocumentLibraryID="<ID>" /LocationsTextFile="<PATH>" [<AUTHENTICATION_PARAMS>] [/DeleteExistingLocations] [/ValidateLocations] [/ContinueOnLocationValidationError] [/LogToConsole] [(/LogToFile /LogFilePath="<LOG_PATH>")] [/Debug]The following is a more detailed usage example with the AUTHENTICATION_PARAMS expanded. The authentication parameters depend on the Library Type of Document Searchability library you are updating.
Aquaforest.Searchlight.DatabaseManager.exe /Operation="UpdateLibraryLocations" /DocumentLibraryID="<ID>" /LocationsTextFile="<PATH>" [{/Username="<USERNAME>" /Password="<PASSWORD>" [(/UseAdfs /AdfsHost="<ADFS_HOST>" /AdfsRelyingPartyIdentifier="<ADFS_IDENTIFIER>")] | /UseModernAuthentication /AzureAppId="<AZURE_APP_ID>" /TenantUrl="<TENANT>" /CertificatePath="<CERT_PATH>" /CertificatePassword="<CERT_PASS>" | /UseAppOnlyAuthentication /ClientId="<CLIENT_ID>" /ClientSecret="<CLIENT_SECRET>" | /AzureStorageAccountName="<STORAGE_ACC_NAME>" /AzureStorageAccountKey="<STORAGE_ACC_KEY>"}] [/DeleteExistingLocations] [/ValidateLocations] [/ContinueOnLocationValidationError] [/LogToConsole] [(/LogToFile /LogFilePath="<LOG_PATH>")] [/Debug]The following authentication parameters are required based on the Library Type of the Document Searchability library being updated and the authentication required to access the locations being added to the library.
| Document Searchability library type | Location authentication type | Required parameters |
|---|---|---|
| File System | N/A | N/A |
| SharePoint On-Prem | Basic Authentication | /Username /Password |
| SharePoint On-Prem | ADFS | /Username /Password /UseAdfs /AdfsHost /AdfsRelyingPartyIdentifier |
| SharePoint Online | Basic Authentication | /Username /Password |
| SharePoint Online | Modern Authentication | /UseModernAuthentication /AzureAppId /TenantUrl /CertificatePath /CertificatePassword |
| SharePoint Online | App-Only Authentication | /UseAppOnlyAuthentication /ClientID /ClientSecret |
| Azure Blob Storage | N/A | /AzureStorageAccountName /AzureStorageAccountKey |
| Azure File Share | N/A | /AzureStorageAccountName /AzureStorageAccountKey |
Parameters
| Parameters | Description |
|---|---|
| /Operation | The operation being performed. For this particular case the value for this parameter is “UpdateLibraryLocations” |
| /DocumentLibraryID | The ID of the Document Searchability document library whose locations are being updated. The ID can be found on the Dashboard or the log file after running a library. |
| /LocationsTextFile | The path of the text file that contains all the locations that are going to be added. Each location must be in a new line. |
| /DeleteExistingLocations | Whether or not to delete existing locations. If this is set, all existing locations of the document library will be deleted before adding the new locations |
| /ValidateLocations | Whether or not to validate each location being added. This will check if the locations already exist in other Document Searchability libraries. If the Library Type is not File System, it will also check if the authentication parameters supplied can be used to access each location. |
| /ContinueOnLocationValidationError | If this is set and /ValidateLocations is also set, locations that are invalid will be skipped. If this is not set and /ValidateLocations is set, processing will be aborted as soon as the first invalid location is encountered and no locations will added or deleted. |
| /LogToConsole | Whether or not to log the processing status to the console |
| /LogToFile | Whether or not to log the processing status to a log file. If this is set, then a value for /LogFilePath must also be set. |
| /LogFilePath | The path of the log file if /LogToFile is set. |
| /Debug | Whether or not to log debug information |
| /Username | The username of the user that have access to the locations being added |
| /Password | The password associated with the username of the user that have access to the locations being added |
| /UseAdfs | Whether ADFS is required to access the locations being added |
| /AdfsHost | Hostname of the ADFS server E.g. sts.company.com |
| /AdfsRelyingPartyIdentifier | E.g. urn:federation:saml |
| /UseModernAuthentication | Whether Modern Authentication is required to access the locations being added |
| /AzureAppId | The ID of the Azure web application |
| /TenantUrl | The tenant where the Azure web application resides E.g. <tenant>.onmicrosoft.com |
| /CertificatePath | The local path of the certificate |
| /CertificatePassword | The password for the certificate |
| /AzureStorageAccountName | The Azure storage account name |
| /AzureStorageAccountKey | The Azure storage account key |
Examples
- Add new locations to a File System library
- do not validate the new locations being added
- do not delete exisiting locations
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="1" /LocationsTextFile="C:\MyFiles\Locations.txt" /LogToConsole /Debug- Add new locations to a SharePoint library
- with basic authentication
- validate the new locations being added
- stop processing if there is an invalid location
- delete existing locations
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="2" /LocationsTextFile="C:\MyFiles\Locations.txt" /LogToConsole /Debug /Username="[email protected]" /Password="my-plain-password" /ValidateLocations /DeleteExistingLocations- Add new locations to a SharePoint library
- with modern authentication
- validate the new locations being added
- do not stop processing if there is an invalid location
- delete existing locations
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="3" /LocationsTextFile="*C:\MyFiles\Locations.txt*" /LogToConsole /Debug /UseModernAuthentication /AzureAppId="eaf4cbf1-afed-4c8b-a291-8658757dea44" /TenantUrl="mycompany.onmicrosoft.com" /CertificatePath="C:\Certificates\AzureCert.pfx" /CertificatePassword="cert-plain-password" /ValidateLocations /ContinueOnLocationValidationError /DeleteExistingLocations- Add new locations to a SharePoint library
- with app-only authentication
- validate the new locations being added
- do not stop processing if there is an invalid location
- delete existing locations
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="3" /LocationsTextFile="*C:\MyFiles\Locations.txt*" /LogToConsole /Debug /UseAppOnlyAuthentication /ClientId="eaf4cbf1-afed-4c8b-a291-8658757dea44" /ClientSecret="client-secret" /ValidateLocations /ContinueOnLocationValidationError /DeleteExistingLocations- Add new locations to an Azure Blob Storage library
- do not validate the new locations being added
- delete existing locations
- log to file
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="1" /LocationsTextFile="*C:\MyFiles\Locations.txt*" /AzureStorageAccountName="teststorageacc" /AzureStorageAccountKey="vvWfxkddddfff" /LogToConsole /LogToFile /LogFilePath="C:\Logs\Log1.txt" /Debug /DeleteExistingLocationsUpdate existing locations in SharePoint library
- with ADFS authentication
- do not validate the new locations being added
Pre-requisites
- /DeleteExistingLocations must not be set
- /LocationsTextFile must contain one or more paths that already exist in the document library being updated** **
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**UpdateLibraryLocations**" /DocumentLibraryID="4" /LocationsTextFile="C:\MyFiles\Locations.txt" /LogToConsole /Debug /UseAdfs /AdfsHost=" sts.company.com" /AdfsRelyingPartyIdentifier="urn:federation:saml"WARNING: If a location already exists in another Document Searchability library (which can happen if you add locations and do not set /ValidateLocations), the authentication for the location in that library will also change.
Delete locations
This operation allows you to delete locations from a document library using different methods:
Delete all locations
Delete specific locations
Delete specific locations by matching regexes
It aslo allows you to log a list of locations matched by regexes without deleting them from the database. This is useful to debug the regexes you supply.
Usage
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**DeleteLocationsFromLibrary**" /DocumentLibraryID="*\<ID\>*" **{**/DeleteAll **\|** /LocationsTextFile="*\<PATH\>*" **\[**/IsRegex**\] \[**/DryRun**\]}\[**/LogToConsole**\] \[(**/LogToFile /LogFilePath="*\<LOG_PATH\>*"**)\] \[**/Debug**\]**Parameters
| Parameters | Description |
|---|---|
| /Operation | The operation being performed. For this particular case the value for this parameter is “DeleteLocationsFromLibrary” |
| /DocumentLibraryID | The ID of the Document Searchability document library whose locations are being deleted. The ID can be found on the Dashboard or the log file after running a library. |
| /DeleteAll | Delete all locations from the library |
| /LocationsTextFile | Only delete the locations specified in this text file. Each location to be deleted must be in a new line. - NOTE 1: If /DeleteAll is not set, then a value for this parameter must be set. - NOTE 2: This text file can either contain a list of plain text paths OR regex paths to delete. It cannot contain a combination of both plain and regex paths. - NOTE 3: If the text file contains regex paths, the /IsRegex parameter below must be set. The regex paths will matched with unencoded URLs in the case of SharePoint and Azure library types. If a specific URL is being matched using Regex, it should not end with a “/” - Valid regex: - ^https://company.sharepoint.com/sites/site1$ - Invalid regex: ^https://company.sharepoint.com/sites/site1/$ |
| /IsRegex | Set this if the locations in /LocationsTextFile are regexes. If multiple regexes are specified in the text file, each location will be matched with all of the regexes until a match is found. If no matches are found, the location will not be deleted. |
| /DryRun | Use this to get a list of locations that would have been deleted based on the contents of /LocationsTextFile without actually deleting the locations from the database. - This is useful if the text file contains regex paths to check which path(s) the regex(es) are successfully matching against. - NOTE: This is only applicable when when /DeleteAll is not set. It should be used in conjuction with /Debug and /LogToConsole and/or /LogToFile so as to be able to view the paths identified for deletion. |
| /LogToConsole | Whether or not to log the processing status to the console |
| /LogToFile | Whether or not to log the processing status to a log file. If this is set, then a value for /LogFilePath must also be set. |
| /LogFilePath | The path of the log file if /LogToFile is set. |
| /Debug | Whether or not to log debug information |
Examples
- Delete all location from a document library
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**DeleteLocationsFromLibrary**" /DocumentLibraryID="1" /LogToConsole /Debug /DeleteAll- Only delete the locations defined in the text file from a document library
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**DeleteLocationsFromLibrary**" /DocumentLibraryID="2" /LogToConsole /Debug /LocationsTextFile="C:\MyFiles\Locations.txt"- Only delete locations that matches the regexes defined in the text file from a document library
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**DeleteLocationsFromLibrary**" /DocumentLibraryID="3" /LogToConsole /Debug /LocationsTextFile="C:\MyFiles\Locations.txt" /IsRegex- Do not delete locations from database but log the list of the locations matched from the regexes in the text file that would have been deleted if /DryRun was not set.
Aquaforest.Searchlight.DatabaseManager.exe /Operation="**DeleteLocationsFromLibrary**" /DocumentLibraryID="4" /LogToConsole /Debug /LogToFile /LogFilePath="C:\Logs\Logs2.txt" /LocationsTextFile="C:\MyFiles\Locations.txt" /IsRegex /DryRun