EML Export from Storage Location in MailStore <11

2024-08-02T00:00:00+00:00

Storage Locations created in MailStore 10 or earlier can be exported to EML files using the internal metadata. This is useful if the database is corrupt and cannot be repaired, as the EML files can be used to re-archive the messages into a new storage location.

This article applies to MailStore 10 and earlier only, as of MailStore 11 an entirely different process is used for recovering from a damaged FDB database.

MailStore 11</a> introduced new recovery records, this article only applies to Storage Locations created with MailStore 10 or earlier, that have not been upgraded to MailStore 11 or later.

If you have a damaged MailStore Storage Location's FDB file (prior to MailStore 11), it is often possible to recover and export the messages from the database using the internal recovery records.

For the purposes of this article, we will assume the following paths:

description</th> example path</th></tr></thead>

Damaged Storage Location</td> C:\MailArchive\2018-03</code></td></tr>

EML export destination</td>

description</th>	example path</th></tr></thead>
Damaged Storage Location</td>	`C:\MailArchive\2018-03</code></td></tr>`
EML export destination</td>	D:\EML</code></td></tr> </tbody></table> Prepare the environment:</p> Avoid using network drives for storing files.</li> Use locally mounted drives (ideally SAS, NVMe, SATA, or iSCSI). Avoid external storage such as USB drives and network paths.</li> For performance reasons, consider placing the Storage Location and EML export destination on separate physical drives.</li> Ensure you have ample free space. It's recommended to have at least double the size of the corrupted storage location available, but more may be needed in certain cases.</li> </ol> </li> Prepare MailStore Server:</p> If you have any Archive E-mail</em> profiles configured to run automatically they may re-archive messages from the original source, so it is recommended to disable all archiving profiles during this process.</li> Shut down MailStore Server completely or detach the problematic storage location to ensure it's not in a failed state.</li> </ol> </li> Run the Recovery Command:</p> Launch a Windows command prompt, enter the following command, substituting paths as necessary:</p> "C:\Program Files (x86)\deepinvent\MailStore Server\MailStoreServer_x64.exe" --recover --inputDir="C:\MailArchive\2018-03" --outputDir="D:\EML" </code></pre> </li> Wait for MailStore to process all DAT</em> files, extracting each to EML</em> files.</p> </li> </ol> </li> Re-Archiving Process:</p> (Optional)</em> Create a new Storage Location (MailStore Client</em> > Administrative Tools</em> > Storage</em> > Storage Locations</em> > Create Storage Location</em>).</li> Go to Archive E-mail</em> > Create a new profile, select EML and MSG Files</em> (or Directory (File System)</em>) files as the source.</li> Choose the MailStore Export</em> option to re-archive the messages back to their correct location.</li> Ensure the corrupted storage location is not attached during this process otherwise messages may be skipped.</li> </ol> </li> Post-Recovery Cleanup:</p> Once you verify the results, delete the corrupt storage location and the EML export destination. directory.</li> </ol> </li> </ol> The extracted EML</em> files can be used with email software that supports EML format, or MailStore itself.</p> 🔗</a>Encryption Keys</h2> MailStore will attempt to read the keys from the FDB</em> database files on disk if not specified, but if the database is corrupt it may not be possible to retrieve the keys automatically.</p> You can add the --securityKey</code> and/or --masterKey</code> parameters to the command line to specify the security and master keys as needed. Please contact support if you need assistance retrieving the keys as this process varies depending on the version of MailStore (and may not be possible in all cases).</p> MailStoreServer.exe --recover --inputDir="C:\FileGroup1" --outputDir="D:\EML" [--securityKey="..."] [--masterKey="..."] </code></pre> 🔗</a>Notes</h2> EML</em> files are created with X-MailStore-</code> headers that contain metadata to allow for re-archiving or other analysis.</li> EML</em> files can be: Re-archived into the correct location in MailStore.</li> Opened and accessed by any email software that supports EML</em> files, like Thunderbird.</li> </ul> </li> DAT</em> files contain fragments from various messages.</li> Messages in DAT</em> files are not always in sequential order.</li> You may attempt to perform an export on an incomplete set of DAT</em> files, only valid messages will be exported.</li> You may restore or replace DAT</em> files and then export again to fill in missing messages if further DAT</em> files are available.</li> Exporting again will create duplicate EML</em> files, but these will be skipped by MailStore when re-archiving.</li> Special Case: Storage Locations created in MailStore 10 or earlier, but with messages added by MailStore 11, may not export completely with this process.</li> In such cases, contact MailStore support for assistance.</li> </ul> </li> </ul> Extracting unknown e-mail addresses from journal-type profile's debug log 2023-01-21T00:00:00+00:00 🔗</a>Bulk import of messages without a list of all known e-mail addresses</h2> 🔗</a>Preface</h3> This article describes how to perform a one-time archive or migration of messages into MailStore where the messages are available in EML files, PST files, an Exchange journal mailbox or a Multidrop mailbox and there is no definitive list of users / e-mail addresses available.</p> All steps are to be performed in MailStore Client, logged in as an admin, unless otherwise described.</p> I will be using example.com</code> as the domain for my company in this article.</p> 🔗</a>Licensing</h3> You will need a user license for each user for the archiving process. MailStore's 30-day trial has a 500-user limit and can be used for the initial archiving, or you could import users in batches instead.</p> 🔗</a>Creating the initial users (optional)</h3> To start off with, if you will eventually be using Directory Services to synchronize users then you can set it up first. This step is optional, and Directory Services is not actually needed at all, but by first synchronizing users it will reduce the number of users that need to be manually created.</p> I'll be using the full e-mail address in this article but if your Directory Services synchronizes with Active Directory then it may use only the local part. You can modify the "CreateUser" API call slightly to accommodate. While it is possible to rename users/archives after the fact it will save some time to use the correct configuration from the server.</p> If you aren't using Directory Services or simply are not ready to synchronize users, you just need at least one user with an e-mail address attached to their account. You could edit the admin account and set admin@example.com</code> for the address.</p> 🔗</a>Archiving Profile</h3> You can archive from EML files, PST files of type Multiple Users</em> or Exchange Journal</em>, or Multidrop</em> accounts, the process is similar for all message sources.</p> 🔗</a>EML files</h4> Go to Archive E-mail</em>.</li> Create a new profile --> E-mail Files</em> --> Multiple Users</em>.</li> Set the Directory</em> to a directory that contains the EML files to be archived.</li> Set "Messages with unknown e-mail addresses" to "Don't archive".</li> Leave "Delete them in origin mailbox" unchecked.</li> Run the archiving profile.</li> </ul> 🔗</a>Running the archiving profile for the first time</h4> You can start with a subdirectory that contains a few messages to understand the process, and later run the profile again against the same messages without duplicating any messages in the archive. Depending on the amount of mail you have to import, it may take several hours to perform the initial archiving run which generates the list we will use going forward.</p> Any user that already exists in MailStore at this point will have their messages archived, while messages that do not belong to any known user will have a debug log entry created, providing us a list of needed users.</p> 🔗</a>Get the debug log</h3> Click Details</em>.</li> You should see a number of messages that are skipped with the message because the target user archive could not be determined based on the e-mail addresses</code>.</li> Click on Debug Log</em> and Select in Explorer</em>.</li> Copy the log file to another directory as MailStore will remove the file when you close the dialog.</li> Return to the details dialog and close it (this dialog blocks further access to MailStore Client).</li> </ul> You may want to open the log to take a look, in which case look for the lines that look like this:</p> MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: noreply@mailstore.com, sherry.hall@example.com, frank.clark@example.com</code></p> noreply@mailstore.com</code> is an example of an external address, while the@example.com</code> addresses belong to our example company in this documentation. Note that there may be any number of addresses, but most messages will just have two.</p> 🔗</a>Get the e-mail addresses out of the debug log</h3> We need to get the e-mail addresses from the debug log lines above into a list, one e-mail address per line, with all other punctuation and spaces removed. Duplicates will not cause errors, but will take time to process, so it is ideal to remove them from the list but it is not required.</p> There are a lot of ways you can do this, including doing it entirely by hand for a small number of users, or you could use any regex capable editor or other tools. If you are doing it manually and miss a user, just create the user and run the archiving process again, MailStore will skip the messages already in the archive.</p> Aside from preparing this list manually, I will describe three ways to extract the data from the debug log.</p> The goal is to change this:</p> MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: noreply@mailstore.com, sherry.hall@example.com, frank.clark@example.com MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: joe.smith@example.com, frank.clark@example.com </code></pre> To a test file called userlist.txt</code> that looks like this:</p> sherry.hall@example.com frank.clark@example.com joe.smith@example.com </code></pre> Note that the original list had external addresses like noreply@mailstore.com</code> that were removed, and also frank.clark@example.com</code> was listed twice but is now only listed once.</p> If you built the userlist.txt</em> file manually, move ahead to Create the users</strong>, otherwise read on for automated ways of creating the list.</p> 🔗</a>Linux / bash command line</h4> The Linux/WSL steps are located in a separate article</a> and will only take a couple of minutes to complete.</p> 🔗</a>Manually (via Excel)</h4> The Excel steps are located in a separate article</a> and only needed if you do not have access to a Linux console for a few minutes.</p> 🔗</a>Create the users</h3> Return to MailStore Client</em>.</li> Right click on the profile created earlier and select Create Scheduled Task</em>.</li> Click Copy CMD Line</em>, then cancel the scheduled task dialog.</li> Paste the line into your favourite text editor.</li> </ul> The details of the line will vary, mine looks like this:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c import-execute --id=3 --user="admin" --verbose</code></p> We need the part up to and including -c</code>, so like this:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c</code></p> First we need to create each user, which we can do using a for</code> loop through the userlist.txt</em> file created above.</p> for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c CreateUser --userName %x --privileges "none" </code></pre> Then set the e-mail addresses field:</p> for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c SetUserEmailAddresses --userName %x --emailAddresses %x </code></pre> A note about using bare usernames (without the @example.com</code> domain portion): You could remove the domain from the lines in userlist.txt</em> and then when setting e-mail addresses replace the --emailAddresses %x</code> part with --emailAddresses %x@example.com</code> instead.</p> Creating totally different usernames and e-mail addresses (e.g. fclark</code> and frank.clark@example.com</code>) or listing multiple e-mail addresses is out of scope for this article, but you could modify the command lines above to match your environment.</p> 🔗</a>Running the archive profile</h3> Once your users are created, return to the Archive E-mail</em> panel and run your Multidrop Mailbox (File System)</code> profile again, this time it should process all messages.</p> Click the Details</em> link again to review the results. If there are any messages that still don't have a user associated with them, create the missing users and repeat the archive.</p> It is likely that you will have new messages</em> archived and some messages already existed in the archive</em>, but you should not have any further messages have been skipped</em>.</p> 🔗</a>Cleanup unneeded users</h3> At this point all of your source message files have been archived, and these files are no longer needed by MailStore.</p> It is likely you'll want to delete the users created, you can do this using this command:</p> for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c DeleteUser --userName %x </code></pre> Note that this will remove all users listed, including those that were previously created by Directory Services.</p> Instead you could also use MailStore Client's Users</em> list. In this case, select all users, hold control and unselect the current admin user, then delete all users.</p> If any users were created by Directory Services and deleted above they will be recreated by Directory Services again, with their default set of permissions.</p> 🔗</a>Final notes</h3> A few more bits of information about the process that did not fit neatly above.</p> 🔗</a>Creating a user that has permissions to access the entire archive</h3> You will likely now need to create one or more users that have access to the entire archive. From the users editor you can give a user the ability to read a specific archive, but this would take a lot of clicking to give a single user access to many archives.</p> Instead you can create an Auditor</em> user that has read access to all archives without admin permissions. Go to Compliance</em> -> Compliance General</em> and create an auditor user.</p> You can create multiple auditor users if desired and once they're created you can rename them to the user's normal account name. If the target user already exists from Directory Services, delete their old account and rename the auditor user, then change their Authentication Type</em> to Directory Services</code>. Directory Services will update the full name and e-mail address the next time it synchronizes.</p> 🔗</a>User Interface</h4> In the Progress View</em> dialog when you click the Details</em> link, the dialog blocks you from accessing MailStore Client's interface until it is closed. I would therefore commend opening the debug log via Windows Explorer, copy the file, and return to MailStore Client to close this dialog otherwise it appears that MailStore Client is hung if you try to access other dialogs.</p> The debug log is only preserved while you leave Progress View</em> open, if the Debug Log</em> button is missing you are probably looking at Recent Results which does not contain the needed information.</p> 🔗</a>Archiving</h4> MailStore's archiving profiles all skip messages already in the target archive/folder, therefore there is no need to remove messages in the archive if you repeat the process after creating new users or adding more EML files.</p> If you change the folder names in the archiving profiles, this will</em> cause duplicates to be created. I would recommend using the defaults, or at least, be sure to get it right.</p> 🔗</a>Do not delete messages from the source until you're done</h4> While it might seem like you could save some time by having MailStore delete the source messages during each archiving profile execution so that you do not need to re-scan all messages that have already been archived, this is not recommended as it has an unexpected consequence.</p> The important thing to understand is that MailStore does not have any concept of your domain or any way to determine what is a local user or non-local user. MailStore just has the list of currently-licensed users and their e-mail addresses. Imagine we have frank@example.com</code> created, but bob@example.com</code> has not been created yet and then archive with the option to delete messages enabled, here is what will happen:</p> The first message is from noreply@mailstore.com</code> and to frank@example.com</code>, so it gets archived, and deleted. Great, this is what we want.</li> The second message is from noreply@mailstore.com</code> and to bob@example.com</code>, so it gets skipped and logged, but not deleted. Great, this is what we want.</li> The third message is from frank@example.com</code> and to bob@example.com</code>, so it gets archived to frank@example.com</code> only -- This message is successfully archived, therefore if the delete option is enabled the message is deleted. When you later create bob@example.com</code> the message is already gone from the source and therefore is never archived into into bob@example.com</code> at all.</li> </ul> You will likely not even notice that bob@example.com</code> is missing the messages, but even if noticed there is no way to find and fix this later. The message is not lost, it is in frank</code>'s archive, but this can cause problems when reviewing just bob</code>'s archive later.</p> Extracting unknown e-mail addresses from journal-type profile's debug log via Excel 2023-01-21T00:00:00+00:00 🔗</a>Preface</h2> This is part of a larger set of steps to set up a fresh MailStore Server and migrate messages from another archiving platform where the list of older users is not known.</p> This article just covers the steps to use Excel and the Windows Command Line to get the e-mail addresses, the full process is described here</a>.</p> You could do this by hand using Excel as well although this has a lot more individual steps.</p> There are multiple ways to accomplish this, including using scripting, but I will attempt to provide easy-to-understand steps that allow you to see each step, and no scripting is required. Here is what I have done in the past:</p> Start with Windows' find.exe</em> command to filter the log to just the relevant lines:</p> find "MailStore.Common.Interfaces.ServerUnmappableException" < .log > userlist.txt </code></pre> Open the resulting file in Excel.</li> Use Data</em> -> Text to Columns</em>.</li> Select Delimited</em>.</li> Set Delimiters to both "Comma" and "Space" only.</li> Enable Treat consecutive delimiters as one</em>.</li> Click Finish</em>.</li> Look for the columns that contains e-mail addresses at the end of each line. This should start at AA</em>.</li> Copy the results and paste to a new sheet.</li> Repeat for each column that has further e-mail addresses. There may be more than AA</em> and AB</em> as e-mail can have more than a single sender/recipient pair per message. Remember there may be lines that have no 3rd address followed by more lines that have a 3rd address, but if you miss any there is an opportunity to catch them later.</li> </ul> Now switch to the new sheet.</p> Use Data</em> --> Remove Duplicates</em> to clean up the list.</li> Insert a blank row at the top.</li> Enable Data</em> --> Filter</em>.</li> Click the down arrow on the top line to modify the filter.</li> Type example.com</code> in the search</em> field and press enter.</li> Copy the results and save it as userlist.txt</code></li> </ul> Extracting unknown e-mail addresses from journal-type profile's debug log via Linux_WSL 2023-01-21T00:00:00+00:00 🔗</a>Preface</h2> This is part of a larger set of steps to set up a fresh MailStore Server and migrate messages from another archiving platform where the list of older users is not known.</p> This article just covers the steps to use WSL (Windows Subsystem for Linux) to get the e-mail addresses, the full process is described here</a>.</p> 🔗</a>Linux / bash command line</h2> From here we need to extract the e-mail addresses that match your users. If you are comfortable at a Linux command line we can do this easily:</p> grep --only-matching --no-filename --extended-regex "[^ ]+@example\.com\b" .log \| sort \| uniq > userlist.txt </code></pre> If completed this way, move ahead to Create the users</strong>.</p> 🔗</a>WSL (Windows Subsystem for Linux)</h2> Installing and learning WSL is out of scope for this article, but if you already have WSL on a machine that you can use, then from cmd.exe</code> you can call WSL and convert the debug log into the needed text-file format like this:</p> wsl grep --only-matching --no-filename --extended-regex "[^ ]+@example\.com\b" .log ^\| sort ^\| uniq > userlist.txt </code></pre> If completed this way, move ahead to Create the users</em>.</p> Getting started with MailStore SPE's Management API 2023-01-01T00:00:00+00:00 🔗</a>Enable the API</h2> The API might already be enabled, but let's check that it is enabled before proceeding:</p> Launch the MailStore Service Provider Edition Configuration</em> tool.</p> </li> Stop the Management Server</em>.</p> </li> Click Configure...</em>.</p> </li> Ensure API HTTP Server Enabled</em> is checked.</p> </li> Set Listeners</em> to :8474</code> (without the quotes).</p> </li> Copy the Certificate</em> field from Management Server Settings</strong> and paste it in to the Server Certificate</em> field in the API section.</p> </li> Click OK</em>.</p> </li> Start</em> the Management Service</em></p> </li> </ol> 🔗</a>Using the PowerShell wrapper</h2> 🔗</a>Wrapper and sample code</h2> Referring to the PowerShell API Wrapper and sample code found in: https://help.mailstore.com/en/spe/PowerShell_API_Wrapper_Tutorial</a></p> 🔗</a>Set-ExecutionPolicy to run downloaded scripts</h3> Launch PowerShell as administrator</p> </li> Run Set-ExecutionPolicy -ExecutionPolicy Unrestricted</code></p> </li> </ol> 🔗</a>Make our first API call</h3> Download MailStore PowerShell API Wrapper and tutorial example scripts</em> from https://help.mailstore.com/en/spe/images/1/19/MailStore_SPE_Scripting_Tutorial.zip</a></p> </li> Unzip MailStore_SPE_Scripting_Tutorial.zip</code></p> </li> Open the MailStore SPE Scripting Tutorial\PowerShell\Scripts</code> directory.</p> </li> Edit Example1.ps1</em>.</p> </li> Set the username/password.</p> </li> Adjust the managementserver and port (if needed). If you're working on the management server itself then the defaults should be fine.</p> </li> Run .\Example1.ps1</code></p> </li> </ol> The result I get is:</p> error : token : statusVersion : 2 statusCode : succeeded percentProgress : statusText : result : @{version=22.4.0.21151; webClientVersion=22.4.0.21151; copyright=Copyright (c) 2005-2022 MailStore Software GmbH; licenseeName=ehemaliger Testaccount; licenseeID=10510; serverName=spe-server.example.com; username=admin; systemProperties=} logOutput : </code></pre> Example2.ps</em> shows how to pass parameters using "GetInstances" @{instanceFilter = ""}</code>, and then it pulls the users from within each instance as an example of how to repeat an operation through different instances. It should just be a matter of updating the connection string in this script with your servername and credentials.</p> 🔗</a>Call the HTTPS API directly</h2> 🔗</a>Install a browser extension</h3> I'm using RESTClient (Firefox-based extension to make API calls interactively from a browser): https://addons.mozilla.org/en-US/firefox/addon/restclient/</a> but other tools should be fine.</p> For reference my server's API is available at: https://spe-server.example.com:8474</a></p> 🔗</a>Trust the certificate</h3> First access the URI and allow the untrusted certificate (if needed)</p> </li> Do this by visiting https://spe-server.example.com:8474/</a></p> </li> Advanced -> Accept the certificate</p> </li> Close the tab when prompted for credentials.</p> </li> </ol> 🔗</a>Make a first call to the API</h3> Now launch RESTClient</p> </li> Click the Authentication tab</p> </li> Set the admin username/password.</p> </li> Change the Method</strong> to Post</em></p> </li> Call https://spe-server.example.com:8474/api/invoke/GetInstanceHosts</code> (without the quotes)</p> </li> </ol> My response looks like this:</p> { "error": null, "token": null, "statusVersion": 2, "statusCode": "succeeded", "percentProgress": null, "statusText": null, "result": [ { "serverName": "spe-server.example.com", "port": 8472, "serverCertificate": { "thumbprint": "A61597D8316DC39E26466F22C4C42B0C824ABE12" }, "baseDirectory": null } ], "logOutput": null } </code></pre> I used this API method because we can now add a body to test.</p> 🔗</a>Add parameters to apply a filter</h3> I'll add the body serverNameFilter=doesnotexist</code> (again, without the quotes) and we see no errors, but also no servers were returned, so we know the server accepted the parameter as the behaviour changed from above:</p> { "error": null, "token": null, "statusVersion": 2, "statusCode": "succeeded", "percentProgress": null, "statusText": null, "result": [], "logOutput": null } </code></pre> And now I'll change the body to:</p> serverNameFilter=spe-server.example.com</code></p> Now we get the requested server again, confirming everything is working:</p> { "error": null, "token": null, "statusVersion": 2, "statusCode": "succeeded", "percentProgress": null, "statusText": null, "result": [ { "serverName": "spe-server.example.com", "port": 8472, "serverCertificate": { "thumbprint": "A61597D8316DC39E26466F22C4C42B0C824ABE12" }, "baseDirectory": null } ], "logOutput": null } </code></pre> 🔗</a>Make a more complex API call</h3> Use CreateSystemAdministrator</em> https://help.mailstore.com/en/spe/Management_API_-_Function_Reference#CreateSystemAdministrator</a> to create a new system administrator. I selected this example because we need both a json</em> parameter, and a second parameter that is not part of the json</em> block.</p> Change the URI to: https://spe-server.example.com:8474/api/invoke/CreateSystemAdministrator</code></p> </li> Change the body: config={ "userName" : "bob", "fullName" : "Bob the Admin", "emailAddress" : "bob@example.com"}&password=Passw0rd!</code></p> </li> Submit to the server.</p> </li> </ol> The result is "statusCode": "succeeded"</p> { "error": null, "token": null, "statusVersion": 2, "statusCode": "succeeded", "percentProgress": null, "statusText": null, "result": null, "logOutput": null } </code></pre> And to confirm it really did work, I verified I can login with my newly created administrator "bob" using the management web interface port 8470)</p> 🔗</a>curl</h2> We could do the same from curl</em> using this command line:</p> curl -X POST -k -H 'Authorization: Basic YWRtaW46UGFzc3cwcmQhIQ==' -i 'https://spe-server.example.com:8474/api/invoke/CreateSystemAdministrator' --data 'config={ "userName" : "joe", "fullName" : "Joe the Admin", "emailAddress" : "joe@example.com"}&password=Passw0rd!'</code></p> Use SetUserPrivilegesOnFolder from MailStoreCmd.exe 2023-01-01T00:00:00+00:00 To use MailStoreCmd.exe</em> we need to know a bit about the environment, but we can use MailStore Client to generate a working command line for us:</p> Go to the Archive E-mail</em> panel</p> </li> See if you have any EML</em> or PST</em> or File System</em> profiles that already exist and skip the next step.</p> </li> If you don't have a profile you can use, create a E-mail Files</em> --> EML</em> profile, set the path to something that doesn't exist ("C:\does-not-exist</em>"), leave the other options at their defaults until you get the checkbox to run the profile now and uncheck this. Save the profile.</p> </li> Now that you have a profile we can use, right click on it, choose Create task on...</em></p> </li> Press the Copy CMD Line</em> button at the bottom.</p> </li> Paste into Notepad.</p> </li> </ol> Mine looks like this:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c import-execute --id=3 --user="admin" --verbose</code></p> Cancel creating a scheduled task, we don't need it, but this is a shortcut to generating a command line and setting the credentials. Once we have the command line above you can delete the profile you created, we don't need it anymore.</p> Use your command rather than mine as it is personalized to your environment. We need everything up to the -c, this is where we specify the command we want to run, so for example with a base command line:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c</code></p> We can GetServerInfo</em> to prove we are connected:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c</code></p> The result:</p> GetServerInfo Connecting... Disconnecting... **************************************************************** MailStore Command Line Client Version 22.3.0.21002 *************************************************************** { "version": "22.3.0.21002", "machineName": "EXCHANGEDC" } </code></pre> Now we will retrieve the current status of the user:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c GetUserInfo --userName="frank.clark"</code></p> The result:</p> Connecting... Disconnecting... *************************************************************** MailStore Command Line Client Version 22.3.0.21002 *************************************************************** { "userName": "frank.clark", "fullName": "Frank Clark", "distinguishedName": "CN=Frank Clark,OU=Seattle,DC=example, DC=com", "authentication": "directoryServices", "emailAddresses": [ "frank.clark@example.com" ], "pop3UserNames": [], "privileges": [ "login" ], "privilegesOnFolders": [ { "folder": "frank.clark", "privileges": [ "read" ] }, { "folder": "abby.hernandez", "privileges": [ "read" ] }, { "folder": "alexis.page", "privileges": [ "read" ] } ] } </code></pre> Now we will SetUserPrivilegesOnFolder</em> to apply "read, write"</em>:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c SetUserPrivilegesOnFolder --userName="frank.clark" --folder="abby.hernandez" --privileges="read, write"</code></p> And the result:</p> Connecting... Disconnecting... *************************************************************** MailStore Command Line Client Version 22.3.0.21002 *************************************************************** </code></pre> And finally we can check the results with the previous GetUserInfo</em> command:</p> "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c GetUserInfo --userName="frank.clark"</code></p> Connecting... Disconnecting... *************************************************************** MailStore Command Line Client Version 22.3.0.21002 ***************************************************************** { "userName": "frank.clark", "fullName": "Frank Clark", "distinguishedName": "CN=Frank Clark,OU=Seattle,DC=example, DC=com", "authentication": "directoryServices", "emailAddresses": [ "frank.clark@example.com" ], "pop3UserNames": [], "privileges": [ "login" ], "privilegesOnFolders": [ { "folder": "frank.clark", "privileges": [ "read" ] }, { "folder": "alexis.page", "privileges": [ "read" ] }, { "folder": "abby.hernandez", "privileges": [ "read", "write" ] } ] } </code></pre> Active Directory with Microsoft 365 Modern Authentication 2023-01-01T00:00:00+00:00 🔗</a>Microsoft 365 / Hybrid environments</h2> Hybrid environments are complicated, and there is no one-size-fits-all solution. MailStore is more flexible than is immediately obvious but there are a few things to understand.</p> 🔗</a>MailStore's article abouy Microsoft 365 / Active Directory hybrid environments</h2> https://help.mailstore.com/en/server/Archiving_Emails_from_Microsoft_365_Hybrid</a></p> 🔗</a>Further details</h2> It is possible to switch to Microsoft 365's Modern Authentication for the purposes of archiving while still using Active Directory as a user synchronization source, and in fact this is easier as it avoids needing to rename the archive folders in your existing archive to match. We will borrow some of the steps from our published documentation, but there are others that we can skip.</p> As long as your Microsoft 365 environment is synchronized with your local Active Directory, MailStore can use either option as a source for user information and can archive from both Microsoft 365 mailboxes and your on-premises Exchange as well if you happen to have a hybrid environment.</p> To set things up, login to MailStore Client with an admin</em> account, go to Archive E-mail</em>, under Servers</em> click Microsoft 365</em>. Select Multiple Mailboxes</em>. Now click the ...</code> button, which will open the credentials manager</em>.</p> Now we will use the Synchronizing User Accounts with Microsoft 365 (Modern Authentication)</a> help page article, but start at the step In the Credential Manager that appears, click on Create…</em></p> Start at step 2.1</em>, register the app, create credentials and then publish the credentials:</p> Complete step 2.1 Registering of MailStore Server as App in Azure AD</em>.</li> Complete step 2.2 Creating Credentials in MailStore Server</em>.</li> Complete step 2.3 Publishing Credentials in Azure AD</em>.</li> Skip step 2.4 Configuring App Authentication in Azure AD</em>.</li> Skip step 2.5 Configuring the Redirect URI in MailStore Server</em>.</li> Complete 2.6 Configuring API Permissions in Azure AD</em>.</li> Stop at Step 2.7 User Database Synchronization</em> and skip the remainder of the article.</li> </ol> We skip steps 2.4</em> and 2.5</em> as they only apply to synchronizing users with Microsoft 365, but we are intending to stay with Active Directory, but the API permissions are still needed so be sure that you completed step 2.6.</p> Now that the credentials have been created you can set up Microsoft 365 archiving profiles. For information refer to the Archiving Emails from Microsoft 365 (Modern Authentication)</a> article to set up archiving as normal.</p> The principles are the same as your existing "Exchange" profiles, so you can probably skip directly to the section that covers Archiving Multiple Microsoft 365 Mailboxes Centrally</a>.</p>

D:\EML</code></td></tr>
</tbody></table>


Prepare the environment:</p>

Avoid using network drives for storing files.</li>
Use locally mounted drives (ideally SAS, NVMe, SATA, or iSCSI). Avoid external storage such as USB drives and network paths.</li>
For performance reasons, consider placing the Storage Location and EML export destination on separate physical drives.</li>
Ensure you have ample free space. It's recommended to have at least double the size of the corrupted storage location available, but more may be needed in certain cases.</li>
</ol>
</li>

Prepare MailStore Server:</p>

If you have any Archive E-mail</em> profiles configured to run automatically they may re-archive messages from the original source, so it is recommended to disable all archiving profiles during this process.</li>
Shut down MailStore Server completely or detach the problematic storage location to ensure it's not in a failed state.</li>
</ol>
</li>

Run the Recovery Command:</p>


Launch a Windows command prompt, enter the following command, substituting paths as necessary:</p>
"C:\Program Files (x86)\deepinvent\MailStore Server\MailStoreServer_x64.exe" --recover --inputDir="C:\MailArchive\2018-03" --outputDir="D:\EML"
</code></pre>
</li>

Wait for MailStore to process all DAT</em> files, extracting each to EML</em> files.</p>
</li>
</ol>
</li>

Re-Archiving Process:</p>

(Optional)</em> Create a new Storage Location (MailStore Client</em> > Administrative Tools</em> > Storage</em> > Storage Locations</em> > Create Storage Location</em>).</li>
Go to Archive E-mail</em> > Create a new profile, select EML and MSG Files</em> (or Directory (File System)</em>) files as the source.</li>
Choose the MailStore Export</em> option to re-archive the messages back to their correct location.</li>
Ensure the corrupted storage location is not attached during this process otherwise messages may be skipped.</li>
</ol>
</li>

Post-Recovery Cleanup:</p>

Once you verify the results, delete the corrupt storage location and the EML export destination. directory.</li>
</ol>
</li>
</ol>
The extracted EML</em> files can be used with email software that supports EML format, or MailStore itself.</p>
🔗</a>Encryption Keys</h2>
MailStore will attempt to read the keys from the FDB</em> database files on disk if not specified, but if the database is corrupt it may not be possible to retrieve the keys automatically.</p>
You can add the --securityKey</code> and/or  --masterKey</code> parameters to the command line to specify the security and master keys as needed. Please contact support if you need assistance retrieving the keys as this process varies depending on the version of MailStore (and may not be possible in all cases).</p>
MailStoreServer.exe --recover
     --inputDir="C:\FileGroup1"
     --outputDir="D:\EML"
    [--securityKey="..."]
    [--masterKey="..."]
</code></pre>
🔗</a>Notes</h2>

EML</em> files are created with X-MailStore-*</code> headers that contain metadata to allow for re-archiving or other analysis.</li>
EML</em> files can be:

Re-archived into the correct location in MailStore.</li>
Opened and accessed by any email software that supports EML</em> files, like Thunderbird.</li>
</ul>
</li>
DAT</em> files contain fragments from various messages.</li>
Messages in DAT</em> files are not always in sequential order.</li>
You may attempt to perform an export on an incomplete set of DAT</em> files, only valid messages will be exported.</li>
You may restore or replace DAT</em> files and then export again to fill in missing messages if further DAT</em> files are available.</li>
Exporting again will create duplicate EML</em> files, but these will be skipped by MailStore when re-archiving.</li>
Special Case:

Storage Locations created in MailStore 10 or earlier, but with messages added by MailStore 11, may not export completely with this process.</li>
In such cases, contact MailStore support for assistance.</li>
</ul>
</li>
</ul>


Extracting unknown e-mail addresses from journal-type profile's debug log
2023-01-21T00:00:00+00:00
🔗</a>Bulk import of messages without a list of all known e-mail addresses</h2>
🔗</a>Preface</h3>
This article describes how to perform a one-time archive or migration of messages into MailStore where the messages are available in EML files, PST files, an Exchange journal mailbox or a Multidrop mailbox and there is no definitive list of users / e-mail addresses available.</p>
All steps are to be performed in MailStore Client, logged in as an admin, unless otherwise described.</p>
I will be using example.com</code> as the domain for my company in this article.</p>
🔗</a>Licensing</h3>
You will need a user license for each user for the archiving process. MailStore's 30-day trial has a 500-user limit and can be used for the initial archiving, or you could import users in batches instead.</p>
🔗</a>Creating the initial users (optional)</h3>
To start off with, if you will eventually be using Directory Services to synchronize users then you can set it up first. This step is optional, and Directory Services is not actually needed at all, but by first synchronizing users it will reduce the number of users that need to be manually created.</p>
I'll be using the full e-mail address in this article but if your Directory Services synchronizes with Active Directory then it may use only the local part. You can modify the "CreateUser" API call slightly to accommodate. While it is possible to rename users/archives after the fact it will save some time to use the correct configuration from the server.</p>
If you aren't using Directory Services or simply are not ready to synchronize users, you just need at least one user with an e-mail address attached to their account. You could edit the admin account and set admin@example.com</code> for the address.</p>
🔗</a>Archiving Profile</h3>
You can archive from EML files, PST files of type Multiple Users</em> or Exchange Journal</em>, or Multidrop</em> accounts, the process is similar for all message sources.</p>
🔗</a>EML files</h4>

Go to Archive E-mail</em>.</li>
Create a new profile --> E-mail Files</em> --> Multiple Users</em>.</li>
Set the Directory</em> to a directory that contains the EML files to be archived.</li>
Set "Messages with unknown e-mail addresses" to "Don't archive".</li>
Leave "Delete them in origin mailbox" unchecked.</li>
Run the archiving profile.</li>
</ul>
🔗</a>Running the archiving profile for the first time</h4>
You can start with a subdirectory that contains a few messages to understand the process, and later run the profile again against the same messages without duplicating any messages in the archive. Depending on the amount of mail you have to import, it may take several hours to perform the initial archiving run which generates the list we will use going forward.</p>
Any user that already exists in MailStore at this point will have their messages archived, while messages that do not belong to any known user will have a debug log entry created, providing us a list of needed users.</p>
🔗</a>Get the debug log</h3>

Click Details</em>.</li>
You should see a number of messages that are skipped with the message because the target user archive could not be determined based on the e-mail addresses</code>.</li>
Click on Debug Log</em> and Select in Explorer</em>.</li>
Copy the log file to another directory as MailStore will remove the file when you close the dialog.</li>
Return to the details dialog and close it (this dialog blocks further access to MailStore Client).</li>
</ul>
You may want to open the log to take a look, in which case look for the lines that look like this:</p>
MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: noreply@mailstore.com, sherry.hall@example.com, frank.clark@example.com</code></p>
noreply@mailstore.com</code> is an example of an external address, while the@example.com</code> addresses belong to our example company in this documentation. Note that there may be any number of addresses, but most messages will just have two.</p>
🔗</a>Get the e-mail addresses out of the debug log</h3>
We need to get the e-mail addresses from the debug log lines above into a list, one e-mail address per line, with all other punctuation and spaces removed. Duplicates will not cause errors, but will take time to process, so it is ideal to remove them from the list but it is not required.</p>
There are a lot of ways you can do this, including doing it entirely by hand for a small number of users, or you could use any regex capable editor or other tools. If you are doing it manually and miss a user, just create the user and run the archiving process again, MailStore will skip the messages already in the archive.</p>
Aside from preparing this list manually, I will describe three ways to extract the data from the debug log.</p>
The goal is to change this:</p>
MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: noreply@mailstore.com, sherry.hall@example.com, frank.clark@example.com
MailStore.Common.Interfaces.ServerUnmappableException: MailStore is unable to determine where to store this email. Please ensure that e-mail addresses are specified in the users' settings. Senders and recipients: joe.smith@example.com, frank.clark@example.com
</code></pre>
To a test file called userlist.txt</code> that looks like this:</p>
sherry.hall@example.com
frank.clark@example.com
joe.smith@example.com
</code></pre>
Note that the original list had external addresses like noreply@mailstore.com</code> that were removed, and also frank.clark@example.com</code> was listed twice but is now only listed once.</p>
If you built the userlist.txt</em> file manually, move ahead to Create the users</strong>, otherwise read on for automated ways of creating the list.</p>
🔗</a>Linux / bash command line</h4>
The Linux/WSL steps are located in a separate article</a> and will only take a couple of minutes to complete.</p>
🔗</a>Manually (via Excel)</h4>
The Excel steps are located in a separate article</a> and only needed if you do not have access to a Linux console for a few minutes.</p>
🔗</a>Create the users</h3>

Return to MailStore Client</em>.</li>
Right click on the profile created earlier and select Create Scheduled Task</em>.</li>
Click Copy CMD Line</em>, then cancel the scheduled task dialog.</li>
Paste the line into your favourite text editor.</li>
</ul>
The details of the line will vary, mine looks like this:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c import-execute --id=3 --user="admin" --verbose</code></p>
We need the part up to and including -c</code>, so like this:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c</code></p>
First we need to create each user, which we can do using a for</code> loop through the userlist.txt</em> file created above.</p>
for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c CreateUser --userName %x --privileges "none"
</code></pre>
Then set the e-mail addresses field:</p>
for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c SetUserEmailAddresses --userName %x --emailAddresses %x
</code></pre>
A note about using bare usernames (without the @example.com</code> domain portion): You could remove the domain from the lines in userlist.txt</em> and then when setting e-mail addresses replace the --emailAddresses %x</code> part with --emailAddresses %x@example.com</code> instead.</p>
Creating totally different usernames and e-mail addresses (e.g. fclark</code> and frank.clark@example.com</code>) or listing multiple e-mail addresses is out of scope for this article, but you could modify the command lines above to match your environment.</p>
🔗</a>Running the archive profile</h3>
Once your users are created, return to the Archive E-mail</em> panel and run your Multidrop Mailbox (File System)</code> profile again, this time it should process all messages.</p>
Click the Details</em> link again to review the results. If there are any messages that still don't have a user associated with them, create the missing users and repeat the archive.</p>
It is likely that you will have new messages</em> archived and some messages already existed in the archive</em>, but you should not have any further messages have been skipped</em>.</p>
🔗</a>Cleanup unneeded users</h3>
At this point all of your source message files have been archived, and these files are no longer needed by MailStore.</p>
It is likely you'll want to delete the users created, you can do this using this command:</p>
for /F %x in (userlist.txt) do "C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --cred="admin@127.0.0.1" -c DeleteUser --userName %x
</code></pre>
Note that this will remove all users listed, including those that were previously created by Directory Services.</p>
Instead you could also use MailStore Client's Users</em> list. In this case, select all users, hold control and unselect the current admin user, then delete all users.</p>
If any users were created by Directory Services and deleted above they will be recreated by Directory Services again, with their default set of permissions.</p>
🔗</a>Final notes</h3>
A few more bits of information about the process that did not fit neatly above.</p>
🔗</a>Creating a user that has permissions to access the entire archive</h3>
You will likely now need to create one or more users that have access to the entire archive. From the users editor you can give a user the ability to read a specific archive, but this would take a lot of clicking to give a single user access to many archives.</p>
Instead you can create an Auditor</em> user that has read access to all archives without admin permissions. Go to Compliance</em> -> Compliance General</em> and create an auditor user.</p>
You can create multiple auditor users if desired and once they're created you can rename them to the user's normal account name. If the target user already exists from Directory Services, delete their old account and rename the auditor user, then change their Authentication Type</em> to Directory Services</code>. Directory Services will update the full name and e-mail address the next time it synchronizes.</p>
🔗</a>User Interface</h4>
In the Progress View</em> dialog when you click the Details</em> link, the dialog blocks you from accessing MailStore Client's interface until it is closed. I would therefore commend opening the debug log via Windows Explorer, copy the file, and return to MailStore Client to close this dialog otherwise it appears that MailStore Client is hung if you try to access other dialogs.</p>
The debug log is only preserved while you leave Progress View</em> open, if the Debug Log</em> button is missing you are probably looking at Recent Results which does not contain the needed information.</p>
🔗</a>Archiving</h4>
MailStore's archiving profiles all skip messages already in the target archive/folder, therefore there is no need to remove messages in the archive if you repeat the process after creating new users or adding more EML files.</p>
If you change the folder names in the archiving profiles, this will</em> cause duplicates to be created. I would recommend using the defaults, or at least, be sure to get it right.</p>
🔗</a>Do not delete messages from the source until you're done</h4>
While it might seem like you could save some time by having MailStore delete the source messages during each archiving profile execution so that you do not need to re-scan all messages that have already been archived, this is not recommended as it has an unexpected consequence.</p>
The important thing to understand is that MailStore does not have any concept of your domain or any way to determine what is a local user or non-local user. MailStore just has the list of currently-licensed users and their e-mail addresses. Imagine we have frank@example.com</code> created, but bob@example.com</code> has not been created yet and then archive with the option to delete messages enabled, here is what will happen:</p>

The first message is from noreply@mailstore.com</code> and to frank@example.com</code>, so it gets archived, and deleted. Great, this is what we want.</li>
The second message is from noreply@mailstore.com</code> and to bob@example.com</code>, so it gets skipped and logged, but not deleted. Great, this is what we want.</li>
The third message is from frank@example.com</code> and to bob@example.com</code>, so it gets archived to frank@example.com</code> only -- This message is successfully archived, therefore if the delete option is enabled the message is deleted. When you later create bob@example.com</code> the message is already gone from the source and therefore is never archived into into bob@example.com</code> at all.</li>
</ul>
You will likely not even notice that bob@example.com</code> is missing the messages, but even if noticed there is no way to find and fix this later. The message is not lost, it is in frank</code>'s archive, but this can cause problems when reviewing just bob</code>'s archive later.</p>


Extracting unknown e-mail addresses from journal-type profile's debug log via Excel
2023-01-21T00:00:00+00:00
🔗</a>Preface</h2>
This is part of a larger set of steps to set up a fresh MailStore Server and migrate messages from another archiving platform where the list of older users is not known.</p>
This article just covers the steps to use Excel and the Windows Command Line to get the e-mail addresses, the full process is described here</a>.</p>
You could do this by hand using Excel as well although this has a lot more individual steps.</p>
There are multiple ways to accomplish this, including using scripting, but I will attempt to provide easy-to-understand steps that allow you to see each step, and no scripting is required. Here is what I have done in the past:</p>
Start with Windows' find.exe</em> command to filter the log to just the relevant lines:</p>
find "MailStore.Common.Interfaces.ServerUnmappableException" < *.log > userlist.txt
</code></pre>

Open the resulting file in Excel.</li>
Use Data</em> -> Text to Columns</em>.</li>
Select Delimited</em>.</li>
Set Delimiters to both "Comma" and "Space" only.</li>
Enable Treat consecutive delimiters as one</em>.</li>
Click Finish</em>.</li>
Look for the columns that contains e-mail addresses at the end of each line. This should start at AA</em>.</li>
Copy the results and paste to a new sheet.</li>
Repeat for each column that has further e-mail addresses. There may be more than AA</em> and AB</em> as e-mail can have more than a single sender/recipient pair per message. Remember there may be lines that have no 3rd address followed by more lines that have a 3rd address, but if you miss any there is an opportunity to catch them later.</li>
</ul>
Now switch to the new sheet.</p>

Use Data</em> --> Remove Duplicates</em> to clean up the list.</li>
Insert a blank row at the top.</li>
Enable Data</em> --> Filter</em>.</li>
Click the down arrow on the top line to modify the filter.</li>
Type example.com</code> in the search</em> field and press enter.</li>
Copy the results and save it as userlist.txt</code></li>
</ul>


Extracting unknown e-mail addresses from journal-type profile's debug log via Linux_WSL
2023-01-21T00:00:00+00:00
🔗</a>Preface</h2>
This is part of a larger set of steps to set up a fresh MailStore Server and migrate messages from another archiving platform where the list of older users is not known.</p>
This article just covers the steps to use WSL (Windows Subsystem for Linux) to get the e-mail addresses, the full process is described here</a>.</p>
🔗</a>Linux / bash command line</h2>
From here we need to extract the e-mail addresses that match your users. If you are comfortable at a Linux command line we can do this easily:</p>
grep --only-matching --no-filename --extended-regex "[^ ]+@example\.com\b" *.log | sort | uniq > userlist.txt
</code></pre>
If completed this way, move ahead to Create the users</strong>.</p>
🔗</a>WSL (Windows Subsystem for Linux)</h2>
Installing and learning WSL is out of scope for this article, but if you already have WSL on a machine that you can use, then from cmd.exe</code> you can call WSL and convert the debug log into the needed text-file format like this:</p>
wsl grep --only-matching --no-filename --extended-regex "[^ ]+@example\.com\b" *.log ^| sort ^| uniq > userlist.txt
</code></pre>
If completed this way, move ahead to Create the users</em>.</p>


Getting started with MailStore SPE's Management API
2023-01-01T00:00:00+00:00
🔗</a>Enable the API</h2>
The API might already be enabled, but let's check that it is enabled before proceeding:</p>


Launch the MailStore Service Provider Edition Configuration</em> tool.</p>
</li>

Stop the Management Server</em>.</p>
</li>

Click Configure...</em>.</p>
</li>

Ensure API HTTP Server Enabled</em> is checked.</p>
</li>

Set Listeners</em> to *:8474</code> (without the quotes).</p>
</li>

Copy the Certificate</em> field from Management Server Settings</strong> and paste it in to the Server Certificate</em> field in the API section.</p>
</li>

Click OK</em>.</p>
</li>

Start</em> the Management Service</em></p>
</li>
</ol>
🔗</a>Using the PowerShell wrapper</h2>
🔗</a>Wrapper and sample code</h2>
Referring to the PowerShell API Wrapper and sample code found in: https://help.mailstore.com/en/spe/PowerShell_API_Wrapper_Tutorial</a></p>
🔗</a>Set-ExecutionPolicy to run downloaded scripts</h3>


Launch PowerShell as administrator</p>
</li>

Run Set-ExecutionPolicy -ExecutionPolicy Unrestricted</code></p>
</li>
</ol>
🔗</a>Make our first API call</h3>


Download MailStore PowerShell API Wrapper and tutorial example scripts</em> from https://help.mailstore.com/en/spe/images/1/19/MailStore_SPE_Scripting_Tutorial.zip</a></p>
</li>

Unzip MailStore_SPE_Scripting_Tutorial.zip</code></p>
</li>

Open the MailStore SPE Scripting Tutorial\PowerShell\Scripts</code> directory.</p>
</li>

Edit Example1.ps1</em>.</p>
</li>

Set the username/password.</p>
</li>

Adjust the managementserver and port (if needed). If you're working on the management server itself then the defaults should be fine.</p>
</li>

Run .\Example1.ps1</code></p>
</li>
</ol>
The result I get is:</p>
error           :
token           :
statusVersion   : 2
statusCode      : succeeded
percentProgress :
statusText      :
result          : @{version=22.4.0.21151; webClientVersion=22.4.0.21151; copyright=Copyright (c) 2005-2022 MailStore
                  Software GmbH; licenseeName=ehemaliger Testaccount; licenseeID=10510;
                  serverName=spe-server.example.com; username=admin; systemProperties=}
logOutput       :
</code></pre>
Example2.ps</em> shows how to pass parameters using "GetInstances" @{instanceFilter = "*"}</code>, and then it pulls the users from within each instance as an example of how to repeat an operation through different instances. It should just be a matter of updating the connection string in this script with your servername and credentials.</p>
🔗</a>Call the HTTPS API directly</h2>
🔗</a>Install a browser extension</h3>
I'm using RESTClient (Firefox-based extension to make API calls interactively from a browser): https://addons.mozilla.org/en-US/firefox/addon/restclient/</a> but other tools should be fine.</p>
For reference my server's API is available at:
https://spe-server.example.com:8474</a></p>
🔗</a>Trust the certificate</h3>


First access the URI and allow the untrusted certificate (if needed)</p>
</li>

Do this by visiting https://spe-server.example.com:8474/</a></p>
</li>

Advanced -> Accept the certificate</p>
</li>

Close the tab when prompted for credentials.</p>
</li>
</ol>
🔗</a>Make a first call to the API</h3>


Now launch RESTClient</p>
</li>

Click the Authentication tab</p>
</li>

Set the admin username/password.</p>
</li>

Change the Method</strong> to Post</em></p>
</li>

Call https://spe-server.example.com:8474/api/invoke/GetInstanceHosts</code> (without the quotes)</p>
</li>
</ol>
My response looks like this:</p>
{
  "error": null,
  "token": null,
  "statusVersion": 2,
  "statusCode": "succeeded",
  "percentProgress": null,
  "statusText": null,
  "result": [
    {
      "serverName": "spe-server.example.com",
      "port": 8472,
      "serverCertificate": {
        "thumbprint": "A61597D8316DC39E26466F22C4C42B0C824ABE12"
      },
      "baseDirectory": null
    }
  ],
  "logOutput": null
}
</code></pre>
I used this API method because we can now add a body to test.</p>
🔗</a>Add parameters to apply a filter</h3>
I'll add the body serverNameFilter=doesnotexist</code> (again, without the quotes) and we see no errors, but also no servers were returned, so we know the server accepted the parameter as the behaviour changed from above:</p>
{
  "error": null,
  "token": null,
  "statusVersion": 2,
  "statusCode": "succeeded",
  "percentProgress": null,
  "statusText": null,
  "result": [],
  "logOutput": null
}
</code></pre>
And now I'll change the body to:</p>
serverNameFilter=spe-server.example.com</code></p>
Now we get the requested server again, confirming everything is working:</p>
{
  "error": null,
  "token": null,
  "statusVersion": 2,
  "statusCode": "succeeded",
  "percentProgress": null,
  "statusText": null,
  "result": [
    {
      "serverName": "spe-server.example.com",
      "port": 8472,
      "serverCertificate": {
        "thumbprint": "A61597D8316DC39E26466F22C4C42B0C824ABE12"
      },
      "baseDirectory": null
    }
  ],
  "logOutput": null
}
</code></pre>
🔗</a>Make a more complex API call</h3>
Use CreateSystemAdministrator</em> https://help.mailstore.com/en/spe/Management_API_-_Function_Reference#CreateSystemAdministrator</a> to create a new system administrator. I selected this example because we need both a json</em> parameter, and a second parameter that is not part of the json</em> block.</p>


Change the URI to:
https://spe-server.example.com:8474/api/invoke/CreateSystemAdministrator</code></p>
</li>

Change the body:
config={  "userName" : "bob",  "fullName" : "Bob the Admin",  "emailAddress" : "bob@example.com"}&password=Passw0rd!</code></p>
</li>

Submit to the server.</p>
</li>
</ol>
The result is "statusCode": "succeeded"</p>
{
  "error": null,
  "token": null,
  "statusVersion": 2,
  "statusCode": "succeeded",
  "percentProgress": null,
  "statusText": null,
  "result": null,
  "logOutput": null
}
</code></pre>
And to confirm it really did work, I verified I can login with my newly created administrator "bob" using the management web interface *port 8470)</p>
🔗</a>curl</h2>
We could do the same from curl</em> using this command line:</p>
curl -X POST -k -H 'Authorization: Basic YWRtaW46UGFzc3cwcmQhIQ==' -i 'https://spe-server.example.com:8474/api/invoke/CreateSystemAdministrator' --data 'config={  "userName" : "joe",  "fullName" : "Joe the Admin",  "emailAddress" : "joe@example.com"}&password=Passw0rd!'</code></p>


Use SetUserPrivilegesOnFolder from MailStoreCmd.exe
2023-01-01T00:00:00+00:00
To use MailStoreCmd.exe</em> we need to know a bit about the environment, but we can use MailStore Client to generate a working command line for us:</p>


Go to the Archive E-mail</em> panel</p>
</li>

See if you have any EML</em> or PST</em> or File System</em> profiles that already exist and skip the next step.</p>
</li>

If you don't have a profile you can use, create a E-mail Files</em> --> EML</em> profile, set the path to something that doesn't exist ("C:\does-not-exist</em>"), leave the other options at their defaults until you get the checkbox to run the profile now and uncheck this. Save the profile.</p>
</li>

Now that you have a profile we can use, right click on it, choose Create task on...</em></p>
</li>

Press the Copy CMD Line</em> button at the bottom.</p>
</li>

Paste into Notepad.</p>
</li>
</ol>
Mine looks like this:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c import-execute --id=3 --user="admin" --verbose</code></p>
Cancel creating a scheduled task, we don't need it, but this is a shortcut to generating a command line and setting the credentials. Once we have the command line above you can delete the profile you created, we don't need it anymore.</p>
Use your command rather than mine as it is personalized to your environment. We need everything up to the -c, this is where we specify the command we want to run, so for example with a base command line:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c</code></p>
We can GetServerInfo</em> to prove we are connected:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c</code></p>
The result:</p>
GetServerInfo
Connecting...
Disconnecting...

*******************************************************************

      MailStore Command Line Client

      Version 22.3.0.21002

*******************************************************************

{
  "version": "22.3.0.21002",
  "machineName": "EXCHANGEDC"
}
</code></pre>
Now we will retrieve the current status of the user:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c GetUserInfo --userName="frank.clark"</code></p>
The result:</p>
Connecting...
Disconnecting...

*******************************************************************

      MailStore Command Line Client

      Version 22.3.0.21002

*******************************************************************

{
  "userName": "frank.clark",
  "fullName": "Frank Clark",
  "distinguishedName": "CN=Frank Clark,OU=Seattle,DC=example, DC=com",
  "authentication": "directoryServices",
  "emailAddresses": [
    "frank.clark@example.com"
  ],
  "pop3UserNames": [],
  "privileges": [
    "login"
  ],
  "privilegesOnFolders": [
    {
      "folder": "frank.clark",
      "privileges": [
        "read"
      ]
    },
    {
      "folder": "abby.hernandez",
      "privileges": [
        "read"
      ]
    },
    {
      "folder": "alexis.page",
      "privileges": [
        "read"
      ]
    }
  ]
}
</code></pre>
Now we will SetUserPrivilegesOnFolder</em> to apply "read, write"</em>:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c SetUserPrivilegesOnFolder --userName="frank.clark" --folder="abby.hernandez" --privileges="read, write"</code></p>
And the result:</p>
Connecting...
Disconnecting...

*******************************************************************

      MailStore Command Line Client

      Version 22.3.0.21002

*******************************************************************
</code></pre>
And finally we can check the results with the previous GetUserInfo</em> command:</p>
"C:\Program Files (x86)\MailStore\MailStore Server\MailStoreCmd.exe" --h="localhost" --pkv3="161F2004CB4B5735038E49F3C611D34349180BA9" --cred="admin@127.0.0.1" -c GetUserInfo --userName="frank.clark"</code></p>
Connecting...
Disconnecting...

*******************************************************************

      MailStore Command Line Client

      Version 22.3.0.21002

*******************************************************************

{
  "userName": "frank.clark",
  "fullName": "Frank Clark",
  "distinguishedName": "CN=Frank Clark,OU=Seattle,DC=example, DC=com",
  "authentication": "directoryServices",
  "emailAddresses": [
    "frank.clark@example.com"
  ],
  "pop3UserNames": [],
  "privileges": [
    "login"
  ],
  "privilegesOnFolders": [
    {
      "folder": "frank.clark",
      "privileges": [
        "read"
      ]
    },
    {
      "folder": "alexis.page",
      "privileges": [
        "read"
      ]
    },
    {
      "folder": "abby.hernandez",
      "privileges": [
        "read",
        "write"
      ]
    }
  ]
}
</code></pre>


Active Directory with Microsoft 365 Modern Authentication
2023-01-01T00:00:00+00:00
🔗</a>Microsoft 365 / Hybrid environments</h2>
Hybrid environments are complicated, and there is no one-size-fits-all solution. MailStore is more flexible than is immediately obvious but there are a few things to understand.</p>
🔗</a>MailStore's article abouy Microsoft 365 / Active Directory hybrid environments</h2>
https://help.mailstore.com/en/server/Archiving_Emails_from_Microsoft_365_Hybrid</a></p>
🔗</a>Further details</h2>
It is possible to switch to Microsoft 365's Modern Authentication for the purposes of archiving while still using Active Directory as a user synchronization source, and in fact this is easier as it avoids needing to rename the archive folders in your existing archive to match. We will borrow some of the steps from our published documentation, but there are others that we can skip.</p>
As long as your Microsoft 365 environment is synchronized with your local Active Directory, MailStore can use either option as a source for user information and can archive from both Microsoft 365 mailboxes and your on-premises Exchange as well if you happen to have a hybrid environment.</p>
To set things up, login to MailStore Client with an admin</em> account, go to Archive E-mail</em>, under Servers</em> click Microsoft 365</em>. Select Multiple Mailboxes</em>. Now click the ...</code> button, which will open the credentials manager</em>.</p>
Now we will use the Synchronizing User Accounts with Microsoft 365 (Modern Authentication)</a> help page article, but start at the step In the Credential Manager that appears, click on Create…</em></p>
Start at step 2.1</em>, register the app, create credentials and then publish the credentials:</p>

Complete step 2.1 Registering of MailStore Server as App in Azure AD</em>.</li>
Complete step 2.2 Creating Credentials in MailStore Server</em>.</li>
Complete step 2.3 Publishing Credentials in Azure AD</em>.</li>
Skip step 2.4 Configuring App Authentication in Azure AD</em>.</li>
Skip step 2.5 Configuring the Redirect URI in MailStore Server</em>.</li>
Complete 2.6 Configuring API Permissions in Azure AD</em>.</li>
Stop at Step 2.7 User Database Synchronization</em> and skip the remainder of the article.</li>
</ol>
We skip steps 2.4</em> and 2.5</em> as they only apply to synchronizing users with Microsoft 365, but we are intending to stay with Active Directory, but the API permissions are still needed so be sure that you completed step 2.6.</p>
Now that the credentials have been created you can set up Microsoft 365 archiving profiles. For information refer to the Archiving Emails from Microsoft 365 (Modern Authentication)</a> article to set up archiving as normal.</p>
The principles are the same as your existing "Exchange" profiles, so you can probably skip directly to the section that covers Archiving Multiple Microsoft 365 Mailboxes Centrally</a>.</p>

Extracting unknown e-mail addresses from journal-type profile's debug log

🔗</a>Manually (via Excel)</h4>
The Excel steps are located in a separate article</a> and only needed if you do not have access to a Linux console for a few minutes.</p>

Extracting unknown e-mail addresses from journal-type profile's debug log via Excel

Extracting unknown e-mail addresses from journal-type profile's debug log via Linux_WSL

Getting started with MailStore SPE's Management API

🔗</a>Wrapper and sample code</h2>
Referring to the PowerShell API Wrapper and sample code found in: https://help.mailstore.com/en/spe/PowerShell_API_Wrapper_Tutorial</a></p>

Use SetUserPrivilegesOnFolder from MailStoreCmd.exe

Active Directory with Microsoft 365 Modern Authentication

🔗</a>MailStore's article abouy Microsoft 365 / Active Directory hybrid environments</h2>
https://help.mailstore.com/en/server/Archiving_Emails_from_Microsoft_365_Hybrid</a></p>

mailstore-notes.dwlab.me - MailStoreSPE

Grant a user read access to all archives via the audit user function

EML Export from Storage Location in MailStore <11

mailstore-notes.dwlab.me - MailStoreSPE

Grant a user read access to all archives via the audit user function

EML Export from Storage Location in MailStore <11

Extracting unknown e-mail addresses from journal-type profile's debug log

🔗</a>Create the users</h3> Return to MailStore Client</em>.</li> Right click on the profile created earlier and select Create Scheduled Task</em>.</li> Click Copy CMD Line</em>, then cancel the scheduled task dialog.</li>

Extracting unknown e-mail addresses from journal-type profile's debug log via Excel

Extracting unknown e-mail addresses from journal-type profile's debug log via Linux_WSL

Getting started with MailStore SPE's Management API

🔗</a>Set-ExecutionPolicy to run downloaded scripts</h3> Launch PowerShell as administrator</p> </li>

🔗</a>Trust the certificate</h3> First access the URI and allow the untrusted certificate (if needed)</p> </li>

🔗</a>Make a first call to the API</h3> Now launch RESTClient</p> </li> Click the Authentication tab</p> </li> Set the admin username/password.</p> </li> Change the Method</strong> to Post</em></p> </li>

Use SetUserPrivilegesOnFolder from MailStoreCmd.exe

Active Directory with Microsoft 365 Modern Authentication