MithiDocs

Moving data between Active and Open/Deep stores


Note: The Vaultastic Inactive Store will henceforth be called the Vaultastic Open Store. During the transition, you will find references for both names within the documentation and other communications.


Overview

When you set journaling rules on your primary server or upload data to the Active Store using the Ingestion API or via LegacyFlo, the data is uploaded to the Vaultastic Active Store. This data is immediately searchable. 

If your usage does not require you to keep older data immediately accessible, you can move data from the Active to the Open store or to the Deep Store and then remove it from the Active Store. Whenever required, data can be copied to the Active Store for discovery and removed when the task is over.

This helps keep the size of the Active Store in control without compromising the safety of the data.

Moving data from Active to the Open/Deep store

Preparation

Note: This is a one-time configuration.
  • Login to Vaultastic using postmaster@<your Vaultastic domain>
  • Launch the Admin Panel
  • Select the domain and expand the Storage Management section
  • Enable Auto Storage Management
  • Select the Domain or Vault option under Open Store movement for. This option determines the organization of data on the Open/Deep Store. The table below gives the difference between the two.


Open Store movement 

Vault

Domain

Working

Open/Deep Store will have a separate data file corresponding to each vault on the Active Store. Each will contain one month's data transacted by the user associated with the vault.

Open/Deep Store will have a single data file for each day of the month. The data file will contain all the mail transacted by all the users of the domain on that day.

Storage utilization

The de-dup advantage of the active store will be lost as emails linked to a vault are copied to the destination data file

The de-dup advantage of the active store is maintained as the data files for each day of the month are created with emails transacted by all users of the domain on that day

Restoring to active/Downloading

A single user’s email can be restored to a vault on the Active Store

A single day’s email can be restored to a vault on the Active Store

Best fit for

Restoring a single user’s data

This option is most suitable if your most common use case requires you to restore end user's mailboxes frequently.

Searching for data across users on any given day.

This option is most suitable if you need to retrieve few mail intermittently where you have the information of sender / recipient and the approx dates on which the mails were transacted OR when the restoring of end user mailboxes is a very infrequent need.


  • Select the number of months to retain data on Active. The automation which kicks off on the second week of every month, will copy data from the Active Storage to Open Storage and also delete Active Storage data that has been successfully copied to the Open Store as per the example below:
  • Let's say the value of Maintain data on Active Store for [n] months is set to 1,  and today's date is the 1st of May
  • The automation will submit LegacyFlo jobs to copy mail for all vaults of the domain to data files on the Open/Deep Store for the month of March of the same year.
  • The automation will also submit LegacyFlo jobs to delete mail for all vaults archived for the month of Feb of the same year from the Active Store. (Deletion will always be done only for those vaults whose data was successfully copied in the previous month)
  • Select the target Store. You can choose between the Open or the Deep Store. 
  • Select the data type on the Open or Deep Store. The automation will use this as the target data file type to be created on the Open/Deep Store
  • Save the settings
Note: These settings will come into effect from the next month

Monitoring

Using LegacyFlo Requests Dashboard

You can get the status of the automation from the LegacyFlo Requests dashboard

The requests are classified under Copy to Open and Remove from Active.


Using the Vaultastic Open Store or the Deep Store Application 

You can search for the data files created on the Open Store using the Open Store App

  1. Login to Vaultastic
  2. Navigate to the Open Store or the Deep Store Application
  3. Fetch files containing the vault id or month of retention
  4. Scroll down to fetch all files
  5. Download metadata or data files as required


Using the Vaultastic Usage dashboard

You can check the reduction in the size of the Active Store and the increase in the size of the Open Store using the dashboard 

  1. Login to Vaultastic
  2. Launch the dashboard
  3. Check the storage in the Open and Active storage for the different days of the month


Using the eDiscovery App

You can confirm that the mails for a particular month are removed from the Active Storage by using the eDiscovery application

  1. Login to Vaultastic
  2. Launch the eDiscovery Application
  3. Search for mail for a month. 

Moving data from Deep to the Open store

To search through the contents of a file on deep store or extract data from it, a file on the Deep Store first needs to be restored.

Once a file on the Deep Store is restored, you can either copy it to the Open Store for further action or download to your desktop.

The steps for doing these operations are documented in the Deep Store Application Guide.


Moving data from Open to Active store

If you are required to make data in the Open Store discoverable, it has to be first moved to Active Storage. We highly recommend that data should be attached to a new vault so that it can be deleted after the discovery process is complete. The steps for doing so are given below:

  1. Login to Vaultastic Admin panel
  2. Create a vault. Vault name should be in the format user-mm-yyyy. We recommend not to restore to the original vault. For example, amin-03-2019, amit.patil-04-2018.
  3. Assign Self Service access to the Vault
  4. Navigate to the Open Store app. Get the details of the LegacyFlo request to be submitted.
  5. Login to LegacyFlo
  6. Submit and execute the LegacyFlo data migration request
  7. Once you finish working with the Vault, go to the Vaultastic application and delete the Vault from the active store.



Frequently Asked Questions


Why can't the mail from the primary mail server be delivered directly to the Open Store? I do not need e-Discovery.

Using the journaling or forwarding feature, mails are transacted by the users in real-time and are delivered to Vaultastic one mail at a time. These emails are collected on the Active Store where the unit of storage is mail. The unit of storage on the Open Store is a file and hence the mail cannot be collected directly on the Open Store.


What is the minimum duration of mail on the Active Store?

If the "Select the number of months to retain data on Active Store" is set to 1, then the emails will remain on the Active Store for an average of 2.5 to 3 months. 


Why is there a month's gap between collection on Active Store and copy to Open/Deep Store?

There are two primary reasons for this. One technical and one based on the most common use cases of Vaultastic.

The technical reason is to ensure all emails are collected on Vaults and to account for any reasons of delay in mail delivery.

This is also required for most uses of Vaultastic where the latest emails are accessed most often by end-users/supervisors.


Why is there an additional month's gap between the copy to Open/Deep Store and deletion from the Active Store?

Your data is precious. Once the data is automatically copied into the data files on the Open Store, there are a lot of automated checks done to ensure that all data is properly copied to the Open Store before it is deleted from the Active Store. A month's gap between these two operations allows for all these checks to be completed. Only data that has been properly moved to the Open Store will be deleted in the coming month.


What are the minimum retention periods for the Open and Deep Stores?

The data on Open Store has to be retained for a minimum of three months. 

The data on the Deep Store has to be retained for a minimum of six months.