A guest post by Tamir Segal, Senior PM
Snapshots is a technology that enables creating copies of volumes. XtremIO has developed a snapshot embedment that provides the ability to create space efficient snapshots that are managed as standard volumes in the cluster and have the same performance capabilities and data services as production volumes. In this post I provide a general explanation on legacy snapshot implementation, and explain what makes an XtremIO snapshot different.
Some Background
Legacy implementation of snapshot was based on technology called Copy-On-First-Write. The idea of Copy-On-First-Write is that once a snapshot is created, a new storage pool is defined in the system; every write to the production volume triggers a data movement operation to the snapshot pool:
· A new write is received by the system on LBA X
· The system reads the original data on LBA X from the production volume
· The system writes the data to the snapshot pool on LBA X
· The system overwrites LBA X on production with the new data
This means that every write has penalty of an additional two I/O operations. One would think that in high performance media such as SSD that this approach is not problematic; not true! This still has significant overheads in terms of data movement management; it impacts on latency for write and read operations, limits the snapshot on snap performance/capability, and has a negative impact on the SSD media endurance.
The next evolution in snapshot technology solved some aspects of the performance; the implementation of Redirect-On-Write. This method requires metadata management; for each LBA (or a range of LBAs), a pointer based system exists; for every new write, the system writes the data only once and updates the metadata to point to the right location of the data. In some implementation of Redirect-On-Write, the new data added to the production volume is kept in the snapshot pool, meaning that when deleting a snapshot, the production data needs to be moved back to the production volume storage area. Furthermore, when multiple snapshots are created, access to the original data, tracking of metadata changes in the entire snapshot tree, and reconciliation upon snapshot deletion results in a heavy performance penalty.
Other technologies are used to get a copy from volumes. Clones provide a full copy of the data, typically taken from a static snapshot. Split mirrors are used to create clones with more efficiency from a dynamic source. Both technologies offer good performance, however creation and refresh time is very long with either option.
Historically, snapshots were usually created and used for a short period of time, typically to create a copy of live production data for backup purposes; Admins would freeze the production application for a short period of time, take a snapshot and resume normal operation. This resulted in a static copy of production data typically accessed in Read-Only mode that could then be backed up to an external backup device. Reasons why snapshots were used for short periods of time include performance issues, capacity utilization considerations, and the limited number of supported snapshots.
Later, following the development of Redirect-On-Write technologies, snapshot usage was extended for longer periods of time, mainly for testing and development processes. However, snapshot usage was limited because in some cases, performance was impacted; either due to Copy-On-Write implementation of snapshots having a high penalty on performance for the production environment, or Redirect–On-Write, which introduced increased read-latency as the result of scanning of linked metadata data in the array, or overheads on the controllers’ CPUs.
It is important to note that this performance issue cannot be mitigated in a legacy dual controller architecture, let alone an active-passive architecture. The main reason is that per volume/snapshot, only one controller is involved in its management. This is not scalable, and a large number of snapshot and production volumes create a significant overhead on the controller that impacts the performance on snaps, production volumes, or both. XtremIO, however, provides an advanced scale out architecture; the Storage Controllers in the cluster are constantly engaged in managing the IO data flow and metadata, regardless of the entity type (snapshot or production volume), meaning more CPU power and memory remains available (that is; a single controller, verses multiple controllers), and the workload is always evenly distributed over all available resources.
How Are XtremIO Snapshots Different?
An XtremIO snapshot is tightly integrated into the XtremIO array architecture, thus benefiting from the in-memory metadata and dual stage metadata with the additional advantage of an efficient data structure to support the fast creation of a snapshot that provides the same performance as production volumes, both on reads and writes, regardless of the snapshot’s tree span or depth. XtremIO implements a variation of Redirect-On-Write snapshot architecture; where all redirection of blocks are managed in the array’s memory, making it flexible, fast and efficient. Furthermore, any writes to any production volume or snapshot in the cluster goes through the same process; it is fingerprinted, deduplicated using XtremIO inline deduplication, and written to the global storage pool. The inline dual stage metadata manages pointer updates in the same way, regardless of whether or not it is a write to a volume or to a snapshot. This means that the performance challenges of standard Redirect-On-Write are not applicable for XtremIO. Combine XtremIO’s unique snapshot technology with an easy-to-use, hassle-free snapshot management methodology and you get the XtremIO snapshot.
Another algorithm implemented in XtremIO’s snapshot technology is the capacity utilization optimization feature, called remove shadowing writes. Assume a user writes the whole address space of volume P (Step 1), and then creates a snapshot on P (Step 2). Note that P and the snapshot do not hold any metadata or capacity, as no new data has been written to them yet. Assume new data is written to LBA 4 of both the snapshot and production (Step 3).
The data and metadata in the ancestor at LBA 4 is no longer needed, as it is shadowed by the new writes. It can now be freed from the cluster (Step 4). This method provides better utilization of physical capacity and improves the cluster’s overall resource utilization.
Step 1 – Write over the entire address space of Volume P
Step 2 – Create a snapshot on Volume P
Step 3 – Overwrite LBA 4 on Snapshot and Volume P
Step 4 – Remove shadowed writes, free metadata and physical resource of LBA 4 from the ancestor entity
What Does It All Mean?
Firstly, the XtremIO approach to snapshots is different; any snapshot taken in the cluster appears as a read-write volume. It is managed in the same way as any volume in the cluster; mapping or deletion is performed the same way as for a production volume.
Secondly, snapshots have the same data services as volumes; particularly; the inline global deduplication and thin provisioning features are in constant operation.
Thirdly, snapshots are efficient; they only consume memory for new writes and physical space for unique data which cannot be deduplicated. The creation of a snapshot is fast, as no metadata or data movement is involved during the creation process.
All Storage Controllers and resources in the cluster are always involved in the data flow, both for the volume or any of its snapshots. All resources are constantly evenly distributed across the available resources to ensure no bottlenecks exist in the cluster.
In addition, snapshots are fast; you get predictable, consistent performance on production or snapshots.
So Snapshots Are Managed Like Volumes; Why Do I Care?
You care because it means that snapshot creation and management is hassle-free; you do not need to go through a dual process of instantiation or cloning a snapshot in order to be able to access it. You do not need to manage pools to reserve space for snapshots. You do not need to define data services on snapshots. Instead, you create a snapshot; it appears as a volume in the cluster and you only need to map it and; voilà, you have a high performance, read-writeable copy of your data, which is accessible by your server.
Creating a snapshot is a simple ‘one-two-three’ process:
Step 1: Select the source entity (volume{s}, snapshot{s}, set of volumes/snapshots, or a folder, right–click, and select Create Snapshots.
Step 2: Give the snapshot a name, select the destination folder.
Step 3: Map the snapshot in the same way as you map any volume in the system.
Note: Snapshots appear in the same screen as volumes in the UI. The only differences are the icon associated with the snapshot and the fact that it has an ancestor entity, as shown below:
No Performance Impacts? Is it Really True?
Yes it is. XtremIO uses a linked data structure in the array’s memory to manage all the metadata required for snapshots. Only new writes to a snapshot consume metadata. The non-written blocks are either unallocated or are used from the linked data structure of its ancestors. This means that there is no metadata copy operation involved for the process to create a snapshot. It also means that the creation process is fast. In order to avoid complex in-memory scans of metadata, the array manages a bitmap-like data structure per LBA which facilitates a shortcut map to where data resides in the linked data structure. This means that read operations take the same time, regardless of the snapshot hierarchy (span or depth), and has the same read and write performance as the production volume. No configuration, manual optimization or special setting is required to enable it.
So, Can I Create a Snapshot of a Snapshot? How Does it Work?
The answer is yes; you can achieve a full cascading of snapshots without limitations. This can be very deep, span widely, or be anything in between. As snapshots are managed just like volumes, a snapshot can be created from any volume or snapshot in the system without limitations. The result of a snap of a snap is yet another volume in the system that can be accessed in read/write mode.
To create a snapshot of a snapshot, right-click the snapshot, select Create Snapshots, and give the newly-created snapshot a name.
The newly created snapshot appears in the cluster under the Volumes section:
You can also choose to view the snapshot in hierarchical view; done by right-clicking any volume, and selecting Show as Snapshot Hierarchy, as shown below:
This displays a hierarchical view of the snapshot tree, as shown below:
There is no limitation on a snapshot’s topology. It can be as deep or as wide as needed, as shown below:
My Application Runs on Multiple Volumes. Can I Create a Cross-consistent Snapshot?
Cross-consistent snapshotting can be achieved either by selecting a set of volumes to be snapshotted in a single operation, or by snapshotting a folder; an entity that groups a set of volumes together. The result of this operation is a snapshot per volume in the set; each snapshot appears as a volume in the cluster.
Option 1: Select a set of entities for cross-consistency snapshotting:
Option 2: Snapshot a folder
Obviously, CLI and REST can be used to create a cross-consistent snapshot. Using CLI or REST enables integration with the application or a more sophisticated management methodology for snapshots.
What Happens to a Snapshot if I Delete the Production Volume?
You can delete any volume or snapshot within a snapshot hierarchy without limitations; all other snapshots in the hierarchy are not impacted. This holds true even when you delete a snapshot nested within the hierarchy. All snapshots taken before or after the snapshot’s creation remain in the cluster.
Deleting a volume nested within the snapshot hierarchy:
 |
 |
Furthermore, deletion of a volume/snapshot that is nested within the hierarchy has no performance implications, and does not slow down the cluster (IOPS or latency are unaffected).
So, What Are the Supported Use Cases for Snapshots? Where do you Shine?
Snapshots can be used for a variety of use cases, including those listed below:
· Backup – Use snapshots as a means of backup operations to an external backup device
· Protection against logical corruption – Create multiple snapshots over short intervals. Use the snapshots to recover from logical data corruption.
· Test and development – Create multiple copies (snapshots) of the production for Test & Dev purposes
· Offload processing (ETL, near-real-time analytics, etc.) – Frequently (or as often as needed), create a copy (snapshot) of the production data, and access it for offload processing, enabling non-production server resources to gain fast access to a fresh copies of the production data
Leaving backup and data protection aside, XtremIO’s snapshot solution has a very strong offering for test and development, offload processing, and massive consolidation of production data with analytics and test and development use cases.
In offload processing, you can offload ETL or analytical processes from the production server to a different server. As snapshots can be easily created and refreshed with new data, you can:
1. Create snapshots in relatively short intervals
2. Achieve Flash performance IOPS and latency from your copies of the production data
This provides you with the benefit of consolidating production with other business-critical applications and running ETL processes or near-real-time-analytics without consuming resources from the production server at a very high level of performance.
XtremIO is the Perfect Test & Dev Machine. Why?
The reason is a combination of the below factors:
1. The inline-deduplication and thin provisioning savings
2. The large number of supported snapshots
3. The unlimited ‘all-you-could-wish-for’ snapshot topology support.
4. The ease of refreshing the copies
Multiple copies can be created and deleted without limitations. As development data tends to deduplicate well, you benefit from the reduced physical capacity consumption.
The example shown above is a Test & Dev implementation plan based on an actual customer environment. Supporting such a requirement in a legacy array results in hundreds of terabytes and has major performance limitations and a very long refresh time. The cost of maintaining a large number of copies on a non-XtremIO device (as illustrated above) is very high.
The above configuration can be supported with dual X-Brick configuration (based on a 20TB X-Brick). This particular customer was saving hundreds of terabytes in a high-end array at a cost of implementing Test & Development. The savings in costs for the hardware in addition to the improved service (performance, operation, management, etc.), and the ease of use make the solution’s return on investment very attractive.
Furthermore, the performance and ease of snapshot usage may create a new positive challenge. One of our customers who recently implemented Test & Dev over XtremIO received a new complaint from his team; “Our Test & Dev is much faster than our production array, we’ll need to fix that….”
XtremIO Snapshot Demo from itzikr on Vimeo.
