The Backup Window » Scott Waterhouse

Yet More About TSM Integration With Data Domain

Scott Waterhouse — Wed, 02 Nov 2011 18:00:37 +0000

I thought I was done with TSM. And just when you think you are out, they pull you back in.

Only in this case I am being pulled back in by some feedback from a colleague on my previous column. I didnt want to relegate his input to a comment where it wouldn’t get adequate notice, so I am reproducing it here in full and unedited. Full credit for this goes to Mario Correia. Mario has some impeccable credentials as a TSM administrator and consultant, and worked with one of the most respected TSM services groups in the industry. Mario had some great feedback on how often to run TSM reclamation on a Data Domain device.

“You really hit all the points with TSM and DD but I disagree on the comment about more aggressive reclamation at the 20%-30% thresholds.

Typically, 50% is the magic number where you hit the point of diminishing returns. If a tape is 50% utilized, I can take two volumes and consolidate onto one. I still have a net gain of 1 volume. But, once I go beyond that point, I’m not really saving anything from a volume perspective plus I’m also working much harder to get that minimal savings. When you throw in the actual space savings post-dedupe, it’s even less.

If we do the math using 100GB volume size, at 20% reclaim, I’d have to read data from 5 volumes or 400GB of data to reclaim that 100GB that was ‘wasted’. But if I get 10:1, that amounts to having TSM read 400GB of data to save 10GB of physical space on the DD. TSM has to read the full amount of “active” pre-comp data on that volume so it works just as hard as if it were reading from real tape.

The other potential gotcha, potential is the key word, is that TSM stores individual files in bundles called aggregates. When reclamation runs, TSM rebuilds these aggregates to free up space from expired files. Think of compacting your .pst file. I don’t have any numbers on this, but I would be concerned that we would be chipping away at our de-dupe rates because we would have TSM aggressively rebuilding these aggregates. Granted we would do better because of variable block, but I’m not sure it’s worth the risk.

My recommendation would be to use the typical 70-90% as my reclamation threshold (I think DD best practice is 90%), and just add some buffer onto your DD sizing to make up for those inefficiencies.

“Basically, I agree with everything Mario wrote — with the possible caveat that if you have TSM cycles to burn, you might want to turn up the wick a bit on reclamation beyond what he suggests.

On the other hand, there is something else interesting that happens here (which Mario alludes to at one point). Strictly speaking, TSM reclamation shouldn’t reclaim all that much physical space on a deduplicated device. Due to the way TSM progressive incremental works, there is not a lot of net new data introduced (post deduplication). So if I am doing reclamation, and cleaning up objects because there are newer versions, we need to recognize that the newer version may only introduce 10% new segments or less. So when we reclaim that file, we dont reclaim the entire capacity associated with it, we only reclaim the amount that represents unique segments–10% in this example.

So from that perspective, more aggressive reclamation doesn’t achieve the same return on investment when used with deduplicated Data Domain as it does non-deduplicated storage, be it physical tape or virtual tape. Which, I suppose, is just another argument for taking a more moderate approach to reclamation like Mario suggests–because you will just be burning CPU and IO with very marginal returns in terms of physical capacity reclaimed.

Syndicated from: The Backup Blog

More Thoughts on TSM

Scott Waterhouse — Wed, 26 Oct 2011 18:00:12 +0000

I thought I would wrap up my thoughts on TSM, for now, by talking briefly about two things TSM users are typically concerned about: migrating TSM clients and getting off TSM.

The first one of these is something we talked about just recently. And I think I covered off the easiest way to move to a Data Domain target with TSM from a tape or virtual tape target. But one of the things I frequently get asked about is: what about clients backing up using client side compression?

This practice seems to be relatively common amongst TSM users—I could speculate why, but I won’t. Suffice it to say that it is far more common in TSM environments than it is for users of other backup applications. And it is a pernicious practice that is just not good. In any way. Sure it saves bandwidth. Sort of. But bandwidth is free, more or less. And it isn’t a big enough deal to enable remote backup, so really it just saves local bandwidth. Which is even closer to free.

But aside from that, it kills deduplication. (And no, not just deduplication on Data Domain, deduplication on anything.) Because, you see, TSM client compression stays with the backup object for as long as that object is retained. Meaning that even if you turn it off, only future versions of backup objects will be backed up without compression, and all the existing versions in your TSM storage pool will remain compressed. Where they will deduplicate poorly, or not at all.

So, for all of you using TSM client compression, here is my advice: turn it off now. Right now.

OK, have you done it? Good.

No such thing as soon enough here.

Because at the end of the day, you will be moving to a dedup target, if you haven’t already. Sooner or later you will. It is inevitable. Just like death and taxes. Only a heck of a lot more pleasant than either. And the best thing you can do to prepare for this future is stop using client side compression now. At least that will minimize the amount of retained compressed data that will not dedup well.

The other issue here is how to get off of TSM. We have a large number of clients that have done this and are doing this, and one big question that comes up is: what do I do with my retained data?

Well, there is no pretty or easy answer to this question. We can put lipstick on the pig, but it still says oink. And the problem here is that whenever you migrate to a different backup application, you pretty much maroon all the data presently retained and catalogued by your existing backup application. (There are alternatives, but they are often very expensive, to the point where I have yet to see more than one or two customers out of hundreds be able to justify the process.)

So your data is marooned. The best thing is to encapsulate that TSM server in a VM and make a backup and turn it off.

And by the way, this answer isnt signficantly different no matter what backup application you choose. Generally, your data is bound to that app for the duration of the retention, and that is that.

What is better? Well, that is for the next post—why I recommend that there is a better way to do things, and my reasons for making this recommendation.

Syndicated from: The Backup Blog

Using Data Domain with TSM: Making the Move

Scott Waterhouse — Fri, 30 Sep 2011 14:08:11 +0000

Once you have decided that your traditional tape infrastructure is not serving your business needs for backup and recovery particularly well, and that your TSM infrastructure requires some updating, what’s next? As I discussed last time out, having the ability to “fix” TSM by dramatically improving the daily schedule, free up TSM server cycles, and enable better business processes—like disaster recovery testing—is a big win.

But change doesn’t always come easily.

And particularly change in the form of moving from tape to Data Domain with a TSM server can appear to be a confusing process. After all, for other backup applications that perform periodic fulls, it is “just” a matter of pointing the backup at a new target device, and letting the Data Domain system work its magic. This is almost entirely transparent to the administrator and backup application, and the fact that it is so easy is one of the big virtues of the Data Domain approach.

However TSM doesn’t do full backups for many clients, and tapes don’t expire in quite the same orderly fashion as they do for other backup applications. (Huge understatement actually. Hopefully no TSM administrators were drinking coffee as they read this as they would likely be trying to get it out of their nose or off their keyboard right now!)

So how do you move, and what is involved?

There are lots of ways to move to a new media pool in TSM, and few of them are outright bad. There is lots of flexibility, so I don’t want to present this method as the only one. But my preference is to use a process called “reclaim storage pool”. Essentially, this is a process that reclaims the entire contents of a storage pool by moving all valid retained objects to a new storage pool.

So, for the duration of the migration from tape to Data Domain, you will have two principal on-site storage pools (excepting perhaps your disk pool which is the initial backup target). One will be the old tape pool, the other a new Data Domain pool (which could be either virtual tape via a VTL or a sequential disk pool over NFS). Before the reclamation begins, the Data Domain will become the new primary on-site storage pool. All backups will end up here, either directly, if they come from a LAN-Free client, or via migration, if they go to a disk pool first. All new incremental backup data for all clients will go here. Existing retained data will still be sitting in the tape pool.

My preference is to let this run for a week to two weeks to begin to capture the most recent active data, and let a portion of the data on tape expire. Every little bit helps.

Then we begin the reclaim process on the (old) tape storage pool. Reclaiming across storage pool works exactly like reclaiming within a storage pool—except that the valid retained objects will be moved to the new pool. That is, they will end up on Data Domain rather than just another tape. The advantages of using this process are several:

You can interrupt it at any time or cancel it at the console with absolutely no impact (it can be resumed after an interruption without having to start over).
It is the same process already used to reclaim space within a storage pool, and therefore its impact and management is already well understood by most TSM administrators.
It can be done gradually: by gradually increasing the reclamation threshold, you can cause the process to run incrementally. By starting at 90%, then 80%, then 70% and so on, we can gradually move over portions of the valid data in a measured fashion. (You may want to do this by identifying all the volumes that fit the criteria, then reclaiming those volumes too.)
You can exercise as much or as little fine grained control over the migration as you like. Although the steps above are certainly a good idea, in principle, there is not a lot wrong with just setting the reclamation on the old storage pool to 0 (forcing everything to be reclaimed) and letting it go.

It is all as simple as designating the Data Domain as the target for reclaim storage pool: your syntax will determine the storage pool which is the source, the reclamation threshold value, the pool which is the target (Data Domain) and the number of parallel processes (generally as many as you would typically use for reclamation). And that’s it.

All of which brings us to the next part: what are the things to look out for in a TSM migration from tape to Data Domain? That’s up next time.

Syndicated from The Backup Blog.

Inside Avamar: Global Client Deduplication

Scott Waterhouse — Fri, 26 Aug 2011 17:55:06 +0000

I spend a lot of time talking to people about Avamar. I think it is a great application that pretty radically reinvents backup architectures. It does so many things well, and has consistently been ahead of the curve in terms of delivering advanced backup technologies. For example, Avamar was one of the first to market with a true disk based deduplication solution for backup.

And often I tell people that Avamar’s overt association with deduplication is one of its’ challenges too—in the sense that Avamar actually does two things at the client level that are really important. It is not just deduplication, but the duration and resource utilization of the Avamar client that are so impressive. Not only does deduplication reduce the amount of data to be backed up by 99% or more in many cases, but the duration of an Avamar backup is usually 90% less than the duration of a standard backup. To those of us that are concerned about growing amount of data and the impact on backup windows, this is a fundamentally important characteristic, and one that is, in my opinion, just as important as deduplication itself. (And one that I will discuss in a follow up post.)

Having said that, the focus of this post is all about the nature of global client deduplication for Avamar. This is one of the those things that I often get questions about. And they basically all come down to this: if Avamar is doing deduplication at the client level, how can deduplication be “global”? How can deduplication be cross–client in the Avamar architecture?

To answer this, lets follow the progression of the deduplication process in Avamar.

So what happens first? The first time that a client runs through the deduplication process, it will break down all the data on a system down into segments (also called “chunks” by Avamar). Chunks are variable in length, where the size depends on the type of data, and where within a data structure the piece resides. The chunk size from the beginning of a file may be different that the segment size from the middle of the file. Not incidentally, this variable length segment sizing is partly responsible for the very high deduplication ratios that Avamar can achieve.

As all those segments are being determined, they are then hashed. So every segment gets a unique hash id that can be used to identify that segment in the future. And each hash associated with a segment on a client is actually stored in two places: once on the client (in the p-cache) and once on the server (as part of the meta-data the server retains).

But here is the key: before any actual data is sent to the server for backup, the hashes for those chunks are sent to the server first, and an “is-present” operation is done. Essentially, the client asks the server: do you already have this chunk of data? Normally, these hashes are bundled together in large groups by the client to reduce the chattiness of the backup process and reduce the number of IP packets generated during a backup.

If the server determines that it is already retaining a chunk of data that is present on the client by matching the hashes, then there is no need for the client to send that data to the server. Only when the chunk is globally unique—not found on any other client that has done a backup already—is the actual chunk then transmitted to the server for retention.

So if I back up a presentation, and a colleague backs up that same presentation the next day, his backup is going to talk to the Avamar server, the server will inform him that the chunks associated with the presentation are already present on the server, and don’t need to be backed up again. All that will be sent will be the meta-data associated with the presentation (essentially, we want the Avamar server to know that the client had the file on that day, with the proper place in the directory structure, was changed last on a particular date, and so on).

In this way, Avamar can perform backups locally, but deduplicate globally. And as a result, Avamar can typically reduce network traffic between backup clients and the Avamar server by as much as 99% or more. In turn, this makes Avamar extremely effective at backing up remote offices, and desktop/laptop environments. There are few solutions which can offer greater reductions in network traffic during backup, and truly global deduplication.

Syndicated from The Backup Blog

Using Data Domain as a Target for TSM

Scott Waterhouse — Fri, 19 Aug 2011 16:00:10 +0000

I have been spending a lot of time with TSM accounts lately, so I thought I would take the opportunity to discuss some of the gotchas and lessons learned from these environments, as they moved from a tape-centric backup environment to a disk based backup environment using EMC Data Domain technology.

The first and major thing each of these customers have realized is the incredible impact the Data Domain systems have on the schedule that TSM operates under. TSM is architected a bit different (understatement of the year) than most other backup products. And for a lot of reasons, the TSM server is usually busy 24 hours a day. In fact, it is almost always the case that it could be busy for more than 24 hours a day, but there just isn’t enough time and resources to enable it to do everything it wants and needs to do.

Lets take a look at a typical day in the life of a TSM server:

20:00 to 8:00 backups to disk cache (even in 100% tape environments)
21:00 to 9:00 migration from disk cache to tape
06:00 to 12:00 copy activity to create daily off-site (copy pool) tapes
12:00 to 13:00 TSM database backup to tape
13:00 to 13:30 copy activity to create off-site copy of TSM db (copy pool)
13:00 to 16:00 migration from disk cache to tape
16:00 to 17:00 expiration activity
17:00 to 20:00 reclamation activity

Now I have simplified this a bit: normally some of these activities are interwoven. Migration from disk cache to tape is often an ongoing process. Reclamation often spills over into the backup window. But overall, it is a fair generalization to say that almost every TSM server out there using tape needs more hours than there are in a day to get through this activity.

So what usually gets left out? Reclamation. 98% of the time, the shortfall in time and resources is made up by reducing the amount of reclamation that happens. In turn this means that more and more tape is consumed, and the density of data on tape drops. In TSM environments, it is not unusual to see less than 50% of the total tape capacity used by current valid backup data. In some cases, I have seen 70-90% of tapes wasted on unreclaimed data.

Now lets look at a typical schedule after implementing a Data Domain system:

20:00 to 4:00 backups to disk cache (still sometimesa required even with Data Domain)
20:00 to 4:00 backups to Data Domain from LAN-free clients
21:00 to 6:00 migration from disk cache to tape
6:30 to 7:30 TSM database backup to Data Domain
8:00 to 11:00 expiration activity
11:00 to 15:00 reclamation activity

So what has happened? First, we have moved larger backup clients to a LAN-free method, sending their data directly to Data Domain. With the much larger number of virtual resources we have available in a Data Domain system (up to 256 virtual tape drives–or 512 on a GDA) than most environments ever have access to in the physical world, we can make this simple architectural change which is enormously beneficial.

Second, we have got rid of the copy pool activities. This is a huge drain on the time and resources of the TSM server during the course of the day. By eliminating this entirely, and replacing it with Data Domain replication, we save many hours of processing. Incidentally, we also reduce the size of the TSM database (because we don’t have entries for every backup object retained offsite and every version of them retained offsite). We reduce half of the reclamation workload of the TSM server, because it does not need to do reclamation processing against the copy pool. We reduce the CPU and I/O load on the server. These are all very good things.

Third, database backups and offsite copies are complete by 7:30 in the morning. This means that a full disaster recovery copy of the TSM backup pool and database is available for disaster recovery purposes many hours earlier in the day than they would be with physical tape. In fact, depending on the duration of the copy pool job and the timing of your couriers, off-site tape may not make it off-site for 24-36 hours for some TSM users. This means, by the way, that the best RPO that can be achieved is 72 hours. With Data Domain, we have a RPO of no more than 30 hours.

Finally, reclamation is going to run far faster. In general, reclamation from virtual tape on Data Domain is going to run four to ten times faster than reclamation from physical tape. In turn, this means that we can be aggressive with our reclamation policies, and reclaim a tape when it has 20-30% expired data, rather than 70-90%. In turn, this makes for far more efficient use of our backup infrastructure.

The net result here is that we have taken a typical TSM server from requiring 30+ hours to get through its daily activities, to 18-20 hours to get through the same activities.

These are are the first and most significant benefits that our customers are realizing when they pair TSM with Data Domain technology. Next up: gotchas and approach.

Syndicated from: The Backup Blog

The Next Step Forward For Mainframe Virtual Tape: DLm6000

Scott Waterhouse — Mon, 08 Aug 2011 16:21:51 +0000

Mainframe tape is a funny thing. Or at least it can look funny to a person that doesn’t work with mainframes.

Funny because mainframe tape does a bunch of things. In the open systems world, life is simple by comparison. Tape does (or did!) basically one thing: backup. Yes, you could argue that it was used for archiving too, but that argument largely depends on the notion that backup and archive tend to be the same thing for open systems folks. Which isn’t necessarily a good thing, but it is certainly the case more often than not.

In the mainframe world however, tape does at least four things: backup, archive, batch processing, and migration (for DFHSM). Now some of these, it could be argued, are sort of suited to tape. And some are not. In fact, as time has moved on, the value of tape to each of these has degraded.

And as innovation has occurred, that value has degraded even further. Because mainframe tape is very much like open system tape in one key respect: it isn’t very much fun. It is unreliable (relative to other IT systems), it is insecure, it is slow, and it is an operational nightmare. And on top of it all, it makes disaster recovery planning, testing, and execution really really difficult.

Now, it has persisted longer in the mainframe world, for two reasons. One, there has been relatively little innovation, in the sense of a dramatic changes that completely changed perspective and technology on tape. Yes, VTS systems came along. And no, they weren’t all that interesting or radical—they were largely a relatively small disk cache in front of a whole lot of physical tape. Secondly, I don’t think it is any secret that mainframe people are a little more resistant to change than open systems people (and when you run the world’s most stable platform, why wouldn’t you be?).

So the status quo has persisted for a while. However, several years ago now EMC introduced a new product, called a DLm, or Disk Library for Mainframe. And the Dlm was different than anything we had seen before, because it didn’t use tape. It wasn’t just a cache for physical tape. Yes, you could still support tape if you wanted to, but the Dlm didn’t use it, need it, or require it in any way.

And today, we are updating the product again. The latest third generation release is the Dlm6000. Just like previous versions, the Dlm6000 doesn’t use any physical tape. It is a disk based replacement for tape. It uses disk: both primary disk and deduplicated disk. So it is here that the DLm6000 takes such a big step forward from the previous generation in integrating deduplication functionality: not only do you not need tape, but you don’t need nearly as much disk either.

The mix of the two will depend on your requirements, but by offering a mix, we can offer both performance and capacity. Performance that is better than 2x that of our competitors. And capacity of up to 5.7 PB per system. All of which can be managed from the Z/OS console—meaning that determining whether data ultimately ends up being deduplicated, or not, ends up to be trivially easy. As does determining which data gets replicated.

So the DLm offers a unique platform: a single virtual tape system that can fulfill all of the requirements that tape has traditionally met for mainframe users: it can be a target for data used in batch processing, it can be a target for data migrations performed by DFHSM, it can be an archive platform, and it can be a target for backups.

One platform, that does it all. And doesn’t rely on tape.

On top of that, the DLm is, I believe, the only mainframe tape virtualization product today that has no single point of failure, with no metadata that can be “lost” in the event of a component failure. Which helps contribute to another piece of functionality that is near and dear to my heart: the ability to do disruptive/destructive disaster recovery testing without impacting primary data functions, or replication, in any way. (This type of functionality is also found on the Data Domain platform, and in my opinion is hugely important. If you can’t test your DR plan without disrupting data replication, it is just not acceptable. If you have to break the replication, then resynchonize systems, and wait for replication to catch up after a DR test, this is just not good enough. And that isn’t a process you should accept from a modern tape virtualization or disk backup target.)

And in the spirit of just one more thing, the last major improvement the DLm brings is non-disruptive code upgrades.

Goodness.

Syndicated from: The Backup Blog

The Growing Digital Universe: What It Means For Backup

Scott Waterhouse — Tue, 28 Jun 2011 15:56:10 +0000

Over the last little while, we have become used to seeing discussions of data and storage for Big Data. As if Big Data possesses a unique set of challenges around the acquisition, management, replication, and protection of the data. And so it does.

For now.

Because after reviewing the latest set of statistics from the 2011 Digital Universe study it is clear that the challenges (or “opportunities” as Chuck put it on a recent call) faced by Big Data users today are going to be the same challenges that most enterprise IT shops are going to have face in the years to come.

Because there won’t be Big Data and everybody else, there will be Big Data and Really Big Data. So for the long term thinkers in the backup and recovery community, I think there is some value in paying attention to the problems that Big Data users face with respect to the backup and recovery of their data sets, because their solutions and their answers are likely to shape enterprise data protection in the years to come. Generally speaking Big Data can be separated into two types—structured and unstructured data. And one of the things that the Digital Universe study makes clear is that unstructured data is growing faster than structured data.

The striking thing here is that unstructured data is growing on a path similar to that of Moore’s law: doubling every eighteen to twenty four months. Structured data is not growing as fast, but it is still growing faster than staffing.

And one of the key things that emerged out of the latest Digital Universe study is just that: staffing is not keeping pace with growth. But moreover, over the next ten years, there will be:

10X the number of servers (virtual and physical) worldwide
50X the amount of information managed by enterprise datacenters
75X the number of files
Only 1.5X the number of IT professionals in the world

So it is time to ask ourselves: what does this mean for our approach to backup? Could your present backup infrastructure and methodology deal with 10 times the number of servers that your currently protect? Could it scale to protect 50 times the amount of information? Could it deal with 75 times as many files or objects?

I think there are actually two sets of answers to those questions: one technical and one based on business or cost considerations.

First, the technical: For most organizations, I suspect the answers are yes, maybe, and no respectively. Scaling the number of clients is a not such a big challenge for most backup platforms. Scaling to protect 50 times as much data is probably not such a big issue so long as your backup target platform can keep up: and I wrote on that particular issue here.

Scaling to protect 75 times as many files however is an entirely different challenge. As any good backup administrator knows, dealing with large unstructured repositories of data doesn’t challenge you in the normal way. Throughput is usually limited not be the overall size of the data and the platform it resides on, but by the ability of the operating system and backup application to process the file system and metadata. File servers with 20 or 50 million files don’t take days to back up because of the amount of data associated with them, they take days to back up because it takes that long for most backup applications to sort and process the metadata the file system/operating system is providing them. And with the exception of Avamar, which tends to back up large unstructured respositories about 10 times faster than any other backup application, there is just no relief in sight for this particular problem.

From a business point of view however, the answer may well be the same: yes, maybe, and no.

Yes, because presumably as the number of servers we protect grows, so will the capacity of an individual backup server. My cost will remain more or less static in that respect.

Maybe, because as I discussed in the other post I referenced above, so long as your deduplication target is capable of sustaining Intel curves (Moore’s Law) with respect to throughput and capacity, and not Seagate curves (doubling every ten years) then your backup infrastructure will be able to keep pace with your structured data growth. Again, costs will remain more or less static over time. Continual investment will be required, but the magnitude of that investment will not change dramatically over time.

And no because the cost of protecting an unstructured data store that is 75x larger than today’s unstructured repositories may be no longer justifiable.

An anecdote illusrates this. Recently I was working with a large customer that was spending about $12m to $15m annually on backup. They projected that in 3 years, if they didn’t change anything with respect to infrastructure, methodology, or practice, that would grow to nearly $125m. Something had to change. Cost growth of that magnitude wasn’t acceptable to anybody in the organization. Moving away from tape to deduplicated disk was part of the change, but so was a change to practice and policy. And here is the heart of the problem: as tactical problems become strategic problems (and you better believe that a data protection bill for $125m would be a strategic problem for the business) we will need to rethink how we protect data, how we back it up, and what retention policies are appropriate.The next question is, of course, how we accomplish that? How do you work with your business to determine what an appropriate, and appropriately costly, protection mechanism for data is? There is really only one (good) answer to this question in my opinion: what is the data worth?

Let me make one final observation, largely based on personal experience and not founded in any exhaustive or statistically rigorous survey: the big data consumers of today tend to have a very exact understanding of the value of the data they own. Most organizations that don’t have Big Data, even if they have very large amounts of data, dont have any idea how much their data is worth. Genomics companies, Geology and Geophysics companies, Big Media companies–all of them understand very precisely what their content is worth.

And they can pick an appropriate data protection strategy. Maybe spending $125 would be appropriate if the data was worth $20bn. (It wasn’t in the case above.)

But I have yet to work with an organization outside of the Big Data owners, even those with multiple PB of data, who know what their data is worth. So if you don’t know what it is worth, how will you determine what the best backup strategy is? How will you be able to defend the current approach, or change to a new one? Alternately, you may be faced with a bill that will one day escalate to $125m. In that case, it is pretty easy to figure out that you have to change something. But what will you change to?

In my opinion, these are the big tough questions that the expanding Digital Universe will force us to ask.

Syndicated from: The Backup Blog

EMC Avamar v6.0

Scott Waterhouse — Mon, 18 Apr 2011 16:26:04 +0000

Today EMC is announcing a new version of EMC Avamar software and hardware. With the latest release come a ton of new features, bigger, better, faster hardware, and support for integrating Avamar with Data Domain.

So, without further preamble, let’s jump into what is new in Avamar v6.0:

On the hardware side, the new release offers higherdensity nodes with greater performance. Nodes are now offered in 4 different capacities: 1.3 TB, 2.6 TB, 3.9 TB, and 7.8 TB. The standard Utility and Accelerator nodes continue to be available and serve the same function in Avamar v6.0 as they did in previous releases. What is more, is the performance of the node relative to important internal system maintenance and upkeep scales with capacity. The 7.8 TB node offers twice the performance of the 3.9 TB node. For some important tasks like garbage collection and file system checks this means there is no longer any good reason not to use the highest density nodes even for data with exceptionally high change rates.

Even better, prices have declined while capacities have gone up-in some cases the cost per TB has improved by more than 20%. And the power consumption per TB has also dropped considerably-by as much as 65% in the case of the 7.8 TB nodes. All this equals bigger grids which cost less to acquire, less to power and cool in the data centre, and use less of your valuable raised floor square footage to store data.

Under the covers there are lots of significant performance improvements to the new nodes to make sure that things work better than before: EMC has leveraged the Intel multi-core architecture to speed up startup and checking of data stripes. Memory utilization is improved, and overall backup performance increases.

One of the biggest changes that many customers willnotice will be the changes to the networking architecture. The lines between internal and external networks are no longer blurred. There is a network that is clearly an internal management and administration network (which is redundant). And customers will connect an Avamar grid to their external network to provide for backup bandwidth and redundancy if desired. (As an aside, for those of you that want Cisco everywhere in your network, it is now possible to connect your Avamar Data Store nodes directly to your Cisco network. You can maintain the homogeneity of your IP network and continue to manage and support only Cisco devices.)

On the software side, the improvements are every bit as significant:

Support for EMC Secure Remote Support (ESRS)
A new installer/updater service for client and server code upgrades and patches that is customer accessible.
An Avamar client manager that enables management of large groups of clients with a single pane of glass, and allows wizard driven task execution against groups of clients. (Another aside: it also allows moving a client or a group of clients from one Avamar grid to another. For those of you managing large numbers of desktop and laptop clients, or desktop and laptop clients for a globally distributed workforce, this will be a particularly welcome addition that will make administration a lot easier.)
Overall, there have been a least a dozen significant enhancements (and many minor updates) to the desktop and laptopbackup experience with the Avamar DT/LT client that will make Avamar easier, faster, more flexible, and easier to support for these environments. From an end user perspective, the self-service recovery window has been improved, which is a welcome upgrade. From an administrative point of view, the administration of large numbers of clients has been made easier too.
Improvements have been made across a wide range of general clients, and specific modules, including the backup of Oracle systems, Oracle RAC systems, NDMP backups, backups for Iomega ix12 systems (a very cool feature which essentially puts an Avamar client on the Iomega storage system for powerful remote backup of these systems), VMware Image Backup enhancements, Windows 2008 server backups, SQL server systems, Exchange backup, and a host of others. I will return to the topic of Windows applications backup and VMware backup in the near future, because there is enough good stuff there to warrant a separate discussion.

And, to leave the best for last: Avamar is now offering integration with Data Domain systems.

With release 6.0 of the Avamar software, EMC will now support Data Domain systems as a backup target. This combines the performance and scale of the Data Domain systems with the ease of use and simplicity of the Avamar software. Now any application, of any size, can be protected with Avamar. And the scale of Avamar grows very significantly-with the addition of up to 285 TB of capacity per Data Domain system (and up to 4 systems supported per Avamar server).

In the initial release, EMC is supporting the backup of Oracle, SQL, Exchange, SharePoint, and VMware Images to Data Domain systemswith Avamar software. The Avamar GUI will manage the backup, restore, and replication of data for these systems, as well as monitoring and reporting on the Data Domain systems that are the backup targets. Avamar clients get an embedded DD BOOST v2.3.1, which is responsible for actually sending the backup data from the client to the target Data Domain system. Now, even very largedatabases and database with very high daily change rates can be easily integrated into Avamar.

With that, I will conclude my overview of Avamar v6.0. As you can see, there is a lot to like here. And a lot more that I simply didn’t have the space to go into with this post-so I will be returning to the subject in the near future.

Syndicated from: The Backup Blog

EMC Data Domain Archiver Webcast

Scott Waterhouse — Fri, 04 Mar 2011 16:46:09 +0000

EMC is hosting a webcast next week on the topic of simplifying long term retention of backups and archives.

This webcast is available ondemand. [Original airdate: Tuesday, March 8, 2011 at 8:00 am PT / 11:00 am ET / 16:00 GMT.]

For fast and reliable long-term retention, integrate EMC Data Domain Archiver into your backup and archive environment today.

EMC Data Domain Archiver is the industry’s first system for long-term retention of backup and archive. This single deduplication storage system can address your operational backup needs and long-term data retention requirements, eliminating the costs and complexities associated with tape-based backup and archive.

Attend this webcast and discover how Data Domain Archiver lets you take advantage of:

Cost-effective scalability for long-term retention with up to 28.5 PB of logical capacity
Fast, inline deduplication with up to 9.8 TB/hour throughput
Minimized tape costs and simplified data management
Easy integration into existing environments regardless of backup or archive software

Syndicated from: The Backup Blog

Scaling Up and Out: The New Data Domain DD890 and GDA

Scott Waterhouse — Tue, 18 Jan 2011 16:48:35 +0000

Today EMC introduced the industry’s fastest deduplication storage systems. And they are fast. Big and fast. Impressively so, and even more impressively if you have been involved with backup systems for a while—in a “look how far we have come” sort of way.

But to focus on just the speed and scale would be to miss a lot of great functionality that our engineering teams have worked very hard to include in the latest software and hardware release of the Data Domain systems. In fact, this may be one of the biggest releases ever for new functionality and new capability with the release of Data Domain Operation System (DDOS) 5.0.

Before I go there however, let me spend some time wowing you with just how big, and how fast, Data Domain systems have become. How about 175 times faster than the first Data Domain system introduced in 2004. How about 450 times more capacity than what was available 7 years ago? A little while ago I asked the question: “What curve are you on?” The answer to that question is all about the relevance of increasing the speed and capacity of backup systems at a rate that is at least equal to the rate of data growth for most organizations. Simply speaking, if backup systems are not capable of keeping up with data growth, the architectural model of those backups systems has a notable weakness. At very least, the number of those systems required to keep up with your data growth will increase over time; adding cost and complexity to the environment. Exactly the wrong direction to be moving in.

So if it seems like we spend an inordinate amount of time on this notion, there is a very good reason why: it is only through speed and capacity increases like EMC has been able to achieve with its Data Domain systems that we can move in the opposite direction. The direction of growing performance faster than data. By growing performance and capacity faster than organic data growth, we are actually reducing the number of systems that a given environment will require year over year for backup purposes.

So just how big and how fast are the new systems? The newly introduced DD890 now scales to 14.7 TB/hr backup ingest, and up to 285 TB of useable capacity. The Data Domain Global Deduplication Array (GDA) scales to 26.3 TB/hr and up to 570 TB of useable capacity. For the average customer, that would equate to the ability to backup more than 200 TB in an eight hour backup window, and retain more than 10 PB of data.

What is more is that the GDA has made a major step forward in terms of usability and broad market appeal, by adding support for the Data Domain VTL software option. Previously, if you wanted to deploy a GDA, you had one choice on what protocol to use: DD Boost. Now that has changed, and the GDA is useable with either DD Boost or FC (VTL) interfaces. As a result, it is useful for TSM environments, and almost every other major backup application. In my experience, this is doubly important because it is those big IT environments that are likely to be interested in the GDA that are also most likely to be heavily invested in FC infrastructure for backup.

Two other major connectivity improvements have been made with this latest release of DDOS too, and are therefore generally available for all Data Domain systems: IBM i connectivity and NDMP connectivity. With DDOS 5.0 we are now supporting the direct attachment of IBM Power Systems running IBM i to Data Domain systems. Those systems can be shared between IBM i and open systems too, so no longer does the IBM i need its own dedicated (and costly) backup infrastructure. And it can leverage the network efficient replication capabilities of the Data Domain system to provide for fast, and cost effective, disaster recovery.

Also new on the connectivity front is the addition of NDMP over Ethernet functionality. Say goodbye to redirected NDMP backup streams: the Data Domain system can now be a direct target, over IP, of the data path backup data of an NDMP backup. Again, the functionality works with all major backup providers and NAS systems from both major providers.

Then there are a host of lesser improvements in DDOS v5.0 that contribute to this being a very significant release indeed. Some of these are worthy of further discussion—and I will do just that if questions or comments arise on specific capabilities—but briefly the new functionality includes:

Enhanced replication management
Enhanced reporting capabilities
Role based administration/access control
Improved remote management
An improved configuration wizard
All new functionality in DDOS is available via the CLI
Enhanced SNMP functionality DD Boost with encryption for encrypted optimized replication with BE, NBU, and NW
DD Boost load balancing and link failover enhancements
Encrypted replication
LACP support for increased performance
Myriad and sundry smaller hardware, connectivity, and OS enhancements

All in all, this is a very significant release, and a very significant step forward not just for performance and capacity, but for the DDOS itself.

Syndicated from: The Backup Blog