gary’s web sofa

At work we’d been kicking around a migration from our IMAP/SMTP hosted email service for a good while now. Our provider was top notch, but we wanted shared calendars & workspaces too, so we needed to move on. We could have deployed something like Exchange internally, but that means buying in another server, Windows, Exchange, SharePoint and anti-virus software, getting it all put together, and then spending the time maintaining it and fixing it when it breaks. Our engineering resources are a) pretty much fully committed and b) not Windows specialists, so we’d probably end up subbing it out to a contractor. We’re a small bunch (9 inboxes and another 20 distribution lists and email aliases), so the €7k+ spend over 3 years (€260/inbox/year) to deploy this solution didn’t make much sense.

Outsourcing to a managed service provider is the only thing that makes sense if you’re small and have these functionality requirements. I could sign up with someone like Intermedia on a syndicated Exchange server for $125/month. ActiveSync (mobile device synchronisation) is another $2.95/inbox/month, and there’s loads more optional, but essential extras, like additional storage. Basically at the end of it you’re looking at €150/inbox/year, which is a big improvement, especially when it means that I’ve no hardware to manage, and no housekeeping to do.

Google have had their Google Apps for Domains suite out for a year or so now. You get Gmail with your own domain name instead of @gmail.com, a calendar app, a basic browser based word processor, presentation & spreadsheet app that allows multiple users update the same file simultaneously, and a Wiki. We started using it informally when they launched the “Team Edition”, that gave you everything but Gmail, so you didn’t have to move away from your email provider. The Premier Edition gives you 25GB inboxes for $50/inbox/year, and the standard edition, with conventional Gmail adverts and a 6GB inbox is free!

We went with Google and threw the switch last Tuesday and it’s bedding in nicely. Over the past week though a couple of “wouldn’t it be nice if…” things have come up:

• Full contact information, and not just name & email address would be good.
• Proper contact sharing would be nice. Contacts are shared, but only in as much as the auto-complete function can pull addresses from other employee’s address books when you’re composing an email on gmail.com.
• The ability to sync contacts with Outlook and mobile devices would be a huge step forward.

I re-read this article James Gaskin wrote for Network World today. He’s advocating the Pirate Backup System. The work “pirate” has has nothing to do with software, music or movies that you’ve got a five finger discount on. It’s the ARR (matey!) system. ARR stands for Automatic, Redundant and Restorable.

Backups should be automatic. If a human has to do something to ensure your backups happen, then once and a while that human will do it wrong, so your backups are not 100% reliable.

Backups should be redundant. There’s no point doing a backup to a local disk, because it’s possible that the event that causes you to lose your data will cause you to lose your backup. he uses the example of Francis Ford Coppola, who had his laptop stolen earlier this year. Much of the data was irreplaceable, including a script for the movie he was about to start work on. He had a backup alright, but it was a USB hard disk. the thief stole the disk at the same time. A similar scenario I came across before was the company who instructed their sales reps to backup their laptops to CD-Rs. Unfortunately the reps had a habit of storing the backup discs in the laptop case….

Backups should be restorable. It’s an obvious one, but often overlooked. People diligently follow their backup procedure, but fail to ensure it actually works. With tapes this means frequent test restores, but tapes by their nature have a finite lifespan. The verification puts more wear on the tape and can damage the data while it’s being verified. Basically, even after you verify the tape you’re never certain that the act of verification didn’t damage the backup, so you need to verify it again, and again, and again…….

To be honest, it makes me feel a bit warmer in side that the solution my company offers ticks all the boxes….. Our online backup platform is a pure software solution, so there’s nothing to interact with. It just works in the background, and is monitored by the service provider, so it ticks the automatic box. Data is immediately transferred off-site over the Internet, so a theft or natural disaster isn’t going to take out your server and ours at the same time, so that’s redundancy taken care of. Hard disks can take a lot more read and write cycles than tape, RAID and high end storage technologies prevent data loss, and built in continuous integrity checking mean that backup data is difficult to corrupt and is spotted and corrected immediately if it happens, so data on our service is always restorable.

The Pirate Backup System – Network World

I’m setting up a VM on my VMware server to manage network backups. Asigra Televaulting is an agentless backup client that stores the data on offsite disk rather than on tape or local disk. Your backups are pretty useless if they burn up in the same fire that destroys your PC. The idea is for the VM to be in suspend until needed, thus not impacting performance of the other VMs. A command in the crontab will resume the VM in time to hit the schedule. I’ll then use Asigra’s pre/post job scripting functionality to suspend the VM when the backup is finished.
(more…)

c|net have just revisited AoE. ATA over Ethernet is a new(ish) technology developed by a startup called Coraid targeting iSCSI at the lower end of the network connected storage market. Fibre attached storage used to be the only game in town. Each server has dedicated fibre cards to connect to your storage hardware, either directly or through (expensive!) fibre switches. At work we have a *lot* of this kit, and it didn’t come cheap!

iSCSI is significantly cheaper because, in theory, you can use the same LAN to connect to your storage as you connect to the outside world. Basically standard SCSI data packets are wrapped in a TCP/IP wrapper and transmitted between hardware. Everything has an IP address, and it can even be routed over the internet. In practice however the CPU hit involved in achieving fast data transfer when every piece of data has to be processed by the CPU twice in each direction (once to wrap the data in a SCSI packet and once to wrap the SCSI packet in a TCP packet) means that the server isn’t much use for anything else. Thankfully you can get network cards that will do the process in hardware, but they’re even dearer than the fibre cards, so iSCSI isn’t a cheep option either.

AoE dispenses with the TCP layer altogether. It’s a very lightweight protocol that just uses the Ethernet network switch for routing, so you can use your server’s existing LAN hardware. Coraid have taken the decision to create a very low entry cost. It uses generic SATA disks, so all you need to buy from them is the AoE enclosure, which starts at < ???????4K for a chassis that'll take 15 disks. If it doesn't work out, you can re-use the disks elsewhere and just bin the enclosure. 750GB disks are ???????400 each, so I could deploy a 10TB array for < ???????1/GB.

I sincerely hope these guys succeed. Right now Coraid are the only manufacturer of this kit, and even though they're selling a lot of hardware to universities and research organisations (NASA are a customer), the iSCSI vendors are probably not happy with an upstart eating at their table, and may decide to kill them off, leaving customers in the lurch...

link

The “Thumper” (their name, not mine!) is out. It’s a 4U dual Opteron server with space for 48 SATA disks. Fill it up with 500GB disks and you’ve got a brand name storage box for ???????2.50/GB. It runs Solaris 10 and they’re touting it as a perfect fit for their ZFS filesystem. ZFS is being pitched as the replacement to RAID, with replication and rollback built in. These don’t have a hardware RAID controller though. Sun claim that they can do it faster in software than any hardware controller and it makes sense. A couple of 2.6GHz general purpose processors should be able to out perform a <500MHz custom part, even though they need to run an OS too.

What do you do once you need more than 24TB in a single volume. Do you put everything behind a gateway, and consolidate it there, (and deal with a performance bottleneck and single point of failure), or use them as components of an copper Ethernet (rather than fibre channel) SAN and assemble the volumes at each server (which is a management overhead, and a big CPU load on each application server)?

The other tricky thing is what to do if a drive pops? Sun are touting hot swap capabilities, so you don't have to power off the server to swap out the disk, but they're not front accessible, so you need to pull the server out of the rack and pop off the top to change out a disk. You need to be very careful with your cable management at the back of the rack to reliably pull a server forward without disconnecting power, network, KVM etc. So far we've not taken the risk and just gone with a mix of front accessible EMC units and an IBM ESS 800, which is a standalone unit, designed for high-density storage.

Still, Sun do a 60 day trial, so I might just give my friendly neighborhood Sun reseller a call ….