OK, when USA Today has a story about information management you can be sure that the phenomenon is big enough that it will impact non-techies on a large scale.

When tech analyst John Gantz at researcher IDC began tallying up all the digital information generated annually, he first looked in the obvious places …

Gantz ultimately calculated that 161 exabytes of digital data — or about 161 billion GB — were generated in 2006. And the amount is expected to rise fast.

It is worth a quick read, although if you are a data geek a lot of this is pretty elementary, so you might just want to forward the URL on to your favorite business leader or project manager.

The article is a bit light on is the downstream ramifications of the data deluge. Obviously, there will be a lot of SAN units sold, but that opportunity is long gone unless you are able to take advantage of the innovators dillema. Looking over the horizon, there is a huge opportunty for people who can make this data not just searchable, but accessible and usable and trustworthy for decision-making.

An outsourcing article on VentureBlog got me thinking. The point that the author is making is based around reinvention:

Question: Since 1995, two million American manufacturing jobs vanished. How many manufacturing jobs did China add during the same period?

Answer: None. China lost sixteen million manufacturing jobs since 1995, a higher percentage of their manufacturing workforce than the US.

Sounds interesting. We should all flexible, adaptable, forward thinking, skill-building, constant-learning individuals who are always looking to the horizon. Not only should we be pulling ourselves up by our bootstraps, we should be doing it on a regular basis.

Now, compare this to the dour assessment from the New Statesman:

Personnel managers, he finds, look not for people who are committed to and proficient at specific skills but for those who have the “potential” to cope with change. The upward career mobility so familiar to previous generations has been replaced by downward mobility.

By their late forties, people often work at lower skill levels than employees who joined more recently. And because such a high premium is put on flexibility and seizing the new, “you are constantly . . . walking away from your own commitments”.

This surely explains why everything is done so badly. You don’t try to improve your performance and do your present job well; you get ready for the next big change. It may also explain why the government seems always to be struggling to improve the population’s skill levels.

Souns a bit grim, doesn’t it? There are two sides to every coin, and it seems that our ruthlessly efficient focus on productivity improvement and global reach might have some downside.

My Take

Flexibility is important, and we need it in a rapidly changing environment. Make no doubt about it, things are moving faster and faster.  However, taking on the motto of semper gumby isn’t necessarily going to make things work.

1. The need for flexibility is often brought on by indecisiveness, negligence or incompetence, and not opportunity.  When used for the right reasons, flexibility is key.  When used for the wrong reasons it is just an excuse for additional stress.
2. Experience has a tangible value in the workplace, and the savings brought by every hour of work not done. To paraphrase a French philosopher, “A project plan is not done when every hour of work is added together, it is done when every unecessary hour of work is removed.”
3. Age matters!  It is much easier to be flexible when you are young and relatively unencumbered.  At the same time, it is easy to feel a sense of entitlement as you get older and have more to deal with.

Overall, the most useful people will be those with the right combination of drive, wisdom, experience, and flexibility to fit the situation.  A solid organization will recognize this and in itself be flexible to the needs of their employees and ultimately to the bottom line.

Media Temple is a very cool evolution in remote hosting. Part web host, part application server, part grid, it is an interesting look into the parallell computing world we are rapidly moving into.

Most interesting to me was their grid service, which provides an on-demand capability for web hosting that allows a site to handle the slashdot effect without having to blink an eye. Sites (and their corresponding media and applications) are running on multiple servers, which allows traffic to be spread out seamlessly, allowing for spikes in service and usage. This is all done without significant additional configuration, which makes it all the more sweet.

Now, they have had some problems, especially with non-grid oriented applications. However, I think that these are pretty minor compared to the utility that high-performance sites will get from using a grid environment.

Absolutely worth a look …

An interersting finding in IT World relating the people who commit sabotage in the workplace and their work behaviors. The article details that …

The research suggests that potential troublemakers should be easy to spot. Nearly all the cases of cybercrime investigated were carried out by people who were “disgruntled, paranoid, generally show up late, argue with colleagues, and generally perform poorly.”

From an organizational point of view, this seems pretty obvious. People who aren’t happy and aren’t performing are the most likely to act out. This isn’t an IT specific thing, I would guess you see the same behavior in people who work in finance and embezzle company funds.

The article goes on …

According to security management vendor Calum Macleod of Cyber-Ark, most organizations are leaving themselves exposed by “not paying due care and attention to the people who are charged with looking after their systems and applications.” Even outsourcing cannot resolve the problem fully, he said.

This sounds like a wise thing. Organizational problems are the responsibility of the organization, and the responsibility isn’t just to punish or to root out bad seeds. The responsibility is to make a culture where this sort of thing is unlikely to happen, then to monitor for any possibilities that may arise.

While it is individuals that perform these acts, they don’t happen in a vacuum and often there are components of the behavior that can be prevented cheaply, quickly, and easily. Don’t discount your people by ‘profiling’ out suspects and thinking that will solve your problems.

Openfount has released S3Infindisk for EC2, a product that answers a lot of wishes in the EC2 community. One of the biggest issues with EC2 is the lack of persistent storage on the server instances, and this tool is a first attempt at a solution.

Basically, when you are using EC2, everything on your server that isn’t statically defined before the machine runs goes bye-bye as soon as the machine reboots. Not a real problem for software, but for data this is a major bummer. Amazon makes S3 storage available without data transfer fees, which is wonderful. However, it takes real effort to transfer data back and forth between the systems (with something like jsh3ll), and most data-centric tools (especially database servers) expect real-time access to a working file system.

S3Infinidisk bridges this gap, allowing an EC2 instance to use S3 like a real Linux filesystem. While it is a bit of hack, it allows data tools to work the way they need to and it allows an EC2 instance to take full advantage of the AWS environment. This is a huge step forward in making EC2 a more usable environment for utility computing!

I haven’t had a chance to try the product out yet, but am excited to do so. I appreciate the licensing structure (free single-user version + commercial high-performance version), although I would prefer seeing open source. Also, since the tool is based on the FUSE subsystem, I could easily see this spreading like wildfire.