Michael Lopp, who writes as Rands in Repose is, I think, the greatest observer of the realities of life in the software profession. If you want to know what the software industry is like, his book is a great place to start.
The latest article on the Rands in Repose site, Hacking is Important, is a good one, for it's all about the birth of ideas and the death of organizations.
Rands is talking about the emergence of new ideas into an existing world:
I believe bringing anything new into the world is a disruptive act. By being novel and compelling, the new is likely to replace something else and that something else isn’t being replaced without a fight.
He describes the obstacles that the new has to overcome to displace the old:
Pick any company, read its history, and I’m pretty sure there will be a well-documented origin story that will define its beginning and involves someone building something new and possibly of unexpected value. What isn’t documented is the story of every moment before where everyone surrounding the hacker asked, “Why the hell are doing you that?”, “Why would you take the risk with so little reward?”, or “Why are you wasting your time?”
And he talks about companies that try to establish a culture that encourages the exploration of new ideas (e.g., Google's famed "20% time"):
Facebook’s letter documents its core values: focus on impact, move fast, be bold, be open, and build social value.
His basic suggestion is to "institutionalize" the practice of allowing talented, creative employees to freely investigate new ideas:
Someone, somewhere has a bright idea and a handful of talented engineers are whisked off to a different building behind a locked door. Their status is “elsewhere” and their project is “need to know.”This is a well-known approach, perhaps most famous in its form as Kelly Johnson's Skunk Works team of World War II. The core idea is that these people can move faster and make more progress if they aren't burdened by all the bureaucratic obstacles that the company has accumulated over the years:
The folks who create process care about control, and they use politics to shape that control and to influence communications.Of course, "process" is what you call it when you perceive it as an obstacle; "coordination" is what you call it when you perceive it as teamwork. Given the number of mistakes I've made in my career, I'm pretty solidly in the camp that prefers the support and infrastructure of a complete team; reviews catch bad ideas, design documents force you to think your work through, and testing exposes oversights and poor implementation.
One thing I think that Rands underestimates is the amount of ill-will and discord that such institutionalization creates. I've been on those hacking teams, and I've been not on those hacking teams, and I think that Rands soft-pedals it when he says:
Yes, there is internal jealousy about the teams performing the wizardry that resulted in products like the iPad, the iPhone, and AppleTV. There are people wondering, Why wasn’t I invited to the hacking? Yes, this did create some elitism, ...In my experience, the amount of jealousy, elitism, favoritism, rivalry, and discord that this sort of technique creates is substantial, and hard to expunge once it's arrived. The loss of trust is severe, and can be debilitating, and the effects can last for years. The bitterness can be cancerous.
Still, the basic technique is proven and well-known: the Skunk Works team, of course, delivered success after success during their run, and the successes of Google's 20% projects are well-documented as well.
One thing about the Google 20% approach is that it's more egalitarian (at least as I understand it); everybody is allowed to try to develop their own revolutionary ideas. There are plenty of people who will tell you, though, that most 20% projects go nowhere, and those that succeed do so due to many non-technical factors.
I don't have a better suggestion than these, and clearly it is preferable to have your employees working on their new idea as part of your organization, rather than forcing them to quit and move elsewhere in order to work on something new.
But as you implement those hacking teams in your shop, be careful to ensure that you aren't just creating a culture of the "cowboys" who can do whatever they want versus the "good citizens" who keep the lights on and the streets clean.