I recently posted an article on OnStartups about HubSpot's Experiments Framework. I've reposted some of the relavent parts below. The full article can be read at OnStartups.com
The foundation of this framework is based heavily on Clay Christensen’s work inThe Innovators Dilemma. We’re huge Clay Christensen fans at HubSpot (even have a conference room named after him) and have been life-long students of his work. In his work Clay asks a very straightforward question without an obvious solution - specifically: Why do well managed, successful companies repeatedly fail to create new disruptive innovations?
This framework was developed fundamentally to combat that challenge and create a lasting culture of entrepreneurial exploration.
HubSpot’s Experimentation Framework
The framework has 3 stages, each with a distinct goal and approach.
Alpha – Lowering barriers to experimentation
No bureaucracy, no red tape, full access to information. This stage is simply focused on enabling anyone with energy and an idea to try a new solution. Tests are run by everyone and anyone – but are generally done in spare time (nights and weekends) and with few resources. You don’t need to ask permission to run these tests - and by design no one ever knows all the alpha stage experiments actively being pursued. It's open and distributed.
Beta - Determining proper funding
When an experiment reaches Beta stage the ‘founders’ are fired from their day job and work on the experiment full time. While founders determine their own goals and metrics – these leaders are encouraged to be patient for growth but impatient for profitable economics. Like many founders these people also report to a 'board' regularly and are subject to evaluation on future funding. At its core this stage is about providing access to funding for entreprenurial folks with new ideas and transparency/accountability into the success of those early tests.
v1 – Scaling successful experiments
v1 projects have proven economics and now are looking to scale the success. Often this requires growing the team beyond the founders, building dedicated systems and developing regular tracking of core metrics. Founders with experiments graduated to v1 are now considered 'mini-CEO’s' and are tasked with running their project as a start-up within HubSpot.
We established this framework in the hope of driving innovation and empowering the entrepreneurial edges of our organization to create change. It seems to be working - we’ve had several successful founders graduate from the program (Pete Caputa with VAR program, Jordyne Wu with the Services Marketplace) and we created a culture to be proud of. It's enabled us to focus on the core business without foregoing the entreprenurial engery and creativity of our team.