It’s important to know the most efficient way to arrange your optimization team to ensure their productivity and yours.
But what’s the best way to structure your team? Should optimization folks be in a separate team? Or under product teams? Or marketing?
There’s several different ways, and choosing which one works best for your company can be challenging. Here’s a rundown of the frameworks, their functions, and the benefits and challenges of each.
What kinds of teams are there?
There are three types of optimization team models: centralized, decentralized, and center of excellence.
The centralized model is based on one team of employees, typically data scientists, and has the benefit of localizing expertise so that employees can develop a long-term optimization strategy, including developing better statistical algorithms and experimentation tools.
In decentralized (or “embedded”) teams, optimization responsibilities are distributed amongst employees under many different umbrellas within the organization, rather than one department.
The center of excellence model is basically a combination of the above two frameworks, utilizing both a centralized team and also distributing optimization employees throughout the different departmental branches.
How are central optimization teams be structured?
Chad Sanderson, the Experience Optimization Manager for Subway, creates an analogy for how the centralized model is set up:
Benefits of the centralized model
There’s a reason this is the most common form of optimization team organization: the benefits. From an organizational perspective, having all of your data clustered in one place makes it easier to monitor, reduces redundant testing, and streamlines developing a long-term strategy.
But data collection isn’t the only reason this model is used. There’s a significant employee benefit. In a centralized model, it’s a simpler matter to keep the big data picture in mind because there are shared resources and end goals for employees.
Spreading workers out over multiple departments can introduce conflicting motivations for whether or not an employee shares data. Centralizing can help you avoid competition and silos, or data and analytic walls that can prevent the sharing of pertinent data across teams.
Renee Thompson operates as a manager within TechTarget’s centralized model, and says that the localized nature of data helps her team plan for a long-term strategy:
Kristal Decoux agrees:
Lastly, centralizing your efforts can deepen employees’ investment in the project outcomes Promotions, mentorships, etc., can be awarded if the employee is doing well; this is easier to track in centralized systems. This reward system can encourage more personal dedication.
Challenges of a centralized model
Employee investment in data outcomes is a double-edged sword.
When employees are rewarded for data that performs well, it can become tempting to skew the data for their own benefit. The objectivity of the data is lost. Fudge the data, and suddenly all of the statistics gained have lost all meaning.
Even if it’s just one metric, the veracity of everything else is immediately thrown into question.
Yu Guo, a Data Science Manager with Airbnb, agrees that there are certain benefits to using a centralized model, but it isn’t without its drawbacks:
Leadership challenges and company culture can also pose obstacles for data scientists in a grouped centralized model.
Of course, data scientists will usually have to contend with some kind of HiPPO-style leadership at some point. In this case, it’s easy for bureaucracy to trump efficiency, especially in a company culture that doesn’t have a growth mindset. It’s a challenge to get both leaders and data scientists in sync.
As The Harvard Business Review writes:
“[…] Data scientists may feel like outsiders when dealing with the businesses and thus be less attuned to the units’ goals and domain knowledge, which could make it harder for them to connect the dots and share relevant insights.
Moreover, the data scientists may lack the clout to persuade senior management to invest in building the necessary tools or to get corporate and business unit managers to trust the experiments’ results.”
“Imagine that you request an analytics report on a page you’d like to test. You want to know whether traffic and conversion rates are high enough to run a good experiment on the images on the page.
Any savvy analyst can pass you this data. But one who is invested in testing might notice an unusually high bounce rate and use of the search bar on this page; she realizes that visitors aren’t finding what they’re looking for.
Your optimization-focused analyst suggests that you test around this problem instead. She’s handed you an opportunity to make a bigger impact by fixing a glaring issue affecting your customers’ experience.
This type of cross-skills collaboration focused on testing makes your program much more efficient and more successful. An analyst who isn’t truly involved in testing and doesn’t understand the testing program’s objectives won’t apply his or her unique expertise in ways that allow for this.”
How are decentralized optimization team models structured?
Embedded models can be set up several different ways, but it usually consists of spreading optimization responsibilities throughout different branches or departments like your marketing department, product branch, design team, etc.
Decentralized model benefits
There are advantages to this model, mostly that data scientists are forced to gain familiarity and expertise within multiple business branches.
Familiarity with a demographic or geographical area that your business serves can help inform your data collection and customer insight. It also gives a certain degree of employee freedom when it comes to work processes and flows.
Decentralized model challenges
This model can also be messy. There’s significant opportunity for testing and analysis redundancy between teams, mostly due to silos and lack of communication between branches.
On an individual level, decentralization poses some obstacles for employees, as the cross-functional nature of their positions can isolate them from mentoring and peer feedback.
Also, a company culture that that doesn’t value optimization can present big obstacles for data scientists. Matt Roach says it best:
As with the centralized model, efforts at convincing leadership to build experimentation tools can present a significant roadblock, especially without a number of other data scientists to back you up. Often, short-sighted leaders look for cheap quick-fixes, rather than long-term data and culture shifts.
No matter what model you have, ego will always be something you have to contend with, and not just with leadership.
Nick Meyers, Director of Design at FitBit, decided to decentralize his team when he noticed designers couldn’t expedite their product as quickly when they weren’t integrated with product teams. Coordination between departments was at an all-time low, and designers would quit as a result.
So rather than center the designers together on one centralized team, Meyers put one designer on each individual product team instead, paired with product managers and front-end engineers.
Though the transition wasn’t without its challenges–some designers felt isolated and pressured by engineers to value implementation over usability and user experience–the product development and design pace increased because communication between employees on the same project was smoother.
How is a center of excellence model structured?
The center of excellence model combines both centralization and decentralization principles, dedicating some employees to the central function of and dispersing the rest throughout separate departments.
Essentially, this framework establishes best practices and offers guidance for implementation of those practices. The central function develops tools and processes while leaving the collection to the individual branches.
Center of Excellence Benefits
This hybrid function instills processes and standards, but still allows for some creative wiggle room.
The centralized team is in charge of the design, execution, and analysis of controlled experiments, while the employees spread throughout the rest of the organization are in charge of the actual analytics.
Center of Excellence Challenges
Like everything, mixing models does have drawbacks. Often, employees are unsure which team (central or individual) owns which optimization responsibility, and which team receives the budget allocation for new hires when data requires team expansion.
Sometimes, a blend of both models is best for your needs, and even big companies like Microsoft do this.
Paul Magill uses an example of a bank that was torn between the centralized and decentralized model, and finally developed a center of excellence model that worked for them:
“They layered on strips of standardization: a unified language describing marketing job functions; a model of bank-wide career paths; communities of practice with designated “global conveners” that allowed marketing personnel from different units to take coordinated action (such as agreeing on a shared database or consolidating around particular market research partners);new governance bodies such as a brand council; key processes — such as customer engagement and experience, and new market development — that marketing should be running for the bank in all units.
As a result, the marketing organization was able to stop debating centralized vs. decentralized and instead position itself as a unifying force in the organization.
Teams moved away from debating whether, say, data analytics must be centralized or decentralized, and instead started asking questions such as “which specific activities in data analytics could benefit from being shared across units?”
The units have been able to increase the ROI of their marketing programs – mainly by increasing returns, in some cases by lowering investment – by calling on top quality pooled skills in shared centers of excellence.”
How does the size of your company affect which model you choose?
It depends on your company’s resources. As Chad Sanderson explains below, if your company is small, you may want to choose the model that works best within your budget.
Can an optimization team model evolve?
Yes. Ultimately, models are just guidelines for how to structure your company, and your model can change over time to serve the needs of your company as it grows.
Many companies start out with a centralized optimization team and then decentralize it as management sees fit. Yu Guo says even a large company like Airbnb isn’t opposed to changing its optimization model if the data and their needs dictate them to:
Nothing is set in stone, and if someone tells you so, they’re bullshitting you. Much in the same way optimization employees are required to be flexible, so must your model be flexible.
As Ron Kohavi and Stefan Thomke write:
“In companies with multiple businesses, managers who consider testing a priority may not want to wait until corporate leaders develop a coordinated organizational approach; in those cases, a decentralized model might make sense, at least in the beginning.
And if online experimentation is a corporate priority, a company may want to build expertise and develop standards in a central unit before rolling them out in the business units.”
Chad Sanderson expands on this concept, detailing how each model is best at different points in a company’s growth:
Sharing is the key to success
Data is useless if no one knows the numbers or how to interpret them. This is why silos are so problematic.
Your company is working against its own self-interest if accumulated data isn’t being shared across all available platforms to the employees the information is relevant to. Sharing provides clarity and accountability.
Big picture: don’t get too hung up on the model
Models are important, sure. Whatever you choose has to work best for your company, though, and tweaking the framework to serve your needs is relevant than following the model to a T.
At the end of the day, every formatting decision you make is a balancing act. At the end of the day, you have to figure out what’s right for your company.
There are three types of optimization team models currently in usage: centralized, decentralized, and center of excellence. Centralized is the one most commonly chosen in the business world, but each has its advantages and disadvantages. Finding the balance between employee needs and business requirements is key to developing a working system. To figure out which one to implement, you must know:
- Your company size
- Your optimization budget
- Your main-focus metrics
- Your types of optimization employees
- How your employees communicate best with each other
Once you establish these five points, you’ll be able to select which of the three frameworks is best within your company. However, it’s important to remember that the model isn’t as important as the availability of data: make sure your data is available to all relevant parties across all platforms.