Go Back
It’s July and Camp Optimization was upon us yet again! This time, we teamed up with Technology Association of Oregon to bring a panel discussion regarding Embracing Experimentation Culture, a subject near and dear to our hearts. Gathered in the Assembly Lounge at Revolution Hall, our esteemed panel of Roboboogie professionals shared their thoughts on what experimentation is, what it means, and how it can be applied to many disciplines.
This month’s panel was Jedidiah Fugle, Chief Operating Officer, Andrea Pappoff, Producer, and Tyler Hudson, Data Analyst.
So, how is experimentation utilized at Roboboogie?
Jed: One of our core offerings is managing web experimentation programs for our clients. It’s being observant, finding ways to improve things by trying something new, and more. One way we’ve recently utilized it is we noticed a lot of siloing going on between our design and development disciplines, and we wanted to encourage more collaboration. We hypothesized that if we provided a daily standup it could increase productivity and efficiency. So far, it’s been working really well, and we’re letting it run its course before we come back and review our wins. Culturally, we’re always trying to experiment with things that will make us perform and do better.
What does it look like to adopt that scientific approach?
Tyler: If you think back to the scientific method, you start by forming a hypothesis. You want to start with a question that brings innovation over time. As far as starting with the experimentation culture, it really just starts with being curious. Ask questions, see what you could improve, and take some risks!
So, you’re using data to make decisions. What qualifies as data?
Andrea: Data can take a lot of forms. You want to take the qualitative and quantitative data, so for example you can take anything from a group of opinions about a specific matter to getting out and interviewing people which leads into the more quantitative side where you can pull data. You can also tie back to best practices by pulling together all the years of experience you have on your team to pick what data points will work best.
Website experimentation is one of the services offered at Robobooogie. Can you tell us what goes into a website A/B test?
Tyler: A/B testing, at its core, goes back to something as simple as testing a red button vs. a blue button and see what works better. We take that to the next level by introducing new functionality to our client’s website, look to streamline different areas or break them apart, and what collectively makes them function better by understanding every point in the customer journey. When setting up an experiment, you want to make sure you’re collecting the right data, so you always want to make sure you’re strategizing with your team. Clear KPIs will always set you up for success.
What roles contribute to experimentation at Roboboogie?
Andrea: One of our critical pieces to having a successful program is bringing together multidisciplinary teams in order to tackle a problem from all sides. It starts with an opportunity we identify in the data, and then relaying that data to a UX strategist or designer who will ask “How can I make this better?”, and then passing a specific design solution onto the developers. Then you need collaboration between development and analytics, to make sure we’re backing up the design elements with hard data. Ultimately, we want to give our clients the confidence to implement those wins.
What is an interesting or unexpected outcome of web experimentation that you’ve seen?
Jed: It’s really fun in this new era. Organizations have become more comfortable with tracking data, which brings up its own challenges in organizing and understanding that data, and with our clients, we track the most minute detail. For example, we created a pop-up modal that displayed an item that a returning customer to an eCommerce site had looked at the last time they had previously visited, but it dropped overall revenue. It presented an opportunity to go deeper into the data to see what happened. It’s a funny question because there’s always something interesting as you go deeper and deeper into what your findings are.
So you’d say these findings have contributed to your client’s interest in web experimentation?
Jed: Yeah! The phrase we like to use is “Data Storytelling.” Our job is taking those spreadsheets, all that huge amount of data, and telling something meaningful with it. Oftentimes people come back to us and say “Wait a minute, I’ve been sitting on all this data and I’ve never done anything with it?” which leads them to ask us what more they can do with it. That builds a culture of curiosity and more data-driven results.
Are there any other benefits to online experimentation?
Andrea: It’s fun to watch a client open up to the idea of testing. Sometimes when we’ve started with a client, they don’t entirely understand what it means, but as time goes on and you see the data and apply their wins, then you see them grow into being the one who offers up testing. It’s really rewarding to see that testing culture grow!
Let’s talk about failure! How do you deal with that? It seems like for some organizations that would be harder than others.
Tyler: Well, that’s why you’re testing. If you knew 100% that something would work, there wouldn’t be a need to test it. Failure is an important data point. If a test “fails,” you can dive into it and find out why, and it will inform smarter decisions down the road.
If I’m going to bring experimentation to my company, how would I go about approaching my manager to convince them to try it?
Jed: It really starts with being curious and having a genuine interest in the scientific approach. Then you might want to find allies, others in your organization who are interested in the scientific approach. Then you would want to have a conversation with a director or manager—unless you are that director or manager!—get the ball rolling with them. The more data you can bring to the conversation the more success you’ll have.
Tyler: When you find those allies and have that curiosity, you’ll notice a lot of momentum will build behind that. When you feel like your voice is being heard, you’re giving them an opportunity to see the potential and your hard work.
Andrea: There are a lot of misconceptions that there is a lot of risk by adopting an experimentation culture but for most of our clients, the program pays for itself. Working with a strategic partner like Roboboogie, we can help inform where to start, especially with simple, high-value tests that help move the needle quickly.
See you next time!
Save the date for our next Camp Optimization, Thursday, September 26. Want to learn more about how we embrace and utilize experimentation? Drop us a line!
