Interview with Roboboogie Technical Coordinator, Josh Logan

josh logan

Do you have a go-to person to contact with complex technical questions? For Roboboogie, that person is Josh Logan – our rockstar Technical Coordinator. This week, we’re chatting with Josh to learn more about his role at Roboboogie, the importance of QA, and his thoughts on emerging technologies.

How do you describe what you do?

As Technical Coordinator, I am responsible for a couple different aspects of production at Roboboogie. My main focus is to coordinate between our creative, analytics, and development teams to ensure that all project requirements are fulfilled, any technical questions are answered, and that our development cycle is on track. I am also responsible for Quality Assurance to ensure that tests meets all design, interaction, and data collection requirements with no functional issues. My other responsibilities include handling technical and software integrations, network and internal security, and internal process improvements.

What is a skillset that you use every day, but never thought you would have to?

Communication. It’s a common view that technical people tend to be introverted or anti-social, but one of the most crucial aspects of development, especially at a multi-disciplined agency, is strong communication. I am constantly in contact between our project managers, strategists, analysts, design resources, and clients to ensure that our development team has all the information they need, and that all key stakeholders are informed. Having good communication skills is critical for gathering, translating and transferring crucial information between our creative and technical departments.

How do you approach solving problems?

Solving problems is a significant portion of my day. Whether it’s integrating a new technology into our workflow or hunting down a bug in a test or experience we have created, I am constantly improving my ability to solve problems. For me, logic plays a key part. I try to break down the problem into smaller pieces and understand where the issue is stemming from. This helps me see things from a step-by-step  perspective. From there, I can narrow in on the specific problem at hand. One piece of advice: most problems have happened to someone else out there in the world before – sometimes all you need to do is a little research to get on the right track!

What emerging technologies are you currently most excited about?

AR/VR and AI/machine learning. I am really excited about the potential that AR/VR has to change the way that we consume information and entertainment. It’s amazing that you can already play a detailed virtual game world on your coffee table, find out what a new chair will look like in your living room without buying it first, and even see your favorite band in concert while never having to leave your couch. I’m excited to see what is next.

I am also really interested to see how AI and machine Learning technology advances – specifically its role in digital optimization in the future. There will be tech that we can’t even fathom at this point. AI and machine learning could also play a huge role providing insights and personalized experiences that wouldn’t be possible today. It is an exciting time to be alive.

What is your advice to organizations that are new to A/B test development from a quality and security perspective?

Never underestimate Quality Assurance. It is easy for QA to become an afterthought in the test build process. However, ensuring that your users are getting the experience that you intended is key to testing success, and missed issues with functionality or tracking can tarnish the experience and the data integrity. Let’s be honest – QA can be hard, laborious and time consuming, but in the end it’s worth it if you are providing an exceptional experience to your users. In general, don’t be afraid to try something different or out there – the point of A/B testing is to experiment!

Thanks for the wise words, Josh! Tune in next month for an interview with another Roboboogie pro.

2018 New Years Optimization Resolutions

Hero illustration showing a cat, dog, runner, woman drinking coffee, man working at desk and woman running

Hard to believe another year is almost in the rearview. Time to reflect and come up with those nagging, but potentially life changing resolutions. To help get you in the spirit, we’d like to share some optimization wisdom disguised as five popular personal resolutions. We hope this list inspires you to set some goals for optimizing your digital experiences (and perhaps even your personal life) in 2018.

Exercise more frequently

Increase test velocity

Velocity Hurdles

The more you experiment, the more optimization opportunities you’ll uncover, and the more you’ll learn about what makes your customers happy. The happier your customers are, the more likely they’ll be to convert. So don’t be afraid to jump in. Start small and test frequently. You’ll be surprised what you learn, and how quickly it will become part of your routine. Before you know it, you’ll see really incredible results. Keep building on those learnings and make those iterative but meaningful improvements. Frequently adjust your goals to reach new heights and continue to increase your conversion rate.

Learn more about what the power of rapid iterative testing can do. Check out our case study with The Clymb and the impact an iterative approach had on improving customer value.


Make smart investments

Reduce your Ad spend

Optimize your ad spend

A popular (but misguided) method frequently deployed to boost conversions, is simply driving more eyes to a website or landing pages by increasing advertising spend. While you might acquire more customers this way, a far more cost effective method is to improve the experience to get a higher percentage of visitors happily converting.

A wonderful side effect of providing a better experience is that you ultimately end up with happier customers, which increases customer retention, lifetime customer value and the likelihood they will recommend you to a friend.

Check out our case study on how we helped Universal Technical Institute reduce its ad budget, increase leads and improve lead quality.


Get a physical

Let the data lead the way

Let data leads the way

These days, getting access to piles and piles of customer data and analytics is easy. It’s how you interpret that data and decode the story it’s telling, that is the hard part. However, if you are not frequently checking your analytics to ensure the entire customer journey is performing as expected, you could be leaving a considerable amount of revenue behind.

It is important to not just consult your customer data when making decisions, but to let it drive strategy to ensure happy customers and goal-smashing results.

Find out how we let data drive a site redesign for Benchmade knives that lead to happy customers and a 320% increase in online sales.


Learn a new skill or hobby

Start Personalizing

Smart personalization

Digital marketers have recognized the power of personalization and have held the vision of one-to-one marketing up high for years. However, unlocking the full potential of personalization requires a significant commitment to both technologies and resources. But, if you can make it a priority, it’s all worth it. 81% of consumers want brands to get to know them and understand when to approach them and when not to. Like experimentation and A/B testing, you should start small and build your program on successes and learnings. Iterate slowly and methodically, and don’t be afraid to try new things. Regardless of whether or not you hit it out of the park on the first swing, you will learn something extremely valuable about your customers that you can build on and hone in over time. It won’t take long and you will see a strong ROI.

Learn more about how roboboogie harnessed personalization and target marketing to help Specialized Bikes sell through BOGO tires in less than a week.


Spend more time with family and friends

Don’t forget to listen to your customers

Listen to your customers

As data-lead marketers, it’s really easy to get heads down in the data and forget that there are humans behind all those numbers. Don’t forget to come up for air, get out in the real world and talk to your customers. Conversations with even a small sample will reveal a lot about their needs and what makes them happy. Combine this with some usability testing and you’ll likely have no shortage of ideas for optimization and personalization.

Find out how harnessed qualitative user feedback alongside experimentation to drive over 46% ecommerce sales growth for Wacom.

Always Implement Your Wins

roboboogie team acting like pieces of a chess board while on a large chess board

We here at roboboogie love to test. It’s exciting to blend the scientific with the creative to discover a better path toward conversions. Sometimes the most difficult part of optimization is not finding ways to improve ROIs or KPIs, but it’s the implementation of winning tests.

Why implementing your wins is so important

We recently wrapped a test with three variations that made related products available directly from a user’s cart. We learned a ton about user behavior from this test and we were able to grow revenue in each variation by $17,326, $70,743 and $77,640 respectively in right around 5 weeks. Each variation was allocated 25% of the total traffic including the baseline. 

total revenue gained by each variation vs the original in A/B testing. Variation 3 wins $414,000 to the Originals $336,000
This one test might have gains up to $3.2 million in one year for the highest yielding test if it was pushed to 100% traffic for 52 weeks. Math: ($77,640 [revenue increase] x 4 [traffic increased from 25% to 100% so 4 times] / 5 [divide to get a week’s worth of revenue so we can multiply by weeks in a year] x 52 [weeks in a year]).

So you’re probably thinking that these types of tests are generally shipped off and developed and launched within the hour, right? I mean, every week a customer could be losing up to $60,000! Sadly, too often winning variations never get integrated into a site. Why? Almost always this is because there is no clear path toward implementation before testing began. Testing is the goal and making those tests into a feature of the site is an afterthought.

Create a clear path to implementation

By failing to implement our wins, we make iterative testing more difficult and costly on site performance. Once we determine a winner, we are looking to maximize our effect on users by running smaller tests on winning variations to key in on the most impactful changes. Those slight variations need to alter only a small percentage of the previous winner, but without a previous test being implemented those small changes could bring a large baseline of code from the previous test. This baseline shared across multiple variations creates a larger testing snippet as well as more places where errors could crop up and hinder a test’s results.

The most common roadblock we see is a separation between departments. The testing department is usually spearheaded, surprisingly, not by the development team, but by some combination of marketing and sales looking to find conversions and engagement. The dev team often has a backlog of features to build or bugs to fix. Finding time for developing a winning test not on the roadmap can be tricky. That’s why it’s imperative that prior to testing, all departments that are needed to develop and deploy a winning variation understand their responsibilities and have resources available for development, QA and launch. Optimization and testing needs to be a company-wide buy in.

Jeremy Sell developing at his desk

Steps for success

Strangely, securing budget can stand in the way of implementation even with a clear ROI win. The best processes we’ve seen from our customers usually includes budget for implementing winning variations prior to testing. Sometimes this can be tricky as there is no data to back up the proposal; therefore, testing teams are then stuck trying to secure a budget for new tests and implementation of old tests at same time. Testing is more exciting, proves future budget in a simple way, and usually takes precedence if only one budget can be secured. In order to safeguard the implementation of successful test results, keep the following in mind:

  1. Secure budget for implementation immediately
  2. Get all departments needed on board before you start
  3. Implement early and implement often

If you are working with an optimization partner, like roboboogie, there are a couple ways to smooth this transition to site integration. Both involve offering access to a larger portion of your codebase. If your partner has the ability to build tests into your actual site and trigger those tests via Optimizely’s REST or JavaScript API, then the losing variations can simply be removed from the page and all you’re left to do is QA and launch. This approach can be difficult while your in house development team is potentially changing those same pages, but will save the most time. The second approach is to offer enough of your site so that your optimization partner can rebuild the winning variation to be as close to compatible as possible, leaving your in house development team to make small edits before QA and then launch. Both approaches will significantly lighten the load for your internal team.

Not integrating your wins is like buying new cabinets to remodel your kitchen and then leaving them in your basement. You’ve spent real money and probably would have a better product upon finishing the job. Creating a clear path toward implementation will help you and your team increase your metrics which provides a clear case for continued testing. So secure budget for implementations, know your development team’s availability or have your optimization partner on board to build winning variations, and get all groups involved to buy into the science of testing. Good luck!

Written by Jeremy Sell