Optimization Deconstructed | Part II: Fundamentals of Optimization

Specialized bikers riding on rough terrain

Welcome back to our Optimization Deconstructed Blog series! This week we will dive into the fundamentals of optimization by examining the role of an experience audit, the importance of using KPIs, and tools to measure success.

The Experience Audit

An experience audit is the first step we take to evaluate the performance and positioning of a company. It starts with understanding two main things: the target audience and the business itself.

The Target Audience

Understanding the target audience is critical to evaluating who we aim to reach during our optimization efforts. We must understand what motivates them, how they perceive the brand, and what they expect from a digital experience in order best address their needs and wants. We develop personas to represent a group of users, usually ideal target customers, to develop a deeper understanding of their needs. By learning their expectations for an experience, we can craft a new digital journey that caters to these users.

Jessica persona creates to represent a segment of Specialized Bikes consumers

The Business

The second aspect of an experience audit involves understanding and examining the role of the business. In order to create an experience that fits the needs and goals of the business, we must understand their brand. This requires a thoughtful knowledge of the brand’s messaging, perspective, and value proposition. Once a thorough and well-rounded cognizance of these two elements has been established, we examine the current experience being offered.

When examining the current experience, we also look at a transactional perspective around usability: How easily can users navigate the site and find what they’re looking for? What is the information hierarchy? These questions, among others, inform our approach moving forward.

Specialized bike riders with their bikes in the back of a truck

The Analytics Audit

Another fundamental and preliminary step we take when working with a new client is the analytics audit. We typically break analytics audits into two categories: behavioral analysis and technical implementation analysis.

Behavioral Analysis

An audit of website analytics (typically Google Analytics or other user behavior reporting solution) is important to evaluate how users interact with the website. Some of the components we consider are:

  • event tracking
  • user pathing
  • key converting pages
  • high value content
  • use of goals, custom dimensions and metrics

While we are performing a behavioral analysis, we identify opportunities to collect more meaningful and actionable data that will allow us to better understand unique audiences and optimize site performance.

Technical Analysis

An audit of a Google Analytics implementation and data output are used to determine data quality and integrity based on best practice standards. We focus on a specific digital property and the corresponding data collected for that digital property, typically focusing Google Analytics and/or Google Tag Manager. We triage how the analytics are implemented on site, event tracking and any customization of the analytics implementation specific to the account. We then evaluate the data integrity as it lives within the reporting tool.

Our analytics audit verifies that the data being recorded and used by an organization can be relied on. It also informs our user experience audits, as we can utilize the behavioral analytics to bolster recommendations around user pathing, high value content, and segment-specific analysis on different user profiles.

Organizations depend on an analytics implementation that provides valid data. It is the backbone upon which they build their marketing plans and calculate their return on investment. Without verified data in their analytics, organizations may be making business decisions based on incorrect assumptions and projections. Additionally, the use of analytics has an impact on website optimization and improving the overarching customer experience. The technical implementation audit gives businesses the peace of mind that they are collecting data they can trust. We utilize the behavioral analytics moving forward to inform UX and visual design, marketing strategies, and personalization efforts.

Post-Audit Insights

In some cases, as with our engagement with Specialized, we’re able to use non-digital analogies to help reveal where an experience is falling short. For example, in a brick-and-mortar scenario, what does the shopper want their experience to offer? What additional resources are necessary to fulfill these wants? By taking these aspects into consideration, we can then draw from best-practices in the digital space, and create an intuitive experience based on what users are already familiar with. We use the experience audit as an opportunity to look through a heuristic lens to set a baseline experience, and make incremental improvements from that baseline.

Specialized bike rider in a forest setting

Determining KPIs

In order to craft better brand experiences, we identify the Key Performance Indicators (KPIs) we aim to measure. Measurable success metrics are critical to the success of any optimization effort. Tracking progress and results along a given track gives us the opportunity to not only see what’s working (and to what degree) but also to implement our wins. KPIs also allow us to utilize analytics to discover details about how customers are using the site and where they are spending their time. This allows our analytics team determine which aspects of the website are achieving success and which areas need improvement. This provides us with a strong jumping-off point for figuring out what obstacles exist in the user’s overall experience. This data leads to the further development of strategy towards optimization efforts.

Four main KPIs that we track for the majority of our experiences are:

  1. Exit Rate (percentage of users who leave without engaging content)
  2. On-page engagement (interaction with an element we are testing)
  3. Non-revenue conversions (newsletter sign-up, social share)
  4. Revenue-based conversions (purchase, lead)

Google Analytics performance summary showing 46.4% increase in navigation to products

Optimization Tools in our Arsenal

Throughout our analytics and testing process, we use best-in-class tools for running tests and measuring the success of our optimization strategy. Some tools we use include Optimizely, VWO, Google Optimize, User Testing, and others.

  • Optimizely is the foremost tool we use because of its ability to run testing on high-traffic websites. It also includes one of the industry’s best personalization platforms.
  • VWO is similar to Optimizely, but is more economical for lower-traffic websites.
  • Google Optimize requires an existing Google Analytics account, and therefore works best for organizations that rely heavily on GA already. This platform provides businesses similar functionality to VWO in terms of A/B testing. Upgrading from the free version to Google Optimize 360 unlocks the ability to import audiences from GA, which allows organizations to better utilize their site data for personalization efforts.
  • User Testing serves experiences to users who are screened to match determined user personas. Their experience is recorded, and we are then able to see how they engage with the site, hearing feedback about their experience.
  • Hot Jar is CRO tool, self-described as “a new and easy way to truly understand your web and mobile site visitors.” The platform offers heatmaps, playback, conversion funnels, form analytics, feedback polls, surveys and user testing recruitment.
  • Crazy Egg is an analytics and visualization tool that shows how users are spending time on a website.  They provide “heatmaps” that illustrate where users and clicking and scrolling, and provide user data about where people are coming from and who’s clicking on what with which frequency.  Originally set up as a heatmap specialist, Crazy Egg now offers user recordings to let you see how visitors are interacting with your website, as well as viewing snapshots of activity.
  • roboboogie Self-Segmentation Modal is a proprietary tool we built in house allowing users to voluntarily participate in a survey to identify themselves and their motivation on site, allowing us to watch behavioral trends, to identify gaps and determine high-value segments (blog post with more in-depth detail coming soon!)

We always use data to drive our design enhancements and the development of our tests. In order to produce enough data to achieve test results that are statistically significant, high traffic levels are extremely helpful. With higher levels of traffic, we are able to produce results more quickly, which ultimately opens up bandwidth to run more tests. The more people who engage with the tests, the more informative the results. Tests are still possible with lower traffic rates, with extended timelines, through platforms like Optimizely, VWO and Google Optimize.

Specialized Bikes Case Study: Ecommerce Site Recognition

Venturing back to our work with Specialized Bikes, we are going to examine the role our experience audit played in our optimization efforts and development of our strategy. Our experience audit was a fundamental process that informed our approach to optimizing Specialized’s website. An essential part of the audit was understanding Specialized’s customers, and, in this case, we created six different personas to capture the characteristics of their different users.

We then realized that a major issue with the site was its inability to register with users as a ecommerce site. Its design and functionality was too closely aligned with their product catalog and telling the brand’s story, which resulted in customer confusion. Instead of a large product presence and strong contextual navigation, the site was filled with information about their technology, innovation, and sponsored riders. The oversaturation of lifestyle imagery and content confused customers who often failed to recognize that they could actually shop on the site. After examining these issues, we decided to focus the strategy moving forward on optimizing purchase flow to generate additional revenue and improve website usability.

We created specific product categories for the gear being sold on the Specialized site, and did a strategic redesign of the company’s shopping cart icon (check out Part III: Elements of Testing for more on this!). With the introduction of these elements, users easily recognized that they could shop on the site, which resulted in more revenue.

By making improvements to site navigation and usability, we observed:


increase in navigation to products


increase in site-wide revenue

Tune in Next Week!

Did you enjoy learning about our Optimization process? Keep an eye out next week for Part III in our Optimization Deconstruction blog series, The Elements of Experimentation. We will be talking about multivariate testing, personalization, and of course our work with Specialized Bikes!

Check out Part I of our series, Foundations of the Industry.

Contributors: Andrea Pappoff, Duncan Lawrence, Etain O’Longaigh, Jedidiah Fugle, John Gentle

Must-Follow Steps for Sending an Email with Watson Campaign Automation (formerly Silverpop)

Watson Email Campaign Automation Program

You can find great email checklists all over the internet. Here are a few of my personal favorites:

Unfortunately, none of these account for Watson Campaign Automation’s (formerly Silverpop’s) unique flair. As an enterprise system, the assumption from IBM is that their users have an advanced knowledge of email platforms. In general I would say that is true, but even with industry knowledge there are some unique factors you need to remember.

Email Troubleshooting & Tips for Watson Campaign Automation

We have worked our way through a myriad of problems using Watson Campaign Automation, slowly compiling our checklist of things that must be reviewed either during template creation or before we click the big “scary” send button. Hopefully these will save you some time and frustration.

Use the correct Contact Source

Much of Watson’s magic comes from your database. In order to get the most from the email platform, make sure that you select the right Contact Source.

Watson select contact source screen shot

Note: Contact Source at the template level is only necessary if you are using personalization or Contact Source-associated Hyperlinks features. If you are setting up the template for a program without those, do not worry about connecting a Contact Source at the template level.

Use the correct From and Reply-to addresses

Not much feels worse than a manager asking why they are received 30,000 out-of-office replies. Before clicking Send, double-check the Address Settings for validity and correct email addresses.

Using preheader text? Turn off the auto-append feature

The option to auto-append a Click to View in Browser link is useful, but preheader text will not work in Watson Campaign Automation if you allow them to append the link automatically. This is because the link is appended at the very top of the <body> section of your email, overriding any of your wonderfully crafted preheader text.

Watson Campaign Automation's built-in View Online toggle

It is still important to add your own Click to View in Browser link, which can be done by utilizing the Hyperlinks tab to adjust the link settings to Custom click to view.

Set your tracking level

Within the Mailing Settings you can set your desired tracking level. Generally, you will just want to ensure that Unique is selected as you can still get the other tracking numbers from reports. If your Privacy Policy, Terms & Conditions or country-level privacy laws are more restrictive, however, you will need to choose the matching level.

Watson Campaign Automation - Select the correct tracking level for your email

Use the right type of link

If the only thing you do to create a link in Watson is use an <a href=””> tag, you are missing out on the powerful features of IBM’s marketing tool. Watson allows you to use the following types of links:

  • Tracked hyperlink
  • Not tracked hyperlink
  • Bookmark hyperlink
  • Clickstream hyperlink
  • Forward to a friend hyperlink
  • Web form hyperlink
  • Double opt-in confirmation link
  • Opt-out (to page) link
  • Opt-out (one click) link
  • Opt-out (custom) link
  • Opt-out to mail link
  • Email (mailto) link
  • Custom click to view link
  • File download link
  • Social network link

There are a lot of options and each have their respective benefits. In practice, we typically use only the Tracked hyperlink, and Custom click to view link. I recommend that you create your email template with standard <a href=“”> links, add the template to Watson, and then use the Template Editor’s Hyperlink tab to review all included links and ensure you have the correct one chosen. You can set up your development system to include the different link types, but doing so may cause you to miss the next step on this list.

Verify your links

While you are in the Hyperlinks tab of the Template Editor, confirm that all your links are rendering as expected. By hovering over the page previews, Watson will tell you if a link is verified:

Watson unsuccessful URL verification

Warning: some websites do not have proper response headers. A rendering looks like it will probably be fine, but in reality the 404 page is sending a 200 response header – which shows you the link is working, even though it is not.  LinkedIn is an example of this:

Watson Campaign Automation successfully verifying a URL

Populate your UTM tracking code

Do you use Google Analytics like the rest of the known marketing world? Yes? That’s what I thought.

In order to include GA tracking, you need to manually populate UTM codes on each of the links during development. In case you need a reminder, you can use Google’s super easy UTM builder tool to create the parameter string.

First time in there? Here are my recommended starting points for your UTM codes:

  • Campaign Source: watson (or silverpop if you have historical data within Analytics that you would like to keep consistent)
  • Campaign Medium: email
  • Campaign Name: [your-campaign-name]

That should be enough to start. If you want to dig deeper and use more segmentation opportunities (particularly if you are A/B testing), you will probably want to read up a bit more on UTM codes and their value within analytics.

Confirm that Unsubscribe works

Unlike a program, Watson does not require or warn you if an Unsubscribe link is not available during validation. Save yourself the frustration (and potential lawsuit) by double- and triple-checking to make sure that the Unsubscribe link works.

Use Our Checklist

We have put together these steps (and a few more) in our Watson Campaign Automation Email Checklist. The list concentrates more on the technical side, after you have used campaign planning checklists tend to develop the email strategy and content.

Click here to use the checklist.

Have anything we are missing? Let us know at hello@teamroboboogie.com.

Written by Duncan Lawrence

Optimization Deconstructed | Part I : Foundations of the Industry

Silhouette of riders with bikes and sunset in background

Let’s Talk Optimization!

We are excited to bring you a new blog series taking a closer look at roboboogie’s approach to optimization showcased through work with one of our amazing clients, Specialized Bikes. This post will be the first in a series of six, as we dive into deconstructing our process through a handful of real-life client success stories. To kick this series off, let’s begin by laying down how we define foundations of the (somewhat nebulously defined) optimization industry.

What Optimization Means to Us

Optimization involves efforts of iteratively improving a process over time. To us, it lives in the sweet spot intersection of science and creativity. Optimization can be applied to many areas, but we’re pros at applying optimization practices to measure and maximize the effectiveness of a digital experience. What does that mean? We’re all about gathering raw data (in both qualitative and quantitative form), interpreting it, and harnessing the data-stories to inform design and functionality. As the importance of digital strategy becomes increasingly evident across the marketing world, optimization becomes the key to staying competitive. Our optimization process utilizes data from a variety of sources (behavioral analytics, customer surveys, usability testing, onsite testing, market trends, heuristic evaluations, and any other form of knowledge we can get our hands on) to evaluate current experiences, identify gaps in performance, and inform business actions. In order to discover new ways to optimize existing brand assets, optimizers like us use tools like segmented targeting and A/B testing to customize and deeply personalize each aspect of a marketing strategy with the needs of the customers kept in mind. We use data-driven design to improve customer journeys and ultimately, make the internet better! Every decision we make at roboboogie revolves around being strategic with customer data. 

How Do We Work?

Our unique approach to the world of marketing, design, experimentation and personalization utilizes both sides of our brains: creative and analytical. Our approach can help guide companies through the optimization process through our service offerings which include: 

  • Data Insights: We de-clutter customer data to tell the ‘what, why, when and how’ stories you can use build more profitable customer relationships. Our insights include qualitative research, analytics and journey mapping to fully understand and document customer needs and motivations.
  • Data Design: Our proven customer-centric, data-backed design process guarantees both form and function. Whether you’re looking for a ground-up redesign, or want to make some design enhancements to your digital experience, we’ve got the skills to take your digital experience to the next level.
  • Data Convert: Whether you’re just starting out with conversion rate optimization, or you’re looking to take your CRO program to the next level, our proven customer-centric, data-backed, design-forward approach can help you exceed your online sales and conversion goals.
  • Data Target: Reach the right customers, with the perfect message, at the ideal time and place, and they will convert. Whether you need creative ideas for your next targeted campaign, or you’re seeking a strategic partner to roll out a comprehensive personalization program, we’ve got the tools and experience to help you get the most out of personalization.

Getting the Most from Customer Data

Here at roboboogie, we believe experimentation is the key to success. Data-driven marketing and design through testing is the key to business decision-making in the digital space. Customers are telling us what they want everyday through their site behavior – we just have to ask the questions (and test our hypotheses) to uncover creative solutions that best suit the needs of customers. This customer-oriented approach increases conversions and satisfaction, while ensuring the user is getting the most from a brand encounter. This all centers around getting to know target customers better- who they are, what motivates them, how they perceive a brand, and what they expect from an online experience all factor in to how we approach solving a problem through optimization.

Introducing Our Case Study: Specialized Bikes

Specialized Bikes, a major innovative U.S. brand of bicycles and cycling-related products, engaged roboboogie to help optimize their digital experience to grow their online community and improve online sales. Their desktop website design, which closely aligns to their product catalog, was not achieving goals for selling gear components and apparel. Additionally, their mobile experience was rapidly becoming a significant engagement touchpoint with customers, and also represented significant opportunities for optimization. Our optimization focus areas for this client were:

  1. Raise shopping site awareness
  2. Provide a clear path to products
  3. Improve site usability

Through our unique DATAactions approach, we were able to systematically identify and solve for opportunities while iteratively improving over time to meet and exceed business goals:

We harnessed customer and site data to develop consumer personas to help Specialized better understand the users that were coming to their site and where the experience gaps were.

Facing internal resource and timeline challenges we stepped in to make strategic design changes, developed promotional content and served targeted messaging through the Optimizely platform.

Through a series of A/B testing campaigns, we improved site usability and, among other improvements, increased average time on site, product engagement, and ultimately revenue.

In one particular testing & personalization campaign, as part of a seasonal push to sell more accessories, the roboboogie team implemented a buy-one-get-one campaign for road tires called “Let’s Go Dutch.” Through a series of style and copy testing for the promotion, in addition to tire shopper segment building, the optimized and personalized experience produced a highly successful 21% lift in revenue. For more on our personalization efforts with Specialized Bikes, check out our case study: Harnessing Optimizely to Power Personalization Campaign Targeting.

Over the next four posts, we’ll get into more specifics about our optimization approach with Specialized Bikes, the impact we had, and the tactics we used to achieve our goals. Keep an eye out for our next installment in this series, Fundamentals of Optimization, where we’ll talk about how we take on a new client, the tools we use.

Contributors: Andrea Pappoff, Duncan Lawrence, Etain O’Longaigh, Jedidiah Fugle

Opticon 2017: Where Creativity, Science …and Massages collide!

Data points being shaped into a structure

Roboboogie will hit the road this October to support Optimizely as sponsors of their fourth annual optimization conference, Opticon. The conference will take place at the Wynn Hotel in Las Vegas, NV.

Opticon 17 Promotional Image: Opticon, where creativity and science collide. Wynn Las Vegas, October 17-19

Opticon is the leading event for practitioners of website optimization and A/B testing. It will be attended by industry experts and marketing experts, including 1,200+ of the most influential change makers transforming businesses’ today with the scientific approach. This gathering brings together business and marketing leaders, optimization specialists and industry professionals to share best practices, new strategies, ideas and relationships that elevate their optimization programs. As the leading experimentation conference, Opticon brings together creativity and science to drive innovation and transform organizations.

Come learn more about our partnership with Optimizely, our unique data-driven experience design and conversion optimization skillset, and chat about industry learning from clients such as Wacom, Specialized Bikes, Nike, Universal Technical Institute, Urban Airship, Simple Banking, Kelty, The Clymb, NAVEX Global (and even Optimizely!).

This year, we will be sponsoring (you guessed it) Opticon17’s Massage Lounge. Non-stop networking wears people out, and the robo-powered relaxation station will come packing the details down to massage chairs and masseuses. Come get first dibs on a place to unwind and snag some choice roboboogie swag.

About Optimizely

Optimizely is the world’s leading optimization platform, providing A/B testing, multivariate testing and personalization for websites and iOS applications. The platform’s ease of use empowers organizations to conceive of and run experiments that help them make better data-driven decisions. With targeting and segmentation, Optimizely meets the diverse needs of any organization looking to deliver unique experiences to their audience.

To learn more visit: https://teamroboboogie.com

Interview with Roboboogie Sr. Strategic Designer, Lacie Webb

Lacie Webb's chalk mural - we make the internet better

Have you ever wanted an to take an inside look at the role of a Strategic Designer? We sat down with Roboboogie Sr. Strategic Designer Lacie Webb to pick her brain about all things user experience design and to give you a glimpse at her approach to optimization.

How do you describe what you do?

I’m a Sr. Strategic Designer. There are usually pretty inconsistent descriptions of what that means for different people, and a lot of the time those descriptions seem to focus solely on digital touch-points. For me, User Experience is much, much more. UX is how someone gets from Point A to Point B. From home to work. From not knowing to finding the answer. UX starts with identifying who the user is and what their needs are. Since I work with clients, I also have to balance business requirements with user needs to create a solution that works for everyone, but usually a happy user = a happy client! Once I have a clear understanding of the problem, I can work towards creating an improved experience, usually via sketches, wireframes, and interactive prototypes. The UI side of my job focuses around making sure that the UX I created is easily understandable by using design, color, and clear visual hierarchy.

What is a skill set that you use every day, but never thought you would have to?

Without a doubt verbal communication! When I was first starting out, I thought my solutions could speak for themselves; if I made the most beautiful wireframes, that clients would just get it. I quickly learned that this is not the case. While I do focus on honing my craft and try to create beautiful wireframes, often times digital experiences are abstract, many-layered, and complex. I’ve worked very hard on my presentation skills to be as clear as possible when describing how a site will behave to my clients.

How do you approach solving problems?

By drawing. A LOT. Seriously. There’s that saying “Fail and fail fast,” and nothing could be more accurate. I have never found the right answer on the first try and I hope I never do. I like to challenge myself to get as many ideas out as I can. Even though we usually use one or two of those ideas, and some of the other ideas are just BAD, that’s ok! Those solutions, while they don’t solve one specific problem, may be perfect in the future or for some other client or problem. So pretty often I just start by roughly sketching all those ideas on whiteboards, post-it notes, or paper, and my workspace ends up looking like a scene from A Beautiful Mind.

What user experience trends or tools are you currently most excited about?

I’m excited about the rise of storytelling and moments of surprise and delight. For example, when you move cards around in Asana, and you see a little monster graphic pop up. Those small moments that without them the experience would be fine, but with them it makes it great and memorable. Those are the details that stick with users and often are the hardest things to get buy-in on. I want to high-five the designer who fought for that little monster.

What is your advice to designers that are new to create variations for A/B tests?

I would go back to my previous advice of making LOTS of ideas. Even if the variations between ideas are subtle, they’re still valid changes and you still learn from the data. Don’t get too personally attached to your solutions. Leave yourself open to the chance that your idea might fail. It doesn’t mean you’re bad at your job, it just means you learned a little bit more about your users. And finally, if you find it challenging to come up with a lot of ideas, don’t be afraid to ask for help from others. You’re not a wizard and more brains are always better.

Thank you, Lacie for giving us an idea of what makes a Sr. Strategic Designer tick! If you would like to learn more about Roboboogie, visit www.teamroboboogie.com.

Stay tuned for next month’s interview with another Roboboogie pro!