July 2019 | Roboboogie.

Here at Roboboogie, our bread and butter is web experimentation. We’re obsessed with A/B testing and how it can lead to better online experiences. But working within a multidisciplinary environment can be tricky—especially when it comes time to hand-off the test from design to development. Nobody likes to have a ton of meetings, but without them, it can lead to confusion, overlap, stalling, and possible failure, all of which are worth avoiding.

As a result, we’ve developed our own in-house process for these test-design-to-development hand-off meetings. The meetings are quick, only require a few people (designer, developer, producer), and create clarity, which saves everyone’s time.

Here’s the official Roboboogie Test Development K/O Checklist™ (RTDKOC™):

Is the test signed off by our team and clients?
Do we have mobile and desktop designs?
Can tablet be built from desktop and mobile designs?
Are we using standard viewport sizes for designs?
Is the PSD accessible if a developer needs to slice?
What assets need to be handed off by the designer?
Is all functionality understood?
Any pages that are outliers and may not be accounted for in designs?
What are the traffic implications?
What needs to be tracked?
What types of animations do we need?

Is the test signed off by our team and clients?

This is the most important task to check off. In speed-to-market, it’s important to ask if all necessary stakeholders have signed off on a project. This prevents anything from being drastically altered or completely shut down by someone who hasn’t given their approval. This is the most time consuming if projects move too fast and someone does not sign off on a hypothesis.

Do we have mobile and desktop designs?

Is the test running responsively across all devices? If yes, have the designers handed off designs for mobile and desktop? This prevents design bottlenecking by ensuring the developers aren’t responsible for the translation from desktop. A little extra time spent upfront frees time for everyone else further down the line.

Can tablet be built from desktop and mobile designs?

Sometimes tablet or mobile size doesn’t translate well from desktop. When both of these issues arise at the same time, it’s important to get direction or wireframes from designers to shore them up. Usually a quick chat will do, but addressing the situation before development will save time for designers instead of asking them to context switch whenever there’s an issue.

Are we using standard viewport sizes for designs?

To create pixel-perfect designs, it’s important to know what you’re working towards with design to get the exact sizes for all spacing, fonts, and DOM elements. Desktop is generally designed at 1440px and mobile at 380px, with all assets handed off at 2x. When handing off assets at 2x, if a PSD or XD file is 2880px for desktop, then all sizes must be even numbers to be divisible by two. This helps avoid sizes resulting in decimals.

Is the PSD accessible if a developer needs to slice?

At Roboboogie, all developers can slice assets according to our image style guide. Making sure the PSD is accessible saves our designers time and allows the development team the opportunity to slice and size according to development needs.

What assets need to be handed off by the designer?

Some assets need more adjusting by people with stronger design experience. These generally include SVGs and JPEGS that need backgrounds removed when transforming to PNGs.

Is all functionality understood?

Walk through the test together. This way, everyone understands any added or altered functionality. This presents the opportunity to discover any functionality that may become time-consuming during development, so it’s important to reevaluate the hypothesis and determine if it is meaningful.

Any pages that are outliers and may not be accounted for in designs?

Ok, you’re ready to start redesigning. But wait, how many legacy pages are there? Product pages? Templates? There are sometimes two, three, maybe more! Can we apply the designs to older templates? Do products outside the main categories have a different display? Getting clarity and accounting for future impacts is key to staying in budget and delivering on time.

What are the traffic implications?

Before starting development or design, knowing that the traffic implications might run through a full month’s visitor count in a week should be fully understood and signed off by all stakeholders. (i.e, if there’s a pop-up purchase modal on every page, be prepared to account for that.) The knowledge might postpone or outright kill a test.

What needs to be tracked?

If tracking in Google Analytics, ask the question: have we introduced any new interactions, or replaced old ones? There’s nothing worse than concluding a test and realizing you missed tracking an important variable.

What types of animations do we need?

Knowing how a designer envisions a change in design to give users feedback through design is what transforms a flat design into an interactive experience.

That’s it!

It takes time, and it’ll look different at every company, but keep in mind that collaboration and continuing to experiment with process is the key to winning. Make sure you’re always continuously improving your process, and you’ll see results!

Roboboogie is a Website Performance Consultancy. We provide a measured approach to improving website performance and some might say we’re obsessed with performance. Drop us a line to learn more and see what we can do to help you make sense of your data.

It’s July and Camp Optimization was upon us yet again! This time, we teamed up with Technology Association of Oregon to bring a panel discussion regarding Embracing Experimentation Culture, a subject near and dear to our hearts. Gathered in the Assembly Lounge at Revolution Hall, our esteemed panel of Roboboogie professionals shared their thoughts on what experimentation is, what it means, and how it can be applied to many disciplines.

This month’s panel was Jedidiah Fugle, Chief Operating Officer, Andrea Pappoff, Producer, and Tyler Hudson, Data Analyst.

So, how is experimentation utilized at Roboboogie?

Jed: One of our core offerings is managing web experimentation programs for our clients. It’s being observant, finding ways to improve things by trying something new, and more. One way we’ve recently utilized it is we noticed a lot of siloing going on between our design and development disciplines, and we wanted to encourage more collaboration. We hypothesized that if we provided a daily standup it could increase productivity and efficiency. So far, it’s been working really well, and we’re letting it run its course before we come back and review our wins. Culturally, we’re always trying to experiment with things that will make us perform and do better.

What does it look like to adopt that scientific approach?

Tyler: If you think back to the scientific method, you start by forming a hypothesis. You want to start with a question that brings innovation over time. As far as starting with the experimentation culture, it really just starts with being curious. Ask questions, see what you could improve, and take some risks!

So, you’re using data to make decisions. What qualifies as data?

Andrea: Data can take a lot of forms. You want to take the qualitative and quantitative data, so for example you can take anything from a group of opinions about a specific matter to getting out and interviewing people which leads into the more quantitative side where you can pull data. You can also tie back to best practices by pulling together all the years of experience you have on your team to pick what data points will work best.

Website experimentation is one of the services offered at Robobooogie. Can you tell us what goes into a website A/B test?

Tyler: A/B testing, at its core, goes back to something as simple as testing a red button vs. a blue button and see what works better. We take that to the next level by introducing new functionality to our client’s website, look to streamline different areas or break them apart, and what collectively makes them function better by understanding every point in the customer journey. When setting up an experiment, you want to make sure you’re collecting the right data, so you always want to make sure you’re strategizing with your team. Clear KPIs will always set you up for success.

What roles contribute to experimentation at Roboboogie?

Andrea: One of our critical pieces to having a successful program is bringing together multidisciplinary teams in order to tackle a problem from all sides. It starts with an opportunity we identify in the data, and then relaying that data to a UX strategist or designer who will ask “How can I make this better?”, and then passing a specific design solution onto the developers. Then you need collaboration between development and analytics, to make sure we’re backing up the design elements with hard data. Ultimately, we want to give our clients the confidence to implement those wins.

What is an interesting or unexpected outcome of web experimentation that you’ve seen?

Jed: It’s really fun in this new era. Organizations have become more comfortable with tracking data, which brings up its own challenges in organizing and understanding that data, and with our clients, we track the most minute detail. For example, we created a pop-up modal that displayed an item that a returning customer to an eCommerce site had looked at the last time they had previously visited, but it dropped overall revenue. It presented an opportunity to go deeper into the data to see what happened. It’s a funny question because there’s always something interesting as you go deeper and deeper into what your findings are.

So you’d say these findings have contributed to your client’s interest in web experimentation?

Jed: Yeah! The phrase we like to use is “Data Storytelling.” Our job is taking those spreadsheets, all that huge amount of data, and telling something meaningful with it. Oftentimes people come back to us and say “Wait a minute, I’ve been sitting on all this data and I’ve never done anything with it?” which leads them to ask us what more they can do with it. That builds a culture of curiosity and more data-driven results.

Are there any other benefits to online experimentation?

Andrea: It’s fun to watch a client open up to the idea of testing. Sometimes when we’ve started with a client, they don’t entirely understand what it means, but as time goes on and you see the data and apply their wins, then you see them grow into being the one who offers up testing. It’s really rewarding to see that testing culture grow!

Let’s talk about failure! How do you deal with that? It seems like for some organizations that would be harder than others.

Tyler: Well, that’s why you’re testing. If you knew 100% that something would work, there wouldn’t be a need to test it. Failure is an important data point. If a test “fails,” you can dive into it and find out why, and it will inform smarter decisions down the road.

If I’m going to bring experimentation to my company, how would I go about approaching my manager to convince them to try it?

Jed: It really starts with being curious and having a genuine interest in the scientific approach. Then you might want to find allies, others in your organization who are interested in the scientific approach. Then you would want to have a conversation with a director or manager—unless you are that director or manager!—get the ball rolling with them. The more data you can bring to the conversation the more success you’ll have.

Tyler: When you find those allies and have that curiosity, you’ll notice a lot of momentum will build behind that. When you feel like your voice is being heard, you’re giving them an opportunity to see the potential and your hard work.

Andrea: There are a lot of misconceptions that there is a lot of risk by adopting an experimentation culture but for most of our clients, the program pays for itself. Working with a strategic partner like Roboboogie, we can help inform where to start, especially with simple, high-value tests that help move the needle quickly.

See you next time!

Save the date for our next Camp Optimization, Thursday, September 26. Want to learn more about how we embrace and utilize experimentation? Drop us a line!

Month: July 2019

A/B Test Building: Streamlining Design-to-Dev Hand-offs