Go Back
This week, we’re chatting with Roboboogie’s very own data and development wizard, Darius Capizzi (also known as Experience Engineer). What does he do all day? What is in store for the future of A/B testing? Who is the 501st legion? Learn all this and more in the full interview below.
How would you describe what you do?
I engineer experiences. I implement tracking for Google Analytics, usually through GTM. I have built several microsites for campaigns. I also build tests in Convert Experiences, Optimizely, etc. Since usually that means running a test over an existing page, I have built some pretty dynamic and unorthodox stuff on several different sites in order to create seamless experiences for the user. I think it’s a unique perspective on web development, since I often don’t have the same options someone working server-side would have. I also occasionally gaze longingly at dogs playing in the dog park (see above), and I always listen to Tyler.
What advice do you have for folks who want to get started with A/B testing?
Consider your KPIs and communicate them to stakeholders early. Education and involvement across the organization can avoid complications later, more directed campaigns, and most importantly better implementation down the road. A winning test unimplemented is money lost. Ensure you have audited the site’s tracking, traffic and larger functionality. Whether you internalize the practice or look for outside help, these items are crucial. But maybe most crucial is realizing how much fun it can be!
What is one test you have built that proved to be especially difficult? What made it challenging and how did you solve it?
A true single-page application (SPA) can be a beast unless some endpoints are surfaced. It can be a bit of an investigation finding the correct scope. I talk about potential solutions in this previous article, so I won’t expand too much.
One of the more complex multi-page application tests I have built was a super navigation. This particularly difficult because the client changes products pricing and categories regularly for sales. Their product list API was limited in that it did not contain a products category, or sub-categories. By querying for category pages themselves, I could see and group items into categories and their sub-categories. Combining this with the product list API, and a set of hardcoded data points, I was able to dynamically generate a complete concise product list. Obviously these calls took a bit of time, so I hardcoded the final data set, then make the appropriate calls to update it while the page wasn’t blocked from load. This way we had minimum flashing, and accurate product listings.
What is the most difficult type of site to develop multivariate & A/B tests for?
Every site is very different. I have seen “single-page applications” that behave almost exactly like a multi-page one would. I have seen multi-page ones that update content so often, that it might as well be treated as if it was a SPA. No matter the technology or the implementation, the key is communication and flexibility on all sides.
What digital strategy or A/B testing and personalization trends are you currently most excited about?
I am most excited about the trend towards email marketing. It allows for personalized experiences through considerate remarketing. When someone signs up for your email list, they are already showing interest in your brand – even if it was just for a coupon. People are in their email every day, and it is a primary form of communication in the modern world. Remarketing also allows for more consistent experiences where, for instance, a certain email campaigns can lead to a specialized, targeted landing page. Email traffic will tend to convert at a higher rate, can be more personalized to a user’s actual engagement, and costs less in the long run. This may not be the case for every brand, but I think it will be a much larger channel in the future, and is worth another look.
The Safari ITP 2.1 update recently added complications within testing and personalization programs in that users can only be identified as return users for 7 days on the newest Safari browser versions. Do you believe this will be a continued industry trend? What is your perspective around user-privacy within testing and personalization?
Yes this will likely be the trend. Cookies are dead, long live the (HTTP) cookies. Apple will not likely back down, Firefox is following, and A/B testing platforms (+ everyone else) will have to figure it out. Please refer to this article for more on the specifics of this functionality.
Privacy is key to analytics implementations, and so is being educated about it. What qualifies as PII (Personal Identifiable Information) and/or Personal Data can be vague, tiered, and up for interpretation. It is important to be mindful, and keep a watchful eye out for changes. When it comes to testing, sensitive PII isn’t particularly valuable. We are more focused on onsite behaviors. Though remarketing ads may be driving more traffic to the site and we report on it, it is not integral.
When would you recommend an organization work with a company like Roboboogie to manage an ongoing Test & Optimize program? What are the advantages?
It can be hard starting an unfamiliar initiative internally. There are a lot of details, roles, and disciplines to figure out. We have been running campaigns for a long time, so those more technical solutions are built into our process, and even advanced past the typical implementation.
For instance we use the javascript api built into Convert Experiences, to combine all active tests into one custom dimension based on test name. So that if you were running [ROBO-1] and [ROBO-2], and the user was bucketed into both these experiences in variation 1, the custom dimension would read T1V1T2V1. This way we can use less custom dimensions, build segments quicker, and more easily analyze cross-test influences.
Roboboogie is able to jump start a testing program, give guidance, best practices across disciplines, bring attention to it’s advantages, and make you look good doing it.
What is something you would like to tell the world about Star Wars?
In the beginning of a New Hope (BBY 0), the stormtroopers that entered the Princess Leia’s ship are the 501st Legion. They were the first troops assigned to Anakin after he became a Jedi Knight. Following the fall of the Republic they became Vader’s Fist. Also the “most comfortable chair ever designed” was basically a floating pillow that Han Solo had in his Sky House. Learn more about pillows here.
Thanks for dropping some knowledge on us, Darius! Wanna know even more? Check out more interviews with the Roboboogie team.
