New SEM: 7 campaigns. 40 messages. 120 ads. 532 landing pages. 10 days.

Over the past couple of weeks I took on the role of SEM madman. We hadn’t focused on our own paid search in far too long, so I took it upon myself to dive in. In the process, I applied many of our age-old best practices and made up a few new ones along the way.

Here’s a recap of my experience in narrowing the top of our search funnel and matching messages through the post-click experience:

Getting (Strategically) Organized

First and foremost, we had to define and codify what we were looking to accomplish with our paid search campaigns. We were armed with tons of data, but needed to parse that down to something usable and actionable. I tapped our own site analytics, Google Trends, Wordtracker, and our historical paid and organic keyword performance to form a picture of where we were and where we need to go.

Since we are, to some extent, defining the post-click market, our panorama of potential terms and messages is wider than most. We have to look under more rocks.

I ended up with seven high-level buckets of terms. Within those seven buckets were 40 unique messages. For each message, I wanted to test a minimum of two ads and two post-click experiences (landing pages).

At the highest level, our objective was to improve the quality of respondents from paid search. This included the understanding that fewer people would click, but that better people would convert.

Finding Sanity

Click for larger image.In the face of seven campaigns, forty messages, eighty ads, and eighty landing pages, I needed a system. I’m a spreadsheet kind of guy, so most systems start there for me. This was no exception.

I created an Excel workbook with a sheet for each of the seven campaigns. Each sheet included four columns: keyword; ad; character count; and click-through URL. Each keyword had its own row and there were many keywords within a message. (A message is simply a group of keywords related enough that they can have the same ads and the same post-click experiences hooked to those ads.)

Whenever I wrote an ad, I split the ad-column row into four so I’d have each Google line in its own row: headline; description line one; description line two; and display URL. I then used a formula to automatically count my characters in each row and warn me whenever I exceeded Google’s maximums.

Each message was delineated from the others by blank black rows. With this workbook as my canvas I set out to write my ads. On average, I wrote three ads per message (about 120 altogether).

A page from the workbook can be seen above right.

Taming Chaos with Consistency

What happens in a lot of search marketing campaigns is that the pre-click message gets separated from the post-click message. This is often because the two camps are entirely separate departments or even organizations, but it can happen even within one department. One way to minimize the chance for that disconnect is to name the messages and carry those names through the campaigns. So that’s what I did.

Each message name was used as the name of the Google ad group and the name of the LiveBall traffic source. This way the ‘landing page software’ ad group hooked up with the ‘landing page software’ traffic source and the gods smiled. It’s now quite easy to see the flow from keyword to ad to landing page and visualize the participant’s context. It also makes it easier to process the analytics. Regardless of the number of ads within an ad group or the number of landing experiences tested on an ad group, the message is top of mind.

So, at this point I created a (message-named) LiveBall traffic source for each message and included its URL in my master spreadsheet. I had keywords, ads and URLs — everything I needed to load up Google. And that’s what we did next — creating a one-to-one relationship of campaigns-to-campaigns, and of messages-to-messages. Everything was paused until we got the post-click pieces in place.

Matching and Making Landing Pages

Like I said, I wanted to test at least two experiences per message, so I needed around 80 conversion paths. Of course the resources necessary to create 80 original landing experiences from scratch are enormous. I needed an unfair advantage. (Of course I already had LiveBall which is a huge advantage, but I needed another one.)

Click for a larger imageEnter my favorite shortcut — the Flash object. Using a Flash file that is built around variables (placeholders) instead of real images and text let’s you take control of everything within the Flash without ever going back to Flash development. In a nutshell, you can make incredibly polished looking ‘graphics’ without ever touching Flash, Photoshop or anything else resource intensive. You just associate images and type text — the Flash object applies fonts and behaviors for you and like magic you have perfect graphics.

Note that Flash has only recently become Google quality score friendly for landing experiences. Google and Adobe recently got together to make Flash more ‘Google friendly’ and Flash developers can now make Flash in a way that lets its text be read by Google (just like ordinary HTML). Google now crawls these Flash files and can read their content — making a quality score determination possible.

An illustration of two landing pages based on one Flash object can be seen above right.

Segmentation Strategy

My prototype conversion paths used two different segmentation alternatives. My initial (A) paths tested product segmentation (for us that’s platform vs. services); while my (B) paths tested solutions interest vs. ROI interest (softer than the product-segments).

For the record, our top-line mission here was to narrow the funnel and get higher quality prospects to engage. This meant abandoning the ‘FREE’ messaging (white papers, webinars, etc.) in favor of more direct selling language to attract more immediate and more qualified buyers. That said, the gorilla wasn’t being offered superfluous bananas — if they clicked and engaged, we were pretty sure they were our people.

Click for larger imageSo I had two basic conversion paths. Each experience was about seven total pages and featured 1-3 conversion points (lead capture forms for requesting an ROI calculator, requesting a platform demo or requesting contact). That’s about fourteen pages of web content for each message (A|B test). Multiply that times forty messages and you’ve got around 560 web pages. Hefty.

A screenshot showing one of the 76 landing experiences in flowchart view can be seen at right.

532 Landing Pages in 3-1/2 Days

Obviously I used LiveBall to roll out my 532 pages (using the aforementioned Flash objects) to minimize my pain. I localized each experience by matching its copy and imagery to the ad group. It took me about three and a half days to create and launch what ended up being 76 landing experiences. I did it all myself (because I am a control freak).

What’s next?

Our plan is to iterate challenger landing experiences as soon as winners emerge (with confidence) in each message/ad group. For some ad groups this will happen within a few days, others may take a few weeks. I’ll blog about performance and iteration as we extend the campaigns.

Update on Wednesday, December 10, 2008 at 10:45AM by ion

I’ve fielded some questions regarding testing, tracking and managing all this stuff, so I thought I’d post a screenshot of a performance view in one source of traffic (one of the 40 messages). What you’re seeing is the performance of an A/B test of two landing experiences behind keyword ads for ‘landing pages’ (the most generic and low-level of our SEM terms). You can see the overall conversion funnel (top left), the segmentation profile (pie chart, top-right) and the head-to-head landing experience performance on curves showing 80% and 95% statistical confidence. Click for a larger version.

A performance view of one of our SEM messages in action. Shows an in-progress A/B test. Click for a larger image.

Comments

[Jeroen]

Dear Justin,

What a hell of a job you started with. You are using a really good strategy to set up paid search campaigns including relevant landingpages. But there's one thing: I don't understand that you're using Flash in your landingpages. Isn't it just better to use plain HTML for getting a better Quality Score and decrease your PPC bid? When I use Flash in my landingpages and I'm targetting Dutch Keywords with Google AdWords my CPC bid is to high!

[ion]

Jeroen,

Use of Flash definitely can decrease your score, but you can also use it in a way that doesn't negatively impact your score. Google and Adobe worked together to help solve the past pains, but you have to make an effort in the original FLA file:

In your Flash object source (your original FLA in CS3 using AS3) your developer should make sure that all text in the Flash is set to be both selectable and to render as HTML (these are both properties on a text object that must be manually and optionally set). You can tell immediately on the frontend (final render of the Flash SWF in the page) if this has been done: If you can drag your cursor and select the text, you're in good shape (on at least the selectable piece).

When text is selectable and rendered as HTML it's also crawlable by Google — which makes your Flash-specific quality score issues a thing of the past.

Thanks for reading. Good luck.

[Intermediate Statistics]

Justin, this sounds very cool as an approach. Let me ask a nerdy statistics question. With 532 landing pages and 120 ads, there is a huge net of possibilities created.

How do you decide what sample size is adequate to reach a conclusion on which message/advertisement/landing page is a loser and which is a winner?

I find that in building design methodologies, few industry people are willing to think hard about the statistical validity and issues raised by their variables.

In my early days with adwords, I designed a concept called a pool tool which leveraged the data to produce more robust conclusions with less data. I think the concept is both still valid and still exotic. And important.

[ion]

John,

Each message links to one A/B test. In this case, there are two landing experiences being tested for each of the unique messages. We are using an 80% confidence interval (matches Google's CI) to determine the winning experience at the message level. The messages get varying amounts of traffic, so we'll have confidence at varying times. Also, as you know, confidence is reached sooner when the performance differences are greater, so that impacts timing.

Our LiveBall platform handles the testing and the stats. We tell it what CI to use and it does the rest. We have complete transparency even in cases where we want the software to automatically optimize the traffic for us. Have a look at my follow-up (posted above) to see a screenshot of all of this in action.

Thanks for reading. Cheers!

[Searchengineman]

Hi Justin,

This is a nice plug for Liveball, but your quote.

"Note that Flash has only recently become Google quality score friendly for landing experiences. Google and Adobe recently got together to make Flash more ‘Google friendly’ and Flash developers can now make Flash in a way that lets its text be read by Google (just like ordinary HTML). Google now crawls these Flash files and can read their content — making a quality score determination possible."

Please state your source. As our experience is that flash is currently worse than useless. Its fatal. I have yet to see evidence of an all flash site ranking..SEO or using PPC. The only source I've read was in June that they were working on it. If there is new information, please prove me wrong.

Thank you.

Searchengineman

[ion]

Searchengineman,

Recent changes to Flash (as of CS3) aren't the big story here. There have always been good and bad uses of Flash. In my fourteen years of creating landing pages I've seen no correlation between Google Quality Score and usage of Flash.

That said, here are a couple of ways I've seen Flash take the heat when it shouldn't have:

The score is primarily affected by relevance, but it's impacted by other things—among them—page load time. Since many Flash implementations are heavy, the pages take a long time to (fully) load. This could be misinterpreted as a Flash problem, when in fact it's a general usability problem and one specific to an implementation. Heavy pages will never get a high score—Flash or no Flash.

If Flash is set to render (as far as crawlers are concerned) as an image, then it cannot be read. I mentioned this in my post, but it's critical, so I'll state it again—text must be rendered as HTML and selectable. These are explicit options chosen by the Flash developer for each text element within the Flash file. If this isn't done, Flash wrongly takes the blame. Not invoking these options is the same as putting up an image and not readable text. If Google can't read it, Google can't score it.

Our use of Flash is judicious. We use it to provide polished typography and interactive rollover effects. In my experience, anything beyond that on an SEM landing page is suicidal—but not because of Quality Score. SEM is the quickest of all online media. When people don't know you and aren't yet engaged, they have a very short attention span. We can't even animate up a reveal without negatively impacting our SEM conversion rate.

Here's what I can tell you specifically about our recent SEM efforts: Our Google Quality Scores range from 4-9 on all of our keywords (all of the landing pages are based on a Flash object, so all of the text on that first page is rendered in Flash). Out of 129 KWs about 90% of them are 7-8. All use the same Flash object, differences between them are solely in content and relevance to their keywords. We can improve the lower scores by affecting the content within the Flash—swapping the Flash for HTML has no affect.

We test constantly and have upwards of 60 clients doing high-volume SEM where Google Quality Score is of paramount importance. We have seen no correlation between Flash and GQS.

Hope this helps. Thanks for reading.

[Emily]

How did you optimize keyword bids? Did you use a bid management tool?

[Steve Chambers]

Great post! Very informative.

PS. Nothing wrong with being a control freak! :-)

[ion]

Emily,

Nope, no bid management tool here. I go through a report every few days and make decisions about the bids, positions and ad creative. It probably takes me about 30 minutes every three days to review and make minor tweaks.

Ours are generally pretty unusual keywords which makes the bid management much less of an issue—as the volume and competition are reduced.

Thanks for reading.

[ion]

Steve,

Thanks for the kudos and for reading. I'll be posting a performance look at this campaign in mid-January.

Cheers!