Managing a Modern Marketing Database
For about a fifteen year stretch, starting in the mid to late '80s, database marketers had it easy.
There were only two channels of consequence to worry about – direct mail and telemarketing – and the technical requirements for supporting them were almost identical. Weekly marketing campaigns were implemented as batch processes that ran against the marketing database for several hours: selecting qualifying consumers, assigning keycodes, writing them out to a file, then sending it off to a lettershop and/or outbound telemarketer. Once a week or so, the database was refreshed with the previous week's activity (orders, cancellations, inquiries, etc.), address hygiene steps such as CASS and NCOA were performed, duplicate names were merged, a matchback process linked orders back to the most likely promotion, reports were generated, and the cycle began anew. Marketers tooled along on cruise control, blissfully unaware of the digital age chaos looming just around the bend.
But that was only the first lap in the database race. Comparing marketing needs then to now is like the difference between driving a dump truck and racing in the Indianapolis 500.
Modern marketing databases – or more aptly put, marketing platforms – are asked to power a vastly increased workload. In addition to yesterday's old-fashioned batch queries, digital media demands an interactive, real time capability called On Line Transaction Processing (OLTP). Present-day campaigns create hundreds of segments targeting both customers and prospects for a series of personalized offers across multiple channels: direct mail, telemarketing, email, and SMS text. These campaigns are often coordinated with integrated social media updates to a corporate Facebook page and Twitter account.
Campaign responses not only include buyers and inquiries, but email activity (opens, bounces, click-thrus, and opt-outs), website metrics, and even SMS text replies. Interactive elements to the system are now standard fare: web surveys, consumer preference centers, call center lookup functions, website interfaces, and so on. Meanwhile, the thirst for more real time business intelligence (BI) is unquenched, only temporarily satiated with web-based dashboard reports.
About OLTP vs. batch: the difficulty in serving these two masters relates to how the data is organized and stored. Trying to use one design to meet both requirements is like asking Red Sox and Yankee fans to get along – it ain't gonna happen, and it could get ugly. That's because a system optimized to quickly crunch through massive amounts of data doesn't work very well when asked to flag an email solicit with a bounce in real time – ESPECIALLY when doing both things at the same time. Using a database designed for batch processing for OLTP functions is like starting an assembly line to build a single car.
The answer? Don't try to do different stuff with one database. Modern relational systems let you create multiple databases on the same server, each tuned for a specific marketing function. Thus, one database might support BI reports, another can be linked to an online consumer preference center, while a third is used for email blasts. For especially large and active installations, even multiple servers can be synchronized, allowing these functions to be load-balanced across different machines.
The key to staying on top of this organized chaos is to make sure there's only one system of record (SOR): the good old marketing database that we started with. All feeds to the marketing platform need to be applied first to the SOR, which then gets propagated and transformed as needed to update the dependent databases. To maintain data integrity, a well-designed marketing platform distributes data in a top-down, hierarchical manner. That said, certain components of the marketing platform work in a bi-directional fashion, both receiving data from the SOR and passing data back to it. An example of a bi-directional component is the preference center, where new consumers added to the SOR need to be passed down to the preference center, and consumer opt-outs within the preference center must be passed back to the SOR.
Yesterday's marketing databases have been transformed from single-purpose, batch oriented systems into highly sophisticated marketing platforms. Technological advancements combined with modern design methodologies allow these platforms to support simultaneous and diverse functions such as tracking email opens, generating BI reports, serving up data for web applications, and running large, complicated campaigns. Much like the engine in a finely-tuned racecar, marketing platforms have many moving parts, all of which must be delicately balanced, perfectly synchronized, and periodically tuned to keep businesses humming along smoothly. Without such an engine, multimedia marketers are left out of the running.
Jeff Fowler is president and founder of Decision Software.
