When we left off in “How to Combine Complex Data Systems Part 1” we had completed the customering process for a client, finding a single record to represent an individual, and assigning a key id, and were left with one task householding.
Householding can be a more complicated procedure. Once we have contact information, i.e. address, we can start to group customers based on physical address (this is only for retail customers; businesses are treated differently; more on that in a moment). This way we can identify Ben Armstrong and Ben Armstrong Jr as living at the same address; if we want to target households instead of individuals, then we can send a single marketing item to the head of household, rather than each individual. Like the accounts and parties, households are given keys to identify them.
While doing this initially is pretty straightforward, it gets complicated when we have new data. For example, John Smith and Mary live together in our system. If they move to a new address, do they get a new key, or do they keep their existing key? From a marketing point of view, it makes sense for them to keep their key, since the key is really just an arbitrary id used to recognize that they live together. However, what if they move into an address that already exists as belonging to someone else? What if John and Mary move apart; who should keep the key? Either one, neither? These were issues that were resolved by trying to keep end use in mind. Our client wanted to use this new system for marketing. Therefore, we continually focused on how to best set up the system to provide the most value to marketing.
Getting back to householding for businesses, that was more straightforward; each business individual was treated as its own household.
This process also incorporated new data, updating contact information, adding new customers, removing customers who left. This was more complicated, but ultimately boiled down to the original ideas, plus rules for managing changes (e.g. how should a particular record be handled if there’s a change?)
With all that we have achieved for our client, we now have the process running on a weekly basis, taking each new source system when it’s available, and running it through the process. At the moment, the process is largely automated, but still requires human intervention to kick off. This intervention involves multiple people for (usually) brief interactions, some of whom are working from different time zones, which, while minor, add up and take away productivity in other areas. Therefore, we’re currently working on completely automating the process.
Automation will be a multi-technology solution; SAS will be used to do most of the actual householding, but python and bash shell scripting will be used to completely automate the procedure, by monitoring the state of the data, and kicking off the process when it’s appropriate to execute.
Once this is done, we’ll end up with a marketing database that requires very little human interaction in maintaining, and will just be a continuously updated database, with the latest information always added to it, which is pretty exciting.
Learn more about “The Corios Way“, or reach out to president@coriosgroup.com to find out how Corios can help.
First, clients tend to fall into one of three “t-shirt” sizes in terms of their SAS data and workload needs, which tends to be driven more by storage than by number of users or number of compute hours.
