DataTools Articles

Last week I went on a bit of a road trip, and during that time, managed to catch up with the Team @ DataTools in Sydney.  It was great to see them all, and it was even more enthralling to touch base about their latest venture.  That being, they’ve managed to put Mice into Space!  Well, sort of…  DataTools have morphed existing technologies, to create a unique hybrid solution.  This system has then allowed them to “host” a server in the cloud, for their respective customers to access.

In simple terms the solution is this;  You’ll be able to verify your transaction data against the Rapid Addressing Tool (RAT) day in and day out, against a server in the loud, that you DO NOT have to manage!  And that’s the key; you DON’T have to do a thing.  All you need to do is point your app to the server, and with a few lines of provided code from DataTools, you’re up and running.  It’s all accessed through web services, so anything transactional based will easily slot in.

This process is so slick, that when it comes time to update the RAT, you won’t even notice.  DataTools will take care of all the updates, and seamlessly update the system with zero downtime.  I won’t get too technical on how this is done, but it’s truly a great setup.  What I will get into though, is that the technology behind this is solid. Firstly, the servers are hosted in some of the world’s largest and most renowned data centres.  They’re built on Windows Server 2008 framework, ensuring compatibility and ease of administration.  Each server you deploy is yours and yours alone, you are not sharing.  And another aspect of this system that makes it so powerful, is that there is truly no limit to the power you’re able to deploy!

Most users will want at least 2 or 3 servers deployed for their needs.  They’ll want this for two reasons: transaction power, and redundancy.  Redundancy is vital, should the worst case occur and a server go down, there will be others to pick up the load.  And these redundant (or clustered servers), can be in different data centers, so it truly is a mission critical solution.  But here’s the beauty of this solution; If you need 3 servers, no problems.  Need 10? Easily done!  Need 50? JUST AS EASY!  The way that this has been setup, is that there is no real world limitation to how many servers can be placed within the fail over cluster.  It’s truly an awe inspiring, powerful and completely dynamic solution.

But I haven’t even got to the best part yet…the cost!  The cost is unbelievable.  If you factor in what it would cost your organisation to purchase an apps server, setup an apps server, administer that apps server, and power that apps server.  Then you’d be looking at ~ 6-8k per annum, once you factor in the cost of labor as well.  For commercial reasons, I can’t disclose the actual cost of this solution, but what I can say is that not only is the DataTools solution easier to maintain but it is more cost effective. It would certainly be worth your time to make the phone call and touch base with the DataTools team to find out.  I know that we’ll definitely be using this solution in up and coming projects, so how exactly will you use it?

There has been a shift since we started in this game, where the people we speak to about DataTools products has moved from the Marketing Team, to predominantly the IT Team.  This shift has meant that the features being requested, and the ultimate implementation of the DataTools APIs has changed.  We’ve been used to working with the Marketing Team to meet their requirements regarding the management of data, streamline the handling of lists and duplication, and outlining the cost benefits of a robust addressing system that works with Australia Post.

This shift however, has helped spawn the latest iteration of the Rapid Addressing tool API.  It’s a full blown reverse predictive search, just like you get through Google.  You can check out a demo of it here, on the home page on the right hand side in the middle.

But what does this shift mean for the industry?  For some, it will make a great deal of sense, as this is how an address is often relayed to the person entering the information.  It will also allow you to deal with the whole “Vanity” suburb issues, such as Chevron Island is really in the suburb of Surfers Paradise.  I suppose we could say that it’s a more ‘natural way’ of relaying address information.  There is however one drawback; If you incorrectly type something, at any point, your ability to then match to an accurate address is eliminated.  This isn’t really a failing of the software, but an issue with the way the information is being entered.  It may seem contrary to ask for a suburb first, but this allows you an immediate and robust way of determining if the information is accurate or not.  There are often times however, where someone is adamant that they live in a particular suburb, but they don’t.  This new way of entering information can assist with that type of person.  On the Gold Coast, Cronin Island would be a perfect example.  If you can find me one person that lives on Cronin Island that actually states they live in Surfers Paradise, I’d be VERY surprised!

If we delve more into what this shift means, then we’d start to understand what the driving forces of the business are.  It has been, for the longest time, that various business units would drive IT to meet their needs.  This will still be true in a number of organisations, where getting hold of an IT resource is like hitting the jackpot.  But for others (think Google and Apple), we’re being delivered products that we didn’t even really know we needed.  Another great example, outside of the IT industry, is BMW.  They have had a long tradition of building some really great staples, but have since branched out into niche markets (think X6 & 5 GT).

So, the innovation of a business being driven by IT, is it a good thing?  Or is it just another case of “The Cart Before the Horse”?  It’s not for me to judge, but it should be something that you all question.

Throughout the ‘day to day’ of The Data Company, we’re often having to deal with both sensitive and valuable information/data, given to us by our clients to work on.  However, far too often this data is just handed over freely and without care or consequence.  There have been far too many instances where we’ve had to insist on having a Mutual Non-Disclosure Agreement (NDA) in place and it just shouldn’t be this way!  The issue as I see it, is that as a nation we’re a trusting bunch, and don’t think that others will do anything untoward or against us.  Unfortunately, this isn’t always the case;  This is why we insist on a NDA at all times.  This NDA takes care of two important factors; 1) It provides a level of certainty and understanding for the client, as to how their data will be used, and conversely what it will NOT be used for, and 2) It gives us a barrier against how we perform tasks from being disseminated and given out to our competitors.  It also goes someway to show that we do care about the client relationship, and that we are a professional and morally grounded organization.

So when you’re working with a Data Management provider(no matter who it is), or anyone else that has a need to take your data offsite, ensure that you have a Mutual NDA in place.  This will offer you a level of protection should something happen, and also set some effective ground rules for how the data will be used.  It’s critical for you to understand what your data will be used for, how it will be used, and where it will go.  Knowing these will ensure that your business is kept safe, and that you retain your competitive advantage in the market place.

Also ensure that you’re only providing the data you need to provide.  i.e. If you’re looking to have your address points validated for accuracy, then just apply a Unique Identifier (UID) to the data, and only send them the address points in questions.  If you fill out your spreadsheet with an incremental UID, and order it by this, you’ll then be able to easily copy and paste the results back from your provider.  Just ensure that they’re providing the detail on ALL the information, not just those that they can validate.  This ensures that you’re receiving the information you want, in a format that requires only basic Excel skills.

Data Management tips and tricks aren’t just for when you’re working on your own data internally, they also apply to how you manage and handle the data when it is sent externally.  So the next time you’re about to embark on a Data Management exercise, and you’re provider hasn’t offered you a NDA, make sure you get one and you’re clear about its contents and how your data is going to be treated.

I subscribe to the theory, that we are all smarter than a computer.  We may not be as fast as one, when it comes to processing information, but if you look at who is telling who what to do, than I can only deduce that we are superior!

What we are not so good at though, is repeating tasks, minute after minute, hour after hour or day after day.  We tend to find ourselves spacing out, and loosing focus.  Computers on the other hand do not have this problem!  They’re happy to churn away at the same task, time and time again.  In our business, I have a very simple mandate, something that has served me well for my last couple of jobs as well.  If we can’t automate it, then we’re not doing it!  I am loathed to put any effort into a task that is “one off”.  If it is truly of that nature, then how important can it really be?

There are a number of tasks, that we must deal with daily, when it comes to data management.  Some of those are prior to us even being able to manipulate the data (think data entry), while most are done when this task has already been completed.  Data Entry, is probably the Achilles heal in any database.  Not only due to the fact that a Human has had to manually enter the information, but quite normally, multiple Humans have been involved.  Each with their own little quirks, and individual ways of completing the tasks at hand.  One of my favorite Data Entry experiences, was seeing the office of a Chinese Data Entry company.  They literally had 100’s of operators!  But they had a very unique form of data entry, something that would be too costly to replicate in Australia.  Not only was the information being data entered being double keyed (two different operators key in the same info), but each person was a specialist on a particular part of the form.  I.E, one group would only key First Names, while another the Last Name, and then another the Address, and so on.  It really was an incredible site.

So, back to Australia;  Here we have limited resources, and we need to ensure we maximize what we have.  To this end, something like the Data Tools Rapid Addressing tool is a sure fire way to ensure consistency of information being entered, and a reduction in required key strokes to enter that information.  These two factors alone, will make the backend Data Management teams job , that much easier.

If you then look at some of the repetitive tasks that you’re constantly undertaking, an investment in time to automate these will not only ensure a future time saving, but it will also ensure that the same output occurs each time.

Remember, if you can’t automate it, does it really have to be done?

Validate on fact or fiction.  It’s not always a clear choice!

In this day in age, the access we have to Name and Address data is lessening. From legislative requirements and implications, to companies realising the asset that they have, it is becoming harder than ever to secure an ongoing supply of credible data.

When I say “fiction”, I’m not talking about make believe, I’m actually referring to a reality that we can only assume.  A reality that takes facts and then assumes additional facts about them, to create a state of virtual “factness”(yes I realise that’s not a word!).  For instance, if you had a person living at an address in 2005, and you wanted to know if they still lived there today, what would you do?  If it was only a single person, then a phone call would clear up any doubt.  If you had 100 persons of interest, then 100 phone calls would have to be made.  But what if there were 1,000, 10,000 or even 100,000 records that needed this attention, and the attention that they sort was day in day out?  What would you do?  What could you do?  Often it becomes an overwhelming issue that is often buried and put into the “Too hard basket”, or simply left for another day.  I believe though, that there is a potential way to solve this dilemma, but it’s not for everyone (you’ve been warned!)

Let me walk you through the steps to this reality;

Step 1) Take all the available data, within the areas specific to the data you want to validate.

Step 2) Prove that someone else has moved into that residence.

Step 3) Remove those that have had a new person enter the address.

Step4) What are you left with?

Answer. Two things;  1) All data that you can validate through an alternate source and 2) Data that you can’t prove or disprove doesn’t live there.

You can’t disprove it, because there isn’t a record of anyone else arriving into that address.  However, that doesn’t mean that it’s true.  It does mean though, that you now have a choice to make, a business decision about the risks you’re willing to take to continue to make contact with those records.

Some key questions to ask yourself are;

• Would you be satisfied that the record was viable?
• Would you be willing to spend dollars on that record, based on that assumption?

These questions are not ones that we, or anyone else but you, can answer.  All we could do is to give you the tools and information to get you to that point.  And this then becomes the issue , it’s a judgement call for your business.  Each business will be different; each business will accept a different amount of risk when it comes to marketing.  It’s now up to you to make that call.
And don’t forget, validating data requires some fundamental tools, so don’t forget to leverage the power of the RAT, and the elegance of TWINS III, to ensure that you’re validating the right data this time and the next!

Yep, When. When is a question posed, about a point in time, that generally results in the answer being the lesser of multiple evils; Or a compromise if you will. i.e. When do you bring down the database for maintenance? When do you validate the data? quarterly, bi-annually or annually? When should we procure additional data to refresh our Database?

Your answer to “when”, will factor in different stakeholders perceptions of ‘critical’, as well as the competing factor of the proposed Return On Investment (ROI). But more so, you’ll probably have the deck stacked against you, because you just might not have the fortune to have superiors that truly understand the benefits that effective Data Management can bring to the table.

I’ve spent the last couple of weeks talking to a lot more people in industry, then I did last year. And it’s been a refreshing change of pace to speak to like-minded individuals that get the importance and value that we’ve (I’ve) been placing on Data Management. But how do you convey the importance of this information to people that just don’t get it. I personally love my metaphors, and often draw on what I know (cars). So I ask them, would you ever purchase a car and then never get it serviced? Would you NEVER check or change the oil? Would you ever check the pressure of your tyres? Generally speaking, this gets them thinking a bit, and they can start to understand. But now comes the tricky bit; you’ve got to translate that back to the data you were referencing in the first place.

Above when we spoke about ROI, it’s generally pretty easy to convince someone that spending money on data that is already making them money, to help them make more money, is a good thing. These are generally the “low hanging fruits” within the database, and represent the best chance for quick revenue generation. You could look to them for upgrades on their existing products, additional units of products that they have, or expand the client into different products that might be newly released. What ever the final approach, ensuring that you target these customers efficiently is important. Ensuring that your method of communicating to them is cleaned (such as validating phone numbers or AMAS verifying addresses) and making sure that no duplicates exist. Basic applications of what we’ve been talking about for the last year, throughout these posts.

We want to ensure that not only do you have the tools and knowledge to do your job effectively, but that you are also able to articulate your needs and challenges to those to whom you report. Effective data management is what we all strive for, but this can’t be done without effective communication and Managing UP!

Direct Mail, and why it should be a part of your marketing mix…again.

Direct Mail(DM) used to be the main stay of many organisations, some of which would be pushing out up to 250,000 pieces of DM  a week!  Yes, A WEEK! I know this, because I was personally involved throughout this time, in two separate organisations.  Back then, stamps were around $0.45, and once discounts were applied through barcodes and sorting, it was even cheaper.  These days, with stamps at $0.60, it’s a different ball game.  No longer can an organization afford to send out the same volume as before, and nor is the response rate able to justify the cost.

But, I believe that there is a place still for DM.  It’s a place that provides opportunity that can’t be sought or derived from ANY OTHER SOURCE.  Think about what happened in May 2007…time’s up…In May 2007, the federally funded National Do Not Call Register (DNCR) came into existence, and it changed the Direct Marketing landscape instantly. Since then, Data Provides, Data Managers and Telemarketers alike have been scrambling(stressing out!) to ensure compliance.  As of December 2011, there were 7.13 million registrations on the DNCR. Given that there are approximately 8.6 million households in Australia,(projection to June 2011) this is a staggering number!

So, if your current marketing mix is focused purely on Telemarketing, what are you doing about all those people that you can’t call?  Email is an obvious choice, but this is tightly controlled by the Anti-Spam legislation, as is SMS.  This makes it VERY difficult for businesses to make contact with prospects through these electronic channels.  Even when the list is opt in, or they’ve expressed their consent, you can still be hit with “Please Explain” letters from ACMA.  The only sure fire way to connect with these people, without breaching any laws, is to send them a DM piece.  However, be targeted when you do so.  There is little to gain in polluting the DM waters with propaganda.  Find yourself a targeted list and cherry pick the best, to achieve the best!

Once you’ve got the right data, test, test and re-test.  Testing will ensure that your creative, copy, and even your envelope will not stand in the way of having a successful campaign.  When testing, ensure that your ONLY variables are those that you’re testing.  Test using a quantity of data that will allow you draw some accurate conclusions, and ensure that you’ve got the tools in place to track and record this information.

One of the great pieces of kit that DataTools puts out is the TWINS III desktop suite.  This product, in some of it’s different iterations(contact DataTools for full details), can be empowered to handle all of your mail processing requirements.  This includes ensuring conformity to Australia Post standards, Barcoding (as well as including customer info in the barcode for RTS purposes), sorting, and manifests.  It really is a one stop shop for mail-house work.  It’s the perfect accompaniment to either an in-house mail program, or a dedicated Mail House.

So get out there, put in an extra channel, and start to stand out from the crowd!

With QLD under fire again from rain and flooding, it gave me an idea for a good segway into data making its way into your database….

As Data Managers, our aim is ensure the integrity of the data we manage.  We look at what is there currently, ensure that it meets all the required guidelines, and is suitable for the business purposes for which it’s required.  We’ll often go to painful lengths to maintain the integrity of data, working through problems created for us, and of our own creation.

One of the challenges that we’ll all face, is data entering our systems from sources we didn’t initially intend, or via means that we were not aware of till they occurred.  Users, in general, will find solutions (read ways around) to their problems.  Often these “problems” are security measures WE put in place to protect users from themselves, or to protect the overall integrity of the data for the sake of the business.

As much as this is an annoyance, and often a breach of IT implemented policy, we should learn to embrace it and move forward to solve the particular challenge for that person/department.  I believe that far too often, we’re too quick to slap the wrist of the person who proved to be smarter than us, as they found a way to get the data into the system without our intervention.  I would suggest, we’ve all been there, and at some point in time, have all been that person who “found a way”.  Of course, I’m generalizing, and at the same time alluding to my own proclivities, but that is in essence what we do.  We find ways to solve problems relating to data.

So now you have an issue where a segment of your database effectively doesn’t conform to the remaining data beside it.  There are a number of ways to resolve this, and at the same to ensure that the unauthorized data upload hasn’t created a duplication problem within the database.  If you’ve managed to be alerted by, or alerted to this problem early, then the fix can be quite easy.  If however the time delay has been significant, then we could be talking about transactional data now residing within the newly created records, which in fact are duplicates of existing records.  One of the easiest ways to solve this problem in the early stages is to dump the data and segment it into the authorized and unauthorized data sets.  Using the Data Tools TWINS III app, you can compare these two data sets and choose to remove the duplicates from the unauthorized data set.  Both data sets can then be merged, and uploaded back into the main Database, with no further thought or subsequent issue. (just don’t forget to plug the hole and implement training about importing data)

However, if you’ve not been so lucky, and it’s been some time since the original upload, or uploading of the data immediately causes transactional data to be created, your ability to resolve the issue becomes a little harder. If this is the case, then you’re likely to have access to a main stream Database solution such as MS SQL.  Given this, you can use the power of MS SQL and the DataTools API’s to;

a)      Determine the duplicates within the data

b)      Mark the “Parent” record (with the assumption that all other records will be the children of that record)

c)       “Chain” the duplicates together, meaning that you set a distinct ID that represents the group of duplicates.  This could be at a Name and Address, Phone Number or Email level.  It could also be at all three, it just depends upon the business rules.

d)      Merge the data, being to keep the parent and repoint the transactional records to the parent, by updating the foreign key of the children to the respective parent record.

This is often a little intense, and involves a great deal of care, but the DataTools TWINS III API is a great asset in helping to source the duplicates, as is the RAT API when looking for a standard format within the address.

The easiest fix, is to ensure that this can’t happen.  And for the most part this is probably already in place.  For those of you who were unable to lock things down completely, the above will give you an idea of what steps can be taken to resolve the issue.  For those that are yet to ensure that all input avenues are locked down, the above will provide as an incentive to double time the locking down of your systems!

Welcome back to work and welcome to your 2012 work year!  We’re sure that you’ll have just as much on your plate this year, as last, and that your holiday (if you were lucky enough to have one) just wasn’t long enough

So, 2012, it will be the Chinese Year of the Dragon, so look out all your Ox and Goats!  It’s also up to you what Data Management goals you’re going to make for this year.  DataTools is continuing to strive forward, with the launch of some key products in 2011.  Namely the Correct Address International, a SAAS product targeted at online store vendors that ship internationally and a new feature of the Rapid Address Tool API, that allows for Single Line Predictive Address Search (similar to Google Search).  These two cutting edge products are designed to not just make your life easier, but to provide cost effective solutions within your organisation.

And this becomes an interesting point; Often the cost of implementing a product, or the costs to be saved by implementing a product, are not as important as the “Experience” that is then created by that solution.  As a business you could look at the cost of sending an item out, be that something bulky or just a letter, and then work out the cost to send out that exact same thing again because the address was inaccurate.  There is a cost associated with this, directly as far as postage is concerned, and indirectly through additional processing time and hours of labor.  But that’s often not the real concern, or in other words, those dollars spent are represented in the cost of the final product anyway so it becomes a less relevant issue.

The real issue for some, is the ability to be able to better service the customer.  Customer service in Australia always seems to be under fire, and depending upon the industry you’re in this will be more of a concern to you than others.  If you’re in Banking, a Telco, or deal with customers that then rely upon you for after sales service, you’re ability to meet those expectations will significantly improve your brand in the market place.  Conversely, your inability will have the opposite effect.

Think of it this way, if you had the chance to implement a simple piece of software, or even to the extent of integrating into an existing solution, a process that would 100% guarantee that you would get the accurate address at the time of ingress to the system, and that this in turn would mean that your customers received the goods/services delivered on time and when they expected them, would it matter what the cost was?  For some of you it still would, but I can tell you that the cost of the Rapid Addressing tool is not going to be a limiting factor.  I’ve put together some additional material this week, which highlight the reasons why the DataTools Rapid Addressing Tool was implemented by your peers Click Here.  They’re certainly worth the read, and will give you a far better look into the window, than what I can convey.

Best wishes for 2012!

Welcome to the last Communication piece from us this year.  We’d like to take this opportunity to thank you all!  And from the team at The Data Company and DataTools, we very much like to wish you and your families the best for the Christmas Period, The New Year and 2012!

So, onto this week’s piece…

I’ve been reading up on how to better target this communication, and to see what can be done to set it apart.  What we all see in our inboxes on a daily basis is aptly referred to as ‘Noise’!  So how do we differentiate that noise we want to hear, say ACDC or Sneaky Sound System, from that which we don’t, such as Mariah Carey’s “All I want for Christmas”?  It’s hard, and it’s often a judgment call made on the fly.  It’s usually predicated on how much time we have at that moment, if we’re hitting the 3pm anti-caffeine wall or if the emailer is really clever with a subject heading so captivating, that we feel our lives wouldn’t be effectively fulfilled without reading the content!

So how does all this relate to DM?  Simple, as a part of anything that we do on a day to day basis within the data we’re managing, we’re ‘reducing the noise’ that is contained within the respective datasets.  This “noise reduction” is a critical component, and certainly not something that should be glossed over.  So in reference to DM, how can we “reduce the noise”?  The solution is as obvious as it is simple, Data Validation.  Through the use of the DataTools Rapid Addressing Tool(RAT) no less!  This is an excellent and cost effective place to start, but by no means where you should finish.  But it is where you will likely start.

Essentially, the place to start is the same place where the data is created.  This could be an online entry form, a data entry office or a call centre.  Regardless of the point of entry, if you can resolve the majority of your data concerns prior to them being created in the Data Base, then you’re a step ahead.  Two fortnights ago we focused on rules and education;  Remember this, and always strive to empower your staff/vendors through education and consequence of actions.

The DataTools RAT like I’ve mentioned times prior, is an invaluable tool that will ensure compliance with Australia Post, and provide an effective means to parsing your data consistently.  This is your primary step in noise reduction, from a “backend” perspective.  But don’t forget, the RAT can also be integrated into your Front End apps just as effectively.  Here it can be used in Data Entry, or as a standalone app on a desktop to reduce key strokes and increase accuracy.  It truly is a functional app!

If you can do this, and reduce the “noise” in your datasets, then you’ll have

• More accurate data for Analysis
• Effective and Accurate Reports
• More effective communications
• Reduction in duplicated effort (such as sending mail to the same persons multiple times unknowingly)
• And an increased ROI from your data.

At the end of the day, reducing the “noise” and being effective in your Data Management practices just makes sense.  And the great thing is, it’s NEVER too late!  So get involved, and listen to the sweet sounds of your data singing in harmony with the business!

We have used DataTools TWINS III, for the last decade, and have found it to be an extremely effective “out of the box” tool.  It is something that we use every day, for both internal and external purposes and is one of the most used pieces of tech within our environment.  The TWINS that we’re referring to is the API iteration.  However it should be noted, that you can create custom keys within TWINSIII Desktop, for your own exclusive use.

TWINS, and the match keys that it creates, are more powerful than you might think though.  One of the great things about TWINS, is that you can effectively create your own match keys on different data elements.  One such example comes from a continual struggle we had in effectively matching Company Names within B2B Data.  We found that the different ways that data can be entered, and the order of those entries, just created too much uniqueness for TWINS to identify them as a match.  This lead us down the path of trying to resolve how to match these, without relying upon pure SQL code, or just accepting that we couldn’t do any better job than that.

What we came up with, is a unique way of matching the company names, by splitting the actual name of the organization apart. If we take “The Data Company” as a string, and place it into its separate parts, we would end up with “The”, “Data” and “Company” as three separate strings.  If for example, one entry had “data” and another “datta”, we would still be able to match those two together, even if the composed strings looked like “The Data Company” and “Datta Company, The”.  We found this to be highly effective, and solve our challenges within the B2B environment.  There is of course a little more that we’ve done in the backend to accommodate these changes, such as some proprietary code and algorithms, but this is just to give you an idea that there are ways to implement and bend the use of TWINS to your own needs.

TWINS will continue to be one of the foundation pieces of software in our environment, and should form a part of any “Data Management” solution within yours.  This could be a periodical dump to clean up duplication, to something that runs in real time for highly transactional data sets.  TWINS truly is a tool that will be at home within a “real time” or scheduled process, it’s just a matter of determining what suits your business the best.

We asked, you answered, and now we shall provide. You’ve asked for more Data Management tips and best practices, so that’s what we’ll give you. You’ve also asked for more real world examples and case studies relating to DataTools products, so we’ll mix it up each fortnight and alternate the articles for you.

This week, we’ll focus on Data Management.

And as far as best practices go, nothing is more important than rules. Rules within a database could be such things as constraints(Foreign Keys for example), input masks (Perhaps to standardize a phone number), or more top level rules about what goes where in relation to a table or database. The key to setting good rules, is to ensure that they are strict enough to have a positive impact, but loose enough to enable the business to function dynamically. This may mean that you’ll need secondary processes, to handle exceptions to the rules you set.

In order to set useful rules, you’ll need to understand your business completely. For example, if you operate a business that has dealings in both Australia and New Zealand, setting an input mask to capture a 10 digit phone number will leave you short when it comes to NZ. Conversely, if you set it to 11 (for NZ), then you’ll leave yourself open to drama’s for Aussie numbers. However, if you’re system is smart enough to handle input masks dynamically, then the simple change of the country code would allow you to set this mask dynamically. But here in lies another challenge; What happens when a country moves to a new phone number format? Just like we did back in the 80’s. Putting rules into place will ensure that you have a better than average opportunity at maintaining consistent data, but it will also mean an overhead will be applied to your day-to-day management. It could be that you need to tweak rules, disable rules, or run rule checks on a daily basis to ensure uniformity and reliability. So in other words; What why around your rules has an employee found today? And if so, why did they do this? Most of the time this is not malicious, they are just trying to perform their job as quickly as possible, and with the least amount of effort. Learn from this, as you may find a more appropriate rule by listening to them, or you’ll likely find the hole that needs plugging. Either way, like most things, it’s an evolution.

Rules are of course just one way to impact the quality of your data. The other is through education. More often than not, people are just unaware of the impact of their actions. If you are able to hold regular and informative sessions, that engage your audience, then you can empower them to better service your data, throughout its various touch points. If you’re able to show what impact their actions have, and if those actions were positive, the impact it would have on both them and those down the line, then you might be able to strike a cord, and impact some real change!

These small changes, along with regular analysis and reporting on the data, will ensure that you’re able to keep a step ahead, and know when things are likely to be impacted which would then need outside resolution.

How can we engage with you more effectively?

We want to know!  Our Company prides its self on the service we provide, and a part of that, is giving you what you need.  So please, tell us what you’d like to see coming through each fortnight, and we’ll do our best to tailor the communications for you.

If however, you’d prefer to just answer a quick Poll, please see the poll below:

Ever heard the “it’s not you, it’s me” excuse?  Yeah, me neither!

But in this case, it’s not you, it’s your data.  Data is the cornerstone of any business whether we acknowledge it or not.  If you ask yourself the question, “What would happen if I didn’t have ANY data?”.  Then the answer would be that you wouldn’t know who your customers are, you wouldn’t know how much revenue you’ve generated, you wouldn’t know who you could still prospect to.  It would quite literally fall apart!   What we’re seeing now though, is an influx of online stores, some a virtual store for an already recognised brand, others simply acting as  a retail front for many wholesalers connected via the backend.

So, with online retail becoming more the norm, then the contrary, it should be more and more important for these “Virtual” retailers to solve the problems before they happen.  What’s more, we(customers) are by and large, a bit lazy when it comes to filling out forms.  So if we can have something that helps us to key in the correct information, then not only is it a better experience for the us, the customer, it also ensures that the organisation has accurate data that conforms to a standard.

Last week we touched on “Cloud Based” options, but what we didn’t touch on was the SAAS appliance that is currently being offer by DataTools called Correct Address International.  What you may not know, is that not only is this a great tool for websites that ship overseas, it’s equally at home should the majority of your shipping remain within Australian shores.  I won’t get too much into the mechanics of it all, but suffice to say that with only a few lines of code, you can leverage a web service directly from your form, that you never have to worry about again!  Sounds great right?  And it is!  Those of you in the online sector need to know about this product.  It’s a product that will enable you to reduce erroneous parcel delivery, save on postage, and help to maintain those tight margins that you’re often running to stay in the game.

But that’s not all, and this isn’t some steak knife cliché, but the fact is that with accurate and succinct data you’re able to perform analysis, run reports, and ultimately better target your marketing efforts.  Because now you know where your “markets” are, did you read that…”know”.  Not guess, not think, not “we believe”, “KNOW”!  Nothing drives me more nuts, then sitting in a meeting and hearing all about what people “think”.  Wouldn’t it be nice to turn up to a meeting, brimming with the confidence that comes from you “Knowing” something?  Of course it would be!

So, without this sounding too much like a piece of marketing collateral, jump onto the website and find out more about this product.  Touch base with the team at DataTools.  Apart from your time, it costs you nothing to learn a little bit, about a great product, which could save your plenty!

Is cloud computing the future? Sure, but not for everything. A lot of what we do on a daily basis involves the update or movement of massive amounts of data. These transactions are not only taxing on the infrastructure currently serving us (think ADSL, SHDSL etc), but they are also highly costly in some of the pricing models put out there that charge per transaction.

For my money, I think there is a healthy blend of Cloud and Non-Cloud services that a business can employ. Think of it this way, there are some things that you’d just prefer to not really have to deal with. Email is an easy one; with a number of providers now offering a cloud based service for email (some of them even free), there are few reasons to keep email within the four walls of your office. CRM’s are a potential for being put into that same silo, but many would argue that having a solution in-house is equally warranted. I’m of the mindset that there is a solution that encompasses both.

The thing with Cloud based solutions is, that they are often “set and forget”. Once you’ve done your due diligence in finding the right solution, setup is generally quick and painless and all other aspects of the administration (backup, DR etc) are handled for you. For small to medium businesses, this can represent not only a significant saving of IT’s time, but real dollar$ as well! As an example, Google offers small businesses domain based email for up to 10 users for free! FREE!!! Even the simplest internal solution would mean an instance within a virtualised environment, which is tying up RAM, Disk and CPU Cycles. Not to mention energy. A CRM is no different, there are free solutions out there, as well as low cost alternatives within higher cost solutions. For basic contact management, you could get into a CRM for as little as $5/user/month. Within this you can manage all your interactions, run reports and keep a close eye on where the business is at. If you want more detailed information, then the cost goes up, but at a basic level it will suit the needs of many businesses looking for a solution that can help them keep track of their Accounts and Contacts.

So what does this all have to do with us? Before I mentioned that I’m of the mindset that there is a solution that encompasses both. What I’m referring to is that I believe that a CRM does a great job at being a CRM. Not too often though, have I found a CRM that is also good at Data Management. Generally speaking they have basic dedupe mechanisms in place, but overall are rather lacking any other functionality that we would deem “Data Management” worthy. Time and time again, we’ve found the need to pull the data out of a CRM, work on it, then push it back in. This “off-site” work can’t be done in the cloud, and is really only suited to being within an environment that has the power and low latency communication that only an internal network can offer us today.

Additionally, there are a number of products being offer by DataTools, such as RAT and Twins III that either work best when onsite (as in they’re able to process data faster) or only work within an onsite setup. Do I think Cloud computing is the way of the future? Yes, and with faster broadband and more robust infrastructure solutions, it will be very hard to pass up. For now though, my money is still going into internal systems, developments that leverage internal systems and beefing up our capacity and efficiency. Having said this, we, like you should be, are looking forward and finding ways to move our offerings into a cloud based world. For us, this might mean that we offer some basic and generic services through the cloud, whilst keeping the hard core manual components in-house. Remember, you don’t have to move to the cloud, you just need to be sure that you don’t need to. If you can remain competitive, and offer the service levels that your customers expect, then don’t fix what isn’t broken.

So the last three articles have all been trying to move you all to a place where we start to increase the quality of our respective data sets. That we start to manage these more effectively. That we can have the structure and ability to start resolving the requests for information that come our way. Today will be about, why haven’t you started this yet…?

One of the organisations that I worked for had put a lot of work into its mission statement. One point within that, and to be honest it’s the only one I can remember, was “No Fear”. I have lived my professional life with a healthy respect of these two simple, but powerful words. Having, “No Fear” in the Data Management world, is embarking on a project and failing gracefully. It’s hitting a brick wall of your own ability or understanding. It’s not quite achieving what you set out to do in the first place. And you know what, it doesn’t matter! As along the way, you’ll learn, evolve and surprise yourself and others.

I’ve worked with a few clients now that have taken on board their data management issues internally and failed. But failed in the sense that they didn’t really have the resources or knowledge to do what they knew they needed to do. This is what I want to hammer home today, I’d rather you all attempted to resolve your issues in house (in a sand box of course) then just sat there and did nothing. There are of course going to be a great many number of you that just do not have the resources to do this. You’re so tightly impregnated within the Day to Day, that your ability to focus on anything “Project” related is just not worth talking about. But what of those of you who aren’t? In the last 3 articles we’ve really tried to give you a step by step approach on some basic processes that you can use to improve the quality and integrity of your data.
Just this week, I had an invitation from someone that I’d never met before on Linkedin. This person had gone out of their way to commend me on the fact about how much we care about Data Quality. It blew me away! But don’t get me wrong, I’m not telling you this to blow my own horn! It’s an important validation of our thoughts, principles and guidance that we’re offering to the community. It showed me that I wasn’t talking to a virtual brick wall each fortnight, but that people were actually sitting up and listening.
I’d love to natter on more about the importance of Data Quality, but if you’ve read any of my articles before, you know all too well how I feel about the subject ? So today, I won’t. But I will extend this olive branch, if any of you are even the slightest bit interested about doing something, and just don’t know where to start, drop me a line. I’d rather see you and your organisation get some runs on the board and prove some concepts, then just remain within your stasis of comfort. We’re all here to help, and yes we are also here to make money, but if you’re just plain stuck and can’t move, then no one wins.

So stay strong, remove the fear, and move forward to create something great! And if you do fail, get back up, dust yourself off, and don’t repeat your mistakes!

The third time is the charm, and in this installment, we’ll be taking a peek into the third step on your data management journey.

Four weeks ago we talked about picking the low hanging fruit from your Data basket, and thus providing an incremental gain to the quality within your organisation.  Two weeks ago, we spoke of the merits that Fuzzy Logic can bring to the table.  This week, we’re going to take it a step further, and bring them both together.

Daunting though it might be, providing a successful, dynamic, and robust Data Management solution doesn’t have to do your head in.  It’s essentially a logical group of processes that work towards a common goal.  These are often stepped out, be it in stages, or just separate processes that are dependent upon the prior step.  And to say that they’re dependent, doesn’t mean that you HAVE to do them, but it sure is recommended!  What I’m talking about is Address Validation whereby you’re using the DataTools RAT.  You don’t have to use this prior to using the DataTools TWINS softare to apply a fuzzy match key, but, if you don’t then how do you know that you’re comparing apples with apples.  See, the DataTools RAT will not only validate (and resolve where possible) the address inputs to the Australia Post Postal Address File(PAF), it will also standardise the address into its respective elements.  It’s this part of the process, that ensures you are comparing like for like, or apples and apples.

Understanding how the individual steps work, will help you to then design the final workflow within your system.  The workflow, with respect to what we’re talking about, is essentially the movement of data from one process to another, until it is either in the desired format or arrives at the final destination for its required marketing use.

A good data management solution will be able to cater for a number of data sources.  It will be able to handle unknown header rows of known file types, and be able to dynamically transform this data.  There are complex aspects to some of this type of development, but even putting something rather rudimentary together is going to provide a significant uplift.  But then again, why go to all this effort, when there are software packages like DataTools TWINS III that do just that?

Ask yourself this, “What applications/systems do I know well and know that I can effectively work within?”  Once you answer this, then you’ll have a starting point for where you initiate your development.  And let’s not get freaked out by the word “development”, as your “development” could be as simple as a well documented process that leverages the DataTools TWINS III Desktop version, which then points to a number of key excel or access files.  There are many times in which we think we can create a better wheel.  But for the vast majority, using an existing solution, such as DataTools TWINS III Desktop, will provide all the functionality you need to manage your data effectively.  It’s true that it will not be as elegant as a custom made solution that has the potential to fully integrate with your legacy systems but;  If you have a well documented process that has references to folder structures, naming conventions, and Quality Control components, then it will be a fantastic starting point.  Think of this as “Proof of concept”, or starting point in your journey.  As by doing this you’ll be able to validate, with minimal cost and time, that what I’m saying actually makes sense.  If what I’m suggesting you do works for you, then the next logical step is a combination of integration and automation.  But, let’s walk before we run!

So I know that I harp on this almost every fortnight, but it’s important…Be a doer, and don’t be content or accept sub-par performance out of your data, as it just doesn’t have to be that way!

So you’ve taken your “one small step”, and want to take another.  Or perhaps you want to take a bigger one?  Following on from our last article, there is another fundamental step that you should take.  This is not something that everyone will do, but I believe it is a critical step.  And the step is, implementing a fuzzy logic process to detect and deal with duplicates.

As most of you know, the Rapid Addressing Tool handles all aspects within the “address” sphere.  What a lot of you may not know though, is that the perfect complement to this software, is Twins III by DataTools.  This handy piece of software will take a Name, Address, Phone Number or Company Name, to name a few, and create a “match key”.  This match key is a string of characters that is then easy queried, for the sake of determining the rate of duplication.  You could calculate this rate within the data set its self, or between one data set and another.

If we expand upon last weeks example, of importing a list into our database, it may be important to scrub this list against the database to which you’re importing it into for duplicates.  This could be because you have multiple vendors supplying you with data, it could be your lead generation activity.  In fact, it doesn’t really matter what it is, the principles remain.  By leveraging the power of Twins III in your day to day processes, you’ll be able to better handle disparate data sets, as the Fuzzy Logic of Twins III will ensure that data entry errors are picked up and still marked as matches.  Now I know that we’ve touched on the merits of Fuzzy Logic before, so I’m not going to go into that again.

What we want to talk about today, or drill into you today, is that you need to keep on improving your processes.  If I look back over the last decade, and think of the ways that I was managing data then, to today, there has been an evolution.  It’s as much of an evolution of thought, as it is process.  Sure, there are fundamentals that haven’t really changed; both the Rapid Addressing Tool and Twins III are examples of foundation software that have stood the test of time.  But even these have continued to evolve over this time.

So we don’t want you resting on your laurels, we want you to innovate and instigate the change. If you’re reading this, then you’re either interested in the subject or can play a direct role in forcing this change.  We need a revolution to occur, as that is truly the only way that we can have others embrace the importance of effective Data Management.

So take these articles for what they are, as they are only worth what you yourselves then do after reading them.  If this is just another blog you monitor, or something you read to pass the hours of the day, then wake up!  You have an opportunity to make a difference, to directly impact the quality of your or your company’s data!  You’re in the driving seat, you can take control, and you can be the conduit between us and the decision makers in your organisation that will allow them to see the tangible benefits of improved data quality.

Yes, you could say that this has all the smell of another marketing piece covered in the wool of a blog.  But this is real.  This is a fundamental problem with almost every known database out there, and I don’t know of too many CRM’s that are even close to being effective when it comes to proper deduplication.  Have a look at the data, the raw data, and just do some basic things like ordering it by different columns.  You’ll start to see some disturbing issues quickly.  Dig deeper, and you just may be scared into doing something about it today.  But, if you’re not up for this kind of exploration, touch base with the guys and girls at DataTools, and let them run your data through the Twins III software.  They’ll quickly tell you the level of duplication within your data set, so then you will know exactly how big the problem is.

Whether you subscribe to the “a” or not, the original quote from Neil Armstrong has implications for a great many tasks that we all do on a daily basis.  However, in this example today, we’ll be looking at some of the smaller things we can do, that will have a positive and profound impact on many.  The ripple effect from these small changes will be felt by all users of the data in the organisation, from the initial touch points, through to report viewers and subsequent follow ups through Direct Marketing(DM).

We often hear the phrase, from those that have been and done, to “pick the low hanging fruit first”.  They tell us this, and we in turn pass this knowledge on, so that we can take something easy to fix and solve it first.  This gives the project some runs on the board early, and demonstrates that we actually know what we’re doing.  It’s the same thing with any Data Quality project, if you can make a small change at the beginning of the process, then this will be felt throughout the organisation.  Don’t be disheartened though if no one notices the positive changes you’ve made.  It may not be till you actually point out what has been done, that they actually see it.  And you know, that’s an important lesson right there.  Quantify your issue, validate your results and further quantify your results on the corrected data.

So, the low hanging fruit, where is it? Generally speaking, it’s going to reside in a place that you or your team have direct access to.  It will be something that you can impact without interfering, or being interfered by, anyone else.  I know this is all very vanilla, but each organisation is different.  In order to get you started though, it could be in some of the following areas:

• Initial List/Data import from vendor to internal Data Base.
• Data Capture from web based form.
• Initial Data Capture by inbound Telemarketing agents.

How to “pick” it?

If we take the initial list/data import example, then the solution is as simple as it is obvious.  By using a combination of the DataTools Rapid Addressing Tool(RAT), and some simple SQL code, you can have a quantifiable effect on the address quality through this AMAS certified product.  This product will not only ensure conformity to the Australia Post Postal Address File(PAF), it will also parse the information into the relevant fields as required.  Add to this some simple SQL code that normalises the Names and Phone numbers, and you then have a complete data import solution.

Ripple effects a plenty;

One of the first things to note, will be the uplift in quality for the call centre/DM team.  They will thank you, as they’re now able to clearly see the correct details for that prospect/customer.  This will translate into more conversions, less return mail, and better overall results for the marketing team.  It would also be a fair expectation, that the reporting is now more reliable, as errors in data are now mitigated because of this new process.

So, you enact one small change in your process, which has essentially been that you’ve run your data through a validation/parsing process, and you’re suddenly a hero to the front line employees and management team all at once.  And let me remind you, this is just one small step in many that you can take.

One of the challenges that managers of data face, is the need to ensure Data Quality.  Quite often, this aspect of the business is a secondary thought, and something that most businesses believe they can resolve with intelligent backend processing.  And although there is an element of truth to that, these processes that we so heavily rely on, could be far more successful if the input was cleaner to start with.  Some of you may think that this is in fact an oxymoron.  Why should we go to all this effort on the front end, when we can push it through tools such as the DataTools Rapid Addressing Tool(RAT)?  Simply really, it’s a bit like running your car.  You know that if you have low tyre  pressure, that you will consume more fuel, increase the wear and tear on your car and the car its self won’t handle the same.  It is for this reason that manufacturers recommend that we check out tyres each time we fill up our cars.  So when did you last check your pressures?  For those of you that check your tyre pressure, what have you done recently to ensure that the capture/input of your data is in line with best practice?

The crazy thing is, that to start ensuring that your data quality improves, could be as simple as applying a very high level input mask.  i.e. wildcard@wildcard.wildcard for an email address field or It could be that a first or last name must start with/contain more than 1 alpha character.  These are just simple uses of embedded technology that should be present in most systems.  But be warned, input masks have the potential to cause many issues if used incorrectly. i.e. all postcodes must contain 4 digits, all telephone numbers must contain 10 digits, what if you sell something to someone from overseas?

For those of you with custom systems, or systems that can be customised, then look into solutions like the Datatools RAT API.  This plugin will ensure accuracy of address data AND reduce key strokes, all from the same app.  It is something that can be easily integrated into almost any application.

However, my point today is that we should be Proactive when it comes to data management, and that organisations should move forward with caution when performing major database cleanse/validation exercises.  For as much as this will aid in the ROI for the short term goals, unless there is a long term solution in place to continually administer this, the benefits will slowly diminish, and the data will be back to square one within a short period of time.

So, when you’re looking to tackle a Data Management project that looks at the quality of the data, ensure that there is adequate scope for addressing the root problems.  Those being the data sources, the initial data entry, list acquisitions or day to day transactional updates.  There are many touch points in a business that may need to be addressed, so start with the low hanging fruit and tackle those that will have the greatest impact first.  Backend processing is mandatory, but it should be treated as an adjunct to better practises on the front line, as only then will you have a truly integrated system capable of handling your entire Data Management Hygiene concerns from A-Z.