When data is your product, consistency and cleanliness are more than just nice-to-haves. They can make or break your business.
DiamondStream chose Semarchy to vastly improve the quality of their product and help grow revenue.
DiamondStream is a data analytics company that works with clients to make sense out of transaction information. They primarily serve internet retailers and casino companies who are looking to gain a better understanding of customer behavior and improve marketing efforts.
Dean Donovan, Cofounder and Managing Director of DiamondStream, describes the opportunity he and his team saw when they started the company in 2005. “We were talking with one of the largest providers of customer transaction data to the casinos. They knew they could improve the value of their data if they could add a deeper level of analysis, so we worked with them to build an analytics capability that combines transactional and demographic information to tell casinos more about their patrons.”
In-House Data Cleansing
DiamondStream realized early on that the ability to merge multiple sets of data would be critical to their success. “The casinos already had access to the transaction information,” Donovan explains, “so the value-add for us was in how we could add other layers into that data before doing the analysis.”
“We built our initial solution using Birst as the business intelligence engine,” Donovan says. “But before we would do quality BI work, we needed to clean up the data we were using.” He describes why the cleansing step was so important. “A large part of the value we provide is in matching transaction-level data to specific demographic information about a customer. We can take, for instance, a series of credit card cash advance transactions and match that information to a particular customer who is already in the database. As the transaction history builds up over time, our clients get a more complete picture of that customer’s behavior.”
“Now imagine,” he continues, “that the customer gets a new credit card. They are still the same person, so we don’t want to create an entirely new record just because they’re using a different card. In the address data cleansing step, we look through our database to see if the address on the new card is already present. Ideally, we find a match and tie the new transactions to that already-existing customer record.” For DiamondStream’s clients, a customer record that accounts for address changes is much more valuable than mistakenly considering each additional address to be a different customer.
DiamondStream originally used Melissa Data to help perform the address cleansing. “It was a fairly basic match process,” Donovan says. “All we did was take the two lists of information and apply a few simple rules to deduplicate the data sets.” They would then feed the output from Melissa Data directly into Birst for analytic processing.
Simple address cleansing using results from Melissa Data was a starting point, but DiamondStream began to look for a more comprehensive matching solution to further refine their products. “The quality of our data directly translates to our profitability as a company,” Donovan says, “so we knew this wasn’t an ideal situation. Our custom-built solution wouldn’t easily scale beyond address cleansing and very basic matching. We needed more sophisticated matching, better data enrichment, and deeper cleansing abilities, but expanding our own platform to handle this was going to involve a lot of cost and time.”
A True MDM Capability with Semarchy
When DiamondStream started to investigate how they could rebuild their solution architecture to improve flexibility and effectiveness. “A number of things came together,” Donovan says. “Birst had just released a version of their software that ran on Amazon Web Services (AWS), so we had this new option to move off Birst’s private server. Running Birst in the cloud opened the door for us to build some powerful new capabilities in a cost-effective way.”
A few months later, Donovan and his team attended a business intelligence conference hosted by The Data Warehouse Institute. They began to learn about Master Data Management (MDM) and quickly saw the value a true MDM capability could provide. “We met the folks from Semarchy,” Donovan says, “and started learning about what they had to offer.”
“We thought we had been trying to solve a data cleansing problem,” Donovan continues, “but Semarchy helped us realize we were actually dealing with data management issues. We saw they had a really cool product, and we could tell right away that it would dramatically improve on what we were doing.”
“We’re a relatively small company,” Donovan says, “and we were able to move quickly as we learned more about Semarchy. We didn’t have to go through a long, drawn-out RFP process since we don’t have a lot of internal bureaucracy. We also looked at purchasing MDM software from Informatica or licensing additional APIs from Melissa Data, but we found that compared with Semarchy, the other solutions we looked at were really expensive. And they couldn’t even do everything that Semarchy could.”
In the end, DiamondStream chose Semarchy. “Semarchy was the best solution and the best fit,” Donovan explains. “With Semarchy we felt like we were working with a company that was really on the cutting edge of MDM, and the technology was a great value tradeoff for us. They also fit very well into the new cloud deployment we were moving toward.”
After choosing Semarchy, DiamondStream worked closely with the Semarchy team to implement their new solution. “We jumped right in,” Donovan says. “Once we made our decision, it was a very fast time-to-value. We built and deployed in about a month.”
New Capabilities & Improved Results
As DiamondStream deployed their new capabilities, Donovan says they gained critical insights into their existing database. “We were able to do matching and data cleansing in a much more sophisticated way than on our old platform. Semarchy was giving us such clean data that we were able to address problems we didn’t even know we had before.”
“Semarchy’s engine also gives us traceability into exactly what happens when the matching rules get applied to our database,” Donovan says. “This is a big improvement over our old solution, where if we suspected there was something wrong with the matching, we didn’t have the traceability to see where things may have gone bad. Semarchy is very user-friendly, and this means you can troubleshoot problems in a whole new way. You can look at your edge cases and see how they are getting handled. You can refine the rules because you have visibility into how the logic was applied.”
In addition to the more sophisticated matching capabilities, Donovan points out some other key benefits of their new solution.
Additional Data Set Handling
“Semarchy allows us to bring new information into our database accurately,” Donovan says. “We now regularly integrate the National Change of Address (NCOA) database along with proprietary data sets the casinos may have from their own payment systems or loyalty programs. With Semarchy we decide how that integration should work, we choose which information should take priority in the case of conflicts, and we can feed the results over time back into improvement. When we get an odd result we can trace how the match was made and correct it.”
Improved Regulatory Compliance
Many of DiamondStream’s clients are subject to laws that govern how casinos are allowed to market to their patrons. “Semarchy helps us ensure that we are accurately integrating the marketing opt-out lists,” Donovan says, “and this helps our clients comply with the appropriate state regulations to avoid costly fines.”
DiamondStream worked with Semarchy to design their new infrastructure so that they only turn on the MDM capability when it is needed. “We can turn it on, load the data, carry it through the pipeline, and then shut down Semarchy when we’re done.” Donovan says his team can perform this data cleansing step daily, weekly, monthly, or on-demand, and this allows them to keep the operational costs very low.
Agility & Flexibility
The agility and flexibility of the new platform allows DiamondStream to adapt and grow their business. After implementing the new system, this agility was tested when one of their main information sources changed its data export format. “It was not a huge difference,” Donovan says, “and we were able to quickly modify the system to handle the change. But making the same type of update on a home-grown solution would have been very difficult. Because we’re using Semarchy we were able to take the situation in stride.”
DiamondStream is very pleased with the results they have seen with Semarchy. “The revenue from our patron data is up 10-15%, and that is directly related to how much cleaner our data is because of Semarchy.” Donovan says the platform’s sophistication has reduced the amount of time needed to identify and fix problems that arise during the data cleansing process.
“Semarchy has also allowed us to improve outcomes for our clients,” Donovan says. “For instance, we helped clients who send physical mail reduce the costs of their advertising campaigns. If you do the right matching on your data, you’ll end up with fewer instances where you send multiple pieces of mail to the same person. And if you’ve correctly cleansed and updated addresses from the NCOA database, you’ll end up with less returned items. So for our clients who may be doing a $100,000 direct mail campaign, we save them real money when we eliminate those incorrectly mailed items.”
“Semarchy has improved the quality of our product,” Donovan says, “and this has given us the opportunity to provide a better outcome for our clients. Better data quality gives those clients more confidence in DiamondStream, which in turn allows us to grow our business.”