Dealing With a Mess of Data Systems and Products

Hosts: Andy Clark, Tom Gough

Guest: Greg Bounds 

Topic: Dealing With a Mess of Data Systems and Products

Key Points:

1. Introduction:

·      Andy and Tom introduce Greg Bounds, who works as a data architect for a national nonprofit and also consults on data projects.

2. Defining the Problem:

·      The discussion focuses on inheriting a complex data environment, termed a "dog's breakfast," with a mix of applications, systems, and databases.

·      Tom emphasizes the challenges of dealing with multiple cloud providers and on-premises systems.

3. Scoping the problem:

·      Start by talking to various stakeholders to understand the importance of different systems.

·      Auditing and documenting the systems should follow naturally from these conversations.

4. Managing Stakeholder Priorities:

·      Understand the challenge of balancing different departments' priorities and stresses the importance of understanding the entire data flow.

·      Show the benefits of changes through small, manageable projects.

5. Technical Approach:

·      Look for low-hanging fruit and opportunities to consolidate systems for quick wins.

·      Simplifying the data architecture can de-risk the environment and reduce points of failure.

6. Benefits of Consolidation:

·      Understand there may not be immediate cost savings, reducing the number of platforms simplifies governance and management.

·      It also minimizes the need for specialized knowledge across multiple systems.

7. Planning for the Long Term:

·      The importance of data governance and having a data team that is approachable and open to new solutions.

·      Ensuring the data team has the right skills and capacity is crucial to maintaining a streamlined environment.

8. Key Takeaways:

·      Communication with stakeholders and understanding their needs is essential.

·      Focus on data quality and transparency, even if it means creating "snitch reports" to highlight issues.

·      Look for quick wins to build momentum and show the value of changes.

·      Engage the C-suite to drive adoption and support across the organization.

 You can download Power BI Desktop from here.

If you already use Power BI, or are considering it, we strongly recommend you join your local Power BI user group here.

To find out more about our services and the help we can offer, contact us at one of the websites below:
UK and Europe: https://www.clearlycloudy.co.uk/
North America: https://clearlysolutions.net/

Transcript

Andy:

Welcome to the Clearly Podcast. My name is Andy Clark.

 Tom:

And I'm Tom Gough.

 Andy:

And we have no Shailan.

 Tom:

Did we forget to invite him?

 Andy:

No, I just got bored of him. He wasn’t contributing that much. Bless him. But we have a far better replacement in the form of my local friend Greg Bounds. Greg, can I hand over to you to introduce yourself?

 Greg:

Sure. My name is Greg Bounds. As you said, I live here in Houston. I work part-time for a national nonprofit called Brighter Bites as their data architect. I also have my own clients and have worked with you, Andy, on some data consulting projects.

 Andy:

We are actively working on a few projects together at the moment. OK, so the topic of today’s podcast is dealing with multiple systems and applications. I'm going to phrase it as Tom did to me: "What to do when you inherit a dog’s breakfast." This means taking over a data environment that's a complete mix of applications, systems, databases, and other incoherent elements. Tom, have I defined that well? What would you add?

 Tom:

When we start to talk about inheriting a dog’s breakfast, my thinking was, obviously none of our listeners would get themselves into this mess because they listen to our podcast carefully, take all our advice, and their systems are perfect. But sometimes they’re going to move on and inherit a mess where systems have been purchased from all sorts of places. You might have software as a service, infrastructure as a service, and platform as a service spanning across different providers like Microsoft Azure, Google Cloud Platform, and AWS. You might also have stuff on-premises. This topic was touched on in the last two episodes when we talked about choosing a cloud provider and dealing with elements outside your normal wheelhouse. So, I thought it would be valuable to discuss how to deal with stuff from all over the place and try to consolidate it into a system that is easier to manage with a more centralized skill set.

 Andy:

Greg, you’ve been jettisoned into an organization with exactly this issue. Where do you start?

 Greg:

I think some people say you should audit and document what's going on, and I agree that's important. But you really have to start by talking to people—more than one person. What's important to one person might not be important for the organization. In a classic company, for example, the sales team might say everything they do is the most important thing. So, you need to interview and talk to people across the organization. Then, through those conversations, the audit and documentation of the systems come naturally. To me, speaking with as many people as you can to get a good idea of what's going on and what's important is crucial.

 Andy:

How do you deal with the inevitability that everyone will perceive their own data fiefdom as the most important?

 Greg:

That’s a great question. Whoever might be the favorites of the C-suite might say, "Sales is the most important thing." You have to fight for understanding other departments, like accounting. You need to show how the entire process works from data input to reporting. It’s about making them realize it’s a big process, and you need to understand it all. Sometimes opening their eyes to how messy their system is can help get them on board with tackling it as a whole instead of focusing on just their area.

 Andy:

It’s almost like getting someone to realize they have a problem.

 Greg:

Yes, exactly. Hopefully, everyone is aware there’s a problem. But sometimes people are in denial, especially if they don’t want to learn something new. Getting those people on board can be very difficult. For example, the accounting department can be resistant because they’re busy and stressed and don’t want to change their processes or talk about their systems. Finding easy wins, like automating manual processes, can help get people excited and on board.

 Andy:

Tom, you’ve done the initial interviews and understood the problem. Where do you start when you begin rolling up your sleeves?

 Tom:

I think the first starting point is to look for low-hanging fruit. For example, if a company has data on Azure and Google Cloud Platform but also an Amazon S3 bucket, you might start by migrating the data from Amazon to one of the other platforms. This can give you a quick win and reduce the number of platforms you need to manage. You then look at the overall usage and dependencies to decide which platform makes the most sense to consolidate onto.

 Andy:

When moving data from one cloud provider to another, Greg, where do you see the benefits for the client?

 Greg:

It’s mostly about reducing the load on your data team. There might not be a cost benefit from the technology, but it reduces the need for institutional knowledge across multiple platforms. It also simplifies governance and makes it easier to manage accounts and systems.

 Tom:

It also de-risks things because if you have data in two places, you have two potential points of failure. Consolidating reduces complexity and makes it easier to manage and secure your data.

 Greg:

It also standardizes where new data goes, which simplifies the process. Knowing that new projects will be set up on the consolidated platform removes ambiguity and makes management easier.

 Andy:

How do you decide which order to do things in?

 Tom:

I’d start with the low-hanging fruit or look for areas where you can show tangible benefits, like cost savings from consolidating reporting tools. Complexity and risk are also factors. Sometimes the risk of moving data might be too high, so you might leave it for later. Greg, you had some good points about this.

 Greg:

Yeah, starting with small, easy wins allows you to test your plan on a smaller scale. If it works, you can expand it across the company. Communication is also crucial. Keeping everyone informed and showing them the benefits helps get them on board.

 Andy:

How do you plan for the long term to ensure the organization doesn’t end up back in the same mess?

 Greg:

It involves data governance and change management. Leadership needs to understand the value of these processes. Presenting it as a way to save money and protect the company can help get buy-in. Capacity is also important. Sometimes the data team is just too busy, so you might need to hire consultants or additional staff to help.

 Tom:

Having a data team that is open and approachable is crucial. If they’re always saying no, people will find ways around them. Engaging with the business and helping them find the right solutions is key. It’s also important to have the right skill sets and people skills within the data team.

 Andy:

OK, let’s wrap up with our key takeaways. Greg, what would you like to leave with the listeners?

 Greg:

Communication is crucial. Talk to people across the organization and don’t let one person dictate the process. Also, focus on data quality and find the systems most impacted by messy data. Fixing these can provide significant benefits.

 Andy:

Tom?

 Tom:

Look for quick wins and areas where you can show tangible benefits, like cost savings. Simplifying the data architecture and reducing the number of platforms makes it easier to manage and improves data quality.

 Andy:

For me, it’s the snitch report. Transparency in data quality is important, even if it means naming and shaming. It helps improve overall data quality.

 Greg:

I agree. Transparency and communication are key. It helps get everyone on board and ensures long-term success.

 Andy:

Thanks, Greg, for joining us. It’s been a brilliant contribution. And thank you, Tom. We’re back next week. Goodbye.

Tom:

Cheers.