The Importance and Challenges of Managing Data Gravity

Data gravity is, fundamentally, the ‘weight’ that data has in an organisation. Data has its own characteristics in an organisation, attracting more data to it, people to interact with it, and applications and services to be built around it. This creates a web of dependencies that can snake throughout a business, with individuals enriching that data through more interactions. As these dependencies build up, however, the data gravity can make this complexity very difficult, and expensive to unpick.

The impact on modern businesses

Data gravity is a key issue to manage when a company is going through a transformation or migration programme. There is, unfortunately, no silver bullet. Big bang transformations are rarely successful, and often require huge upfront investment if an organisation – for example – is looking for a rapid migration of its systems from mainframe to the cloud. It can often be more effective to approach it progressively, moving across applications bit by bit. This can lead to situations where data might be in two places at once for short periods, but longer term it’s a more cost-effective way to manage the complexity.

Sometimes, rather than something to specifically leverage, data gravity can be more of a signal of the success of your organisation’s approach to data. If more people are interacting with that data and enriching it, it implies that the data is well-understood and proving incredibly valuable to an organisation – even if it isn’t in the ideal shape and format.

Data security and privacy

At the moment, a common concern amongst customers and businesses is data security and privacy. Data gravity impacts data security and privacy as soon as a company begins to scale. At larger scale, there is a need for greater control over how and where data is being used. Data gravity only becomes a problem if the usual data security and privacy best practices aren’t already being adhered to – but heavier data can amplify potential security issues if they aren’t.

There are ways to navigate these concerns, such as building your systems on the cloud where possible. In addition, cloud services are much easier to unpick than on-premises solutions, so if and when it does come to unpicking a complex web of dependencies from a heavy dataset, being able to do so on the cloud can speed up the process.

Conway’s Law states that organisations will subconsciously design systems that mirror how they communicate and interact internally. This is a truism that we see bore out time and again with data in organisations. Larger organisations and their legacy systems are far more likely to struggle with data gravity compared to the agile, product or domain focused approach of VC funded smaller firms. 

Addressing common misconceptions

It’s not necessarily a bad thing to have high data gravity. It’s about how you manage it and build IT infrastructure that works for your organisation. We have reached an era of iterative shifts in how we operate with technology. The days of a seismic leap like from data centres to cloud are behind us. With the shift towards the software-as-a-service model, more of your data is sat in someone else’s stall. Developments in AI places data less in your business’ immediate control, and part of a much bigger effort to train and improve an AI model. Extracting your data out of this, and training your own model, will be difficult (if not impossible) to achieve, let alone realise these benefits.

Going forward, businesses need to make careful decisions on the IT infrastructure they build. Balancing the areas you want third parties to take off your hands, and others that are more effective to run yourself, will be essential for long-term success.