The Data Layer Part 2: Data Management

Introducing Part 2 of our deep dive, Data Management technologies enabling the sustainability transition

Mark Tomasovic
February 28, 2024

The Data Layer: The Role of Digital Infrastructure in Climate

Part 2: Data Management

New climate assets are inherently digital. Manufactured in the modern age, wind turbines, solar panels, and other assets are fitted with digital sensors and systems that generate millions of datapoints every minute.  The digitization of these assets has resulted in new opportunities alongside new challenges: we can derive valuable insights from the millions of data points collected on each asset, but the sheer volume of data can pose challenges when it comes to managing and interpreting it. This is where software tools can play a role.

In this section of the Digital Infrastructure deep dive, we cover Data Management Technologies – technologies that aggregate, structure, and transport data. These software platforms automate the transformation and movement of data so that the data is useable to build, operate, and maintain climate assets at scale.

Defining Data Management Platforms

From the field office to the back office, software solutions can streamline data aggregation and processing for the data coming from climate assets:

- In the field: Edge computing solutions at the equipment itself enable data processing and autonomous asset control with ultra-low latency, even under low bandwidth conditions in geographically remote areas.

- In the field office: New 5G and cellular connectivity solutions transport high volumes of data, while SCADA systems enable the storage, visualization, and manipulation of the time-series data produced by the asset.

- In the back office: Robotic Process Automation (RPA) technologies enable efficient document processing across finance, accounting, procurement, and other corporate functions that contribute to the overall project cost.

With recent advances in edge computing, connectivity, and machine learning, today’s entrepreneurs are building digital solutions that cut across each of these areas, forming the digital infrastructure needed to remove embedded soft costs and operate climate assets efficiently.

The Field and Field Office

Let’s start with data management solutions at the field and field office. Equipment in the field is managed with an industrial control system. Originally, these industrial control systems were built with hardwired, on-site digital infrastructure. For example, legacy sensors in the field were physically wired to a hardware controller (called a PLC) that is supervised by a person, or operator, in the field office. Operators could visualize data at the PLC and make control decisions at their workstation. The data is then stored on a server where engineers at the HQ can use the data to derive insights on asset performance.

However, today’s digital infrastructure looks different. Today’s best-in-class control systems utilize a combination of edge software and cloud computing to run operations. Instead of performing all control through hardwiring to the field office, edge platforms offer control at the machine itself. These platforms can also transmit data to the cloud where it is cleansed and processed for machine learning applications.

In general, PLCs are still used for critical control tasks that require wired connections. Refineries, chemical plants, and gas-fired power plants will continue to use PLCs at centralized facilities where operations are manual and control intervention is frequent.

However, distributed climate assets are perfect for edge + cloud control. For example, wind turbines and solar farms are safe and simple assets that can operate autonomously and controlled from a remote-control center hundreds of miles away.  These assets are also perfect for optimization via machine learning applications hosted in the cloud.

Still, both forms of control are not mutually exclusive, and edge software can even complement traditional PLCs by providing advanced machine learning on top of PLC control.

The Back-Office

In the back-office, RPA enables efficient document processing across finance, accounting, procurement, and other functions to reduce overall project soft costs. Highly regulated, risk-averse industries like energy development require detailed engineering design with significant amounts of documentation, permitting, and supply chain redundancies. Recent advances in AI have enabled RPA solutions to structure information from multiple vendors without needing to train a new model for each document type.

While RPA solutions are horizontal software with limitless use cases, examples include expediting corporate-level documentation requirements like permitting or construction milestone tracking.  


The shift from a centralized, hydrocarbon-based economy to a decentralized, low-carbon economy requires new digital infrastructure to efficiently orchestrate the flow of data coming from the new climate assets. From the field office to the back office, new data management technologies enable the efficient aggregation, transport, and processing of millions of datapoints generated every minute. In the attached report, we dive deeper into other data management technologies and highlight today’s tailwinds driving entrepreneurs to build the digital infrastructure for the sustainability transition. Think there is an area we missed or a technology you’d like to explore deeper? Reach out!

Up next: Part 3: Data Analytics