Chris Mellor | Blocks & Files founder and Editor-in-chief

[Abstract]] in recent years, whether online or offline, the data flood has spread to every corner of the enterprise. Now, AI the rise " enable users to pass AI chat robot or agent ( AI Agent) to access these scattered data from a unified perspective. So, how do we collaborate with data and AI the wave of development enables enterprises not only to enjoy the convenience of data, but also to embrace AI to build a seamless connection between life, work and leisure AI data Space?

1.    Moving towards hybrid computing and cloud transformation

A few years ago, many enterprises began to adopt public cloud computing to run data and applications in the distributed mode of cloud service providers. IT infrastructure. This mode brings about local IT different business operation and management logic. The vendor implements the flow of data and applications between the public cloud and the local through a common virtual machine environment and standard storage protocols. -- this is the so-called hybrid computing. Subsequently, enterprises began to adopt the subscription business model provided by cloud service providers, and gradually abandoned the permanent procurement and permanent software licensing models.

With the expansion of enterprise business and data centers, as well as public clouds IT with the wide use of resources, the scale of enterprise data assets (that is, the infrastructure needed to manage full data) also continues to rise. Various database records such as customers, products, internal processes, sales, marketing, and operations have sprung up. The number of files has soared from tens of thousands to hundreds of thousands, millions, tens of millions, hundreds of millions, and even exceeded billions.

2.    Silicon demagnetization: from HDD to SSD evolution

early files were mainly stored in mechanical hard disks ( HDD), as the requirements for data access speed continue to improve, SSD ( SSD) on the stage. SSD the price is higher than that of mechanical hard disks, and mechanical hard disks are more expensive than low-speed tapes used for archiving. Therefore, the storage system naturally forms a hierarchical system: high-performance storage uses high costs. SSD, followed by performance and cost balancing HDD, and finally more economical tape storage. But when the number of files expands to hundreds of millions or even billions, relying on IT it is completely unrealistic for people to manually migrate files between different levels.

3.    Automated Data Management: File Lifecycle software

In order to solve the migration problem of massive files between different levels of media, File Lifecycle Management software came into being. Automatically transfers cold data from SSD migrate HDD, and then transfer to the tape. Users do not need to know where the files are actually stored. The management software maintains a unified index and extracts files from any storage layer as needed.

Through a centralized control panel, the system can uniformly manage and distribute data everywhere, accurately locate the data center, office, or shared cloud area where files are located, and the storage level. Just like a library with multiple branches, all branches share a central directory. In addition, data management facilities can also realize on-demand data flow and timely access.

4.    The concept of unified data space

data management software can coordinate the storage and scheduling of data, for example, the public library system can transfer books, microfilms or periodicals from different branches to the reading room. Now, storage, data request, and delivery are digitized.

Even if we fly from Singapore to London, we can still access the data we want, and the data processed on the plane can be updated synchronously after landing. Today, we live in a ubiquitous virtual data space, where data becomes within reach whenever and wherever.

Such a data space is initially connected to a wired computer terminal and a server in the data center. With the popularization of wired Internet, it gradually extends to offices and families. Later on, mobile phones and WiFi the appearance of has completely changed the connection mode. We get rid of the shackles of cables and can access data anytime and anywhere. Smart watches, smart glasses and other devices also maintain real-time connection with the digital world through mobile phones and laptops.

5.    Transition between storage capacity and technology

as the data volume continues to rise, storage devices can only continue " dosage ". Tape drives used to take the lead in the field of large capacity. One tape cartridge can be saved 15TB to 30TB compress data. However, its reading method is relatively primitive. To find specific data, you must read it from scratch, which affects the access speed. Mechanical hard disk is much faster, it can be directly " airborne " at any position, the tape is pushed off at a high speed, and the capacity is also synchronized. Now you can save more than one disk 32TB the original data, also do not need to rely on compression to support the scene.

However, mechanical hard disks were soon overshadowed by solid state drives. SSD the storage unit is directly connected by the circuit, and the access speed drops immediately without waiting for the disk to switch below the read/write head. HDD several streets. It has already appeared on the market 61TB SSD, 128TB SSD. Built-in NAND the particle capacity has also increased all the way, and a single particle can reach 1 Tb, compared with a few years ago 1 Mb A thousand times.

This means that a cabinet SSD capacity can exceed 50 PB. If it is replaced by a traditional hard disk, to achieve the same capacity, probably need 4700 block disk, almost Occupy 11 cabinets. Visible SSD it can not only greatly save data center space, but also significantly reduce power consumption and cooling costs.

6.    AI innovation: data requirements in the era of big models

the breakthrough of big language model makes AI usher in a new stage, from simple questions and answers to Patent Application Summary, hospital X light and CT scanning analysis, computer code generation, and even image and video creation, AI Agent and chat robots can understand human needs in natural language and give responses. Although these models are essentially still based on statistics to predict " next Step " however, the complexity and depth of its response are far beyond imagination. The larger the training dataset, the better the model performance. When accessing the internal data of the enterprise, they can even deal with first-line interactive scenarios such as sales clue screening and basic customer service consultation. And " autonomous agent AI( Agentic AI) " let AI by going up one flight of stairs, chat robots can realize multi-round dialogue and multi-step task collaboration.

7.    AI data Pipeline: AI Agent prepare Data

AI Agent A large amount of data needs to be quickly fed to those who train them. GPU. If you want them to answer questions online in real time, you have to rely on these data for reasoning. Most of the required raw data is usually stored in the form of files or objects. First, the associated parts are selected, then sensitive information is filtered, and finally converted into vectors. The chatbot answered, behind which is actually a search in the vector library. Therefore, a AI data Pipeline to filter, filter, and convert data, and then provide vectorized data AI Agent process. Nowadays, almost every database, data warehouse and Lake warehouse manufacturer is building their own pipelines.

8.    Huawei AI roles in ecology

For Huawei and its customers, now is the perfect time. The company's smart components can be handled AI workflow, whose servers can be stored in OceanStor data training in storage arrays AI Agent at the same time, it is equipped with pipelines for data processing and storage systems and data Lake software that can supply data for reasoning in real time.

Huawei's network equipment is responsible for server, storage and terminal ( PC, mobile phones, tablets) AI data. NAND the synchronization of particle capacity has been greatly improved. Compared with a few years ago, the amount of data that can be stored is not the same, which also makes the end side AI application implementation becomes feasible. In fact, like Perplexity and Grok etc. AI chat robots have launched mobile phones App version, the interaction between human beings and them is about to enter the fast lane of natural voice dialogue.

9.    AI future: ubiquitous access and smart devices

from smart components to servers, networks and storage, from PC, laptops, tablets to smart phones, smart wear -- all products of Huawei have joined in the same-frequency resonance AI feast. With industry-leading advantages and prospective perspectives, Huawei accurately captures “AI data Space " technology and Application Trends in. Deep ploughing AI on the Road, an open and continuous intelligent ecological map is being built at a more robust pace.

* this article is reprinted from 《 Transform magazine 2025 year 4 month

* this article is included in the user special issue of "words and numbers" 3 period