Xie Xiqian | Senior engineer of a provincial city commercial bank
[Abstract]] in order to further improve the backup management of small and medium-sized commercial banks and achieve the goals of full coverage of backup objects, high reliability of backup data and high efficiency of backup and recovery in Xinchuang system, this paper will focus on the difficulties in the construction of Xinchuang data backup, this paper introduces the practical experience of the construction of the domestic backup platform of our bank from the aspects of architecture route selection, architecture design and implementation of the transaction system Xinchuang data backup, so as to provide reference for peers.
With the application of new technologies such as cloud computing and big data, the data volume of related businesses has exploded. In particular, unstructured data is rapidly becoming the main force of data growth, the traditional backup mode based on structured data is difficult to meet new changes and requirements. At the same time, domestic and independent operating systems and database products have been gradually popularized and applied, while traditional backup systems lack support for domestic and independent operating systems and databases. For the backup and management of such data, systematic research and design are needed to make up for the defects of the backup system, thus putting forward higher requirements for data guarantee capability.
I. Analysis on the current situation of data backup construction in the transaction system
in terms of backup media, bank transaction system backup has gone through the development stage from physical tape library to virtual tape library, and then to all-in-one backup machine. Virtual Tape libraries use disks as media to simulate physical tape libraries. Compared with physical libraries, it improves the efficiency of data backup and recovery. The all-in-one backup machine based on professional backup storage integrates disks, centralized backup software servers and corresponding network devices, and enhances backup technologies such as deduplication, further improves the economy and maintainability of backup.
Currently, the backup system of city commercial banks basically implements the centralized management mode. Due to the diversity of some systems and backup objects, a small number of databases, cloud platforms, and big data are not included in centralized backup management. In terms of backup media, mainstream systems still use backup software. Physical tape libraries or disk arrays are combined for backup. Before the domestic backup platform was built, the transaction system backup status of our bank was as follows:
1, usage of existing backup software
Take our bank as an example. Similar to other small and medium-sized commercial banks, in the past ten years, we have mainly adopted foreign mainstream centralized backup software (such TSM, Veritas), built multiple backup domains, local centralized backup of all structured and unstructured data, daily backup peak 15TB(15806GB), average 4TB(4120GB), the total number of recorded files is approximately 12 hundreds of millions. The following is the plan and usage of our backup domain:
2, backup object and scale
Currently, the backup objects of our bank are structured and unstructured data. The backup data is saved 1 zhouzhi 1 years. A large number of backup objects, existing backup jobs 200 more than one, centralized backup at night. Due to the different range of backup objects, the backup data volume varies greatly, with the minimum data volume for a single service. GB level, up TB level.
3, full backup and recovery time of the transaction system
in our business system, the structured data that has been backed up is DB2 most data, backed up DB2 dozens of the largest single databases in TB, the backup time of a single backup job is about several hours per day. The recovery time is unknown because no recovery drill has been performed. Currently, the unstructured data backed up is mainly a large number of small files, from KB to Dozens MB it varies from left to right. The backup efficiency is always low due to the large number of files. The recovery time is unknown because no recovery drill has been performed.
4, backup media and network
currently, our backup system mainly uses tape drives, physical tape libraries, and disk arrays. The network mainly adopts private network ( LAN FREE) and hybrid network ( LAN), it is common to use private network architecture for structured data.
5, refinement of transaction system backup requirements
our main transaction system backup requirements are as follows:
II. Analysis of problems and difficulties in data backup construction of transaction system Xinchuang
1, the backup object is not fully overwritten.
Based on the current survey of centralized storage backup domains, structured data and unstructured data have incomplete backup object coverage, which fails to achieve comprehensive protection and does not support domestic environment backup. Therefore, the biggest difficulty in the construction of transaction system Xinchuang data backup lies in how to realize system-wide and full-type data backup.
2, low backup and recovery efficiency
as the data volume of each business system becomes larger and larger, more and more backup jobs are created, and the growth rate is faster and faster. TB the backup recovery efficiency cannot meet the business recovery timeliness requirements. Therefore, how to improve the backup of large data volumes (such as big data, data warehouse, data mart, and data Lake) and the efficiency of massive unstructured data backup is an urgent problem to be solved in the transaction system Xinchuang data backup construction.
3, difficult to verify backup data recovery
Due to the differences in software platforms of various business systems, different environments are required for recovery drills, resulting in insufficient hardware resources and high input costs. At present, our bank needs to spend a lot of time building the basic environment for data recovery and verification, which seriously affects the recovery efficiency. At present, we have not carried out any backup data recovery drills and verification work. Therefore, during the construction of the transaction system Xinchuang data backup, we need to focus on the problem of fast data recovery, and perform periodic and efficient recovery drills on the backup data to ensure the reliability of the backup copy.
4, lack of statistical analysis and comparison
at present, important databases and some unstructured data are basically incorporated into the backup system, but there is a lack of means to analyze and compare the backup data, it is difficult to analyze and predict the historical data growth trend of backup. Future backup systems are not only required to ensure data security, but also need to provide data service capabilities and business innovation support.
III. Selection of architecture route for data backup in the transaction system
the current Xinchuang backup platform mainly includes pure backup software and software dedicated Backup Hardware two architecture routes, each of which has its own advantages and disadvantages, will be explained in detail below.
1, technical architecture of pure backup software
the technical architecture of pure backup software is the separation and deployment of software and hardware. The backup software is installed on a general-purpose server (either Xinchuang or Xinchuang) to back up and restore regular operating system, file system, and database, the software implements re-deletion, compression, backup and recovery management. The backup data is finally stored in distributed storage, block storage, or external storage composed of local disks on the server. Data that needs to be imported into the database can be accessed through D2D2T or D2T(Backup software is required).
The following figure shows the architecture:
the advantages and disadvantages of this architecture are as follows:
Advantages:
( 1) reduce costs and increase efficiency, reduce the input of physical machine resources, and deploy virtual machines or use old servers;
( 2) the flexibility of scale-out is high. If the physical machine resources are sufficient, only software authorization is required;
( 3) high compatibility, can use hardware resources of different vendors at the same time.
Disadvantages:
( 1) implementation is difficult, and various complex tasks such as physical resources and environment required for implementation need to be arranged;
( 2) archiving, copying, and backup are all forwarded through backup software, which is easy to become a performance bottleneck;
( 3) high maintenance costs are not conducive to maintenance, and high maintenance costs are easily caused between backup software and physical machines.
2, software dedicated Backup Hardware architecture
Software the technical architecture of dedicated backup hardware makes up for the shortage of Xinchuang backup platform from the hardware layer. There are two main implementation methods: one is to use a backup all-in-one machine and deploy the hardware and software together, after the device arrives at the site, it can be used out of the box. Second, the backup software is deployed separately, and the storage layer uses dedicated backup hardware devices, which is equivalent to the separation of computing and storage, realizing semi-integration, some backup stores can also implement unified storage of backup data at the storage layer.
The following figure shows the architecture:
the advantages and disadvantages of this architecture are as follows:
advantages:
( 1) the flexibility of scale-out is higher, which supports both horizontal scaling and vertical scaling. The pure backup software deployment architecture only supports horizontal scaling of servers;
( 2) hardware reliability and density improvement, 2U node 2 for the reliability of more than one server device, the processing between controllers is Active-Active, 1 nodes 2PB the capacity is equivalent to that of the server 4-5 devices;
( 3) unified management of storage pools, flexible configuration of storage media, you can select different performance disk media according to different backup scenarios, such SSD SAS HDD SATA HDD multiple disks for hardware-level re-deletion and compression and disaster recovery management , replica data can be synthesized faster and read/write faster than the software layer;
( 4) storage hardware WORM function anti-blackmail, WORM the feature of writing and reading multiple times at a time cannot be changed after the backup data is stored. It has the function of anti-ransoming;
( 5) based on the dedicated backup hardware base, the backup service is optimized end-to-end;
( 6) can be provided at the same time NAS space and integrated backup capabilities help smooth business migration.
Disadvantages:
( 1) compared with pure backup software deployment, the cost is higher.
IV. Design of data backup architecture for transaction system Xinchuang
the data backup architecture design of transaction system Xinchuang mainly includes data architecture design and business architecture design.
1, data Architecture Design
business data includes online backup data, near-line backup data, offline backup data, and recovery drill data. The following section designs and evaluates backup technologies and backup media options.
In addition to technical comparison, detailed comparison is also made for media selection, as shown in the following table:
to sum up, online backup is suitable for block storage media, near-line backup is suitable for all-in-one backup machine or object storage, offline backup is suitable for cloud storage or Blu-ray disc media.
2, business Architecture Design
the business architecture of the Xinchuang backup platform of the trading system of small and medium-sized commercial banks includes six parts: backup software, backup objects, backup management and control, data services, backup data formats, and backup media.
Backup Software it must have business functions such as data re-deletion, data availability verification, data archiving, backup policy management, backup data retrieval, secure access, and backup service directory management. The backup service catalog can transform business requirements into technical indicators and evaluate whether the backup results meet the expected goals to further guide the backup work.
Backup objects including files (audio and video, emails, logs), databases, virtual machines, time series databases, operating systems and applications, distributed storage files, distributed data databases, configuration management data, structured and unstructured data such as big data.
Backup management control including monitoring, analysis, and reporting functions to monitor the overall running status of the backup system ( CPU, memory), backup process running status, backup job Result status, etc., analyze and predict the backup capacity, analyze whether the backup policy is reasonable, network bandwidth is satisfied, etc., to form a final analysis report.
Data Service it includes data recovery drills, data migration, and testing. It tests the integrity of system data and improves the emergency response level and quick handling capability of O & M personnel.
Backup data format the structure includes blocks, files, and objects. Backup media include CD, tape library, all-in-one machine, SAN storage, distributed storage, and flash memory. As shown in the following figure:
v. Experience in data backup in transaction system
our bank is a small and medium-sized city commercial bank. When selecting the data backup platform of Xinchuang, we not only require the data backup platform to have high scalability and extensive ecological capabilities (especially Xinchuang ecology) to meet the requirements IT to meet the needs of continuous technological innovation and evolution, we also need to protect the existing investment management traditional applications, distributed databases and big data platforms, and finally realize a backup architecture to solve the parallel problem of traditional ecology and new ecology. Based on this, our bank adopts a bottom-up, parallel first and then integrated Unified backup construction path, which is mainly divided into the following two stages:
1, parallel running phase
in the parallel running phase, the original IOE system Maintenance TSM backup: the new domestic transaction system uses the new Xinchuang backup platform for backup; The underlying backup storage pool is managed in a unified manner, data and hardware resources are integrated, and resources are utilized efficiently, at the same time, some space is provided for the traditional backup system, standard NAS interface, to achieve traditional software protection for traditional applications, new backup system protection for emerging applications, parallel management. Our bank has adopted backup software. Professional backup storage architecture, in the parallel running phase, the transaction system Xinchuang backup platform is implemented. The architecture is as follows:
the platform uses domestic storage (Huawei OceanProtect X8000) as the base, mainly Lan-free, permanent incremental backup, composite backup, Mount recovery and other new disaster recovery technologies as basic modules, to shorten the backup time window, improve the efficiency of backup recovery and data verification as the construction goal.
Through the Xinchuang backup platform and the existing TSM parallel Mode can not only effectively back up and restore business data of domestic trading systems, but also meet regulatory requirements. At the same time, during the parallel period, it can also smoothly migrate and convert historical backup data step by step, protect the bank's early investment, and manage the whole life cycle of domestic and non-domestic business data backup. The following figure shows the implementation idea:
2, overall protection phase
the overall protection phase will implement unified system-wide backup platform management. As the software and hardware of the traditional platform are over-guaranteed, the production platform is replaced as a whole and switched to the new platform. The newly-built Xinchuang data backup system gradually takes over the original backup system capabilities and backup space as the old system exits naturally. For data that needs to be stored for a long time, if you need to restore it, you can retain the original backup software equipment for regular testing and restoration of historical data.
3, benefits and summary of transaction system Xinchuang backup platform construction
since the construction and operation of the project, it has covered all domestic business systems that have been launched in our bank, including some transaction business systems and important management systems (mainly domestic database backup). First, the permanent incremental backup technology solves the long-standing problem of large-capacity database backup window. Second, the fast Mount technology solves the problem of slow recovery efficiency; third, through the copy management technology, after data desensitization, multiple test platforms that are the same as the production environment can be quickly restored, greatly improving the test efficiency of the development test center. Achieve the following benefits:
(1) meet backup requirements of domestic applications
Able to implement Galaxy Kirin, a domestic operating system V10 data backup and recovery based on sea light / luopeng hardware platform KVM virtualization Platform backup, traditional Windows, Linux and AIX operating system backup, DB2 and Oracle database Backup, etc.
(2) fast recovery of core databases
on the basis of backup and recovery, Mount recovery backup technology ( CDM) achieve minute-level fast data recovery for important business systems, including transaction systems DB2 and Oracle the database can be quickly mounted and restored in minutes to improve the effective utilization of resources.
(3) unified backup storage base
the storage backend implements unified disk backup acceleration, re-deletion and compression, and improves the available space. It is classified into physical tape libraries, distributed object storage, and other storage methods.
(4) secure from the hardware layer
data security at the hardware level is achieved through unified management of the storage resource pool, mainly including the storage layer WORM tamper-proof, secure snapshots, encrypted storage, backup link security, user data security, and platform architecture security.
Finally, summarize the replacement of data backup in China 5 words: Unified backup base, one pool for dual use, enhanced security, steady evolution, cost reduction and efficiency improvement. Through the coexistence of data in a backup system, new and old applications can be backed up, new and old systems can be replaced and evolved to ensure secure data backup and efficient recovery. We try to reuse replicas by using emerging technologies, to achieve the purpose of cost reduction and efficiency increase.
* this article is included in the user special issue of "words and numbers" 2 period
original link: https://www.oceanclub.org/cn/discuss/info/3775
