[Global Network Technology Comprehensive Report]For most people, AlphaGo defeated Li Shishi, the world’s top chess player, and AlphaMaster defeated Ke Jie in Go, which made us truly realize the arrival of the era of artificial intelligence. Subsequently, the market and industry were excited about it, and startups around the development and application of "artificial intelligence" emerged constantly, and traditional enterprises also entered the market one after another, using artificial intelligence technology to open up new market demand. In just a few years, people’s cognition of artificial intelligence has changed from "curiosity" to a normal attitude. It seems that the smoke brought by artificial intelligence has gradually dispersed, and the changes it has brought to society and industry will be forgotten over time.
I don’t know when it started, but we found that the recommendation of shopping websites is more and more in line with our wishes; The effect of taking pictures with mobile phones is getting better and better, and you can get satisfactory selfies without retouching; There seems to be less and less traffic jams on the roads of the Second and Third Ring Roads in Beijing during the morning rush hour. These scenes that we are used to, but secretly "get better", are inseparable from the support of artificial intelligence technology. As the saying goes, "moisten things quietly", just like most universal technologies, artificial intelligence is imperceptibly affecting our lives, and both academia and industry are stepping up research and development and layout of artificial intelligence technologies and applications.
In recent years, artificial intelligence is being favored by the market and society because of its ability to process huge, repetitive and complex data. Due to the universality of artificial intelligence technology and the diversity of realistic scenes, how to design a universal and easy-to-use artificial intelligence development model for different industries has become a hot spot in the field of artificial intelligence research and development. In the past two years, this field has attracted the layout of many internationally renowned technology companies including Microsoft, Google, Facebook, OpenAI and Huawei. In April this year, Tian Qi, the chief scientist in the field of Huawei cloud artificial intelligence, led the team to officially release the "Huawei Cloud Pangu Series AI Model".
According to Tian Qi, Huawei Cloud Pangu series AI big model (hereinafter referred to as Pangu big model) includes NLP (Chinese language) big model, CV (vision) big model, multi-modal big model and scientific computing big model, aiming at establishing a set of universal and easy-to-use artificial intelligence development models to empower more industries and developers and realize the industrial development of artificial intelligence. It has a strong generalization ability. By combining with industry knowledge, it can quickly adapt to different scenarios, and a small number of samples can also achieve high precision. Based on the industrialized AI development model of pre-training and downstream fine-tuning, it accelerates the application of AI industry and enables AI technology to truly empower different industries.
Recently, Tian Qi, the head of Huawei’s cloud Pangu model and the chief scientist in the field of Huawei’s cloud artificial intelligence, shared his deep thoughts on AI industrialization and the application of Pangu model with media reporters.
Launch a new model of industrialized AI development and build an AI ecosystem with partners.
Before joining Huawei, Tian Qi had been deeply involved in the AI field for many years: he graduated from the Department of Electronic Engineering in Tsinghua University in 1992, graduated from Drexel University in the United States in 1996 with a master’s degree, and then went to study at the University of Illinois at Urbana-Champaign in the United States, where he studied under Professor Thomas S. Huang, an internationally renowned expert in computer vision, and received his doctorate. From 2002 to 2019, he served as assistant professor, associate professor and full professor in the Department of Computer Science at the University of Texas at San Antonio.
After joining Huawei in June 2018, Tian Qi became the chief scientist of computational vision in Huawei’s Noah’s Ark Lab. In March last year, he became the chief scientist of Huawei’s cloud artificial intelligence field. Thanks to many years of theoretical and practical research in the fields of computers and AI, Tian Qi has keenly discovered that artificial intelligence has entered a period of rapid development and prosperity driven by industry and capital, and at the same time, he has also seen various shortcomings of AI in large-scale commercialization. Among them, one of the biggest challenges of AI application is the fragmentation, customization and "workshop-style" development mode of AI application. Simply put, a scene needs to develop a model independently, and the whole process needs to be started from scratch and independently tuned. If the model can’t reach the expected goal, it will have to be reinvented, resulting in low development efficiency, and the development cycle often takes one month or even months. "Therefore, AI development urgently needs a new model to complete the transformation from workshop-based development to industrial development," Tian Qi said.
According to Tian Qi, the Huawei Cloud Pangu Model released in April this year is based on the industrialized AI development model of "pre-training+downstream fine-tuning", aiming at accelerating AI’s entry into thousands of industries and realizing inclusive AI. Pangu big model is based on big data pre-training and has strong generalization ability and small sample learning ability. This has achieved a breakthrough in performance and ease of use, that is, a model can be applied to a large number of complex industry scenarios, and a small number of samples can also achieve high accuracy.
Pangu big model is not an experimental model, but now it has landed in many industries and more than 100 scenes, including energy, retail, finance, industry, medical care, environment, logistics and so on. In the case of "Power Intelligent Inspection of State Grid Chongqing Yongchuan Power Supply Company", because most of the areas where Chongqing power grid is laid are hilly and mountainous areas, rivers and valleys, it is necessary to use unmanned aerial vehicles for intelligent inspection of power grid. Among them, the development of traditional intelligent inspection AI model of UAV faces two major challenges. One is how to label massive data efficiently; Second, there are hundreds of kinds of defects, which require dozens of AI recognition models, and the development cost is high. In order to solve the above problems, based on Pangu CV model, Huawei Cloud team used massive unlabeled power data for pre-training, and combined with an efficient development mode of fine-tuning a few labeled samples, put forward a pre-training model for the power industry, which skillfully solved the problem of data labeling. After the application of Pangu model, the efficiency of sample screening is improved by about 30 times, and the screening quality is improved by about 5 times. Taking Yongchuan as an example, 50,000 high-definition pictures are collected every day, and the manual marking time can be saved by 170 people/day. In terms of model universality, combined with the automatic data augmentation carried by Pangu and the optimization strategy of category adaptive loss function, one model can be adapted to hundreds of defects, and one model can replace more than 20 original small models in Yongchuan, greatly reducing the maintenance cost of the model, improving the average accuracy by 18.4% and reducing the development cost of the model by 90%.
Helping the industry’s AI application ability to improve rapidly is the goal.
When talking about the position of Pangu big model technology in the same field, Tian Qi said that although many well-known overseas companies have devoted themselves to this track, Huawei Yunpangu big model still has its own unique advantages.
The first is technological innovation. Pangu NLP model uses Encoder-Decoder architecture for the first time, which takes into account the ability of NLP understanding and generation and leads the performance; The multi-task learning method is adopted to make the large model training more stable; Using Prompt-based, it has stronger ability in small sample learning. Pangu CV model has been extracted on demand for the first time in the industry, and the volume difference of the model extracted in different deployment scenarios can reach three orders of magnitude. Based on the comparative learning of sample similarity, the small sample learning ability is leading in the industry on ImageNet.
Followed by commercial verification. The technology developed by Huawei is producible and commercialized, and the Pangu model also integrates such genes. At the beginning of research and development, targeted development was carried out on scenes such as finance, intelligent customer service and intelligent investment research. "When we designed Pangu Big Model at the beginning, we hoped that it could get a better verification in some occasions, and hoped to drive a goal with commercial value." In Tian Qi’s view, the display of AI technical capabilities needs to be reflected in applications, and better empowering the industry’s AI application capabilities is the embodiment of the final results of Pangu Big Model.
Training AI with limited data is the current environment of AI industrialization.
When talking about his views on AI industrialization, Tian Qi said that the current AI industrialization field is in the early stage of industrial digitalization, and the application scenarios are very fragmented. Many scenarios still need customized development, and it is difficult to have a universal solution. In the face of such a dilemma, Pangu model has set a pre-training and fine-tuning method, so that the model can cover more scenes, and because the model is large enough, it has higher performance and stronger tolerance, that is, stronger generalization ability.
For example, in the financial field, industry experts need to spend a lot of energy every year to analyze corporate financial data and non-financial data, explore the means and motives of corporate financial fraud, identify corporate risks, and help corporate project audit. Due to the scarcity of industry experts, it is difficult to analyze the finances of thousands of companies in detail. In addition, due to the small sample size and extremely uneven distribution of data, traditional machine learning has been unable to accurately analyze the financial data and non-financial data of enterprises.
In order to improve the efficiency of industry analysis and reduce the manual workload, Pangu Model has trained a series of models for data differentiation, and then fused each model efficiently after training one by one. The trained Pangu model can identify 10+ major financial anomalies, 200+ minor abnormal signals, 6 types of fraudulent motives and 120+ fraudulent means.
Mental journey: from academia to business circles
As a typical representative of the transformation from academic circles to business circles, Tian Qi also shared his mental journey and views on AI with reporters. Before coming to Huawei, Tian Qi had taught in a university for 17 years, mainly studying large-scale image retrieval, pedestrian re-recognition, etc., and once achieved fruitful academic results: he became an Associate Editor； of many journals such as IEEE TMM, TCSVT, TNNLS, ACM TOMM, Multimedia Systems Journal, etc. He owns 6 American patents, has published more than 650 articles (including 181 IEEE/ACM journals and 191 CCF A conference articles) in top journals and conferences in computer vision and multimedia, such as IEEE TP AMI, IJCV, TIP, TMM, CVPR, ICCV, ECCV and ACMM, and has been cited by Google for more than 30,700 times, with an H index of 83. Eight papers have won the Best Paper Award or.
Tian Qi won the 2010 Google Facilities Research Award and the 2017 UTSA President’s Outstanding Research Award (one per year). In 2016, he was awarded the top 10 most influential scholars in the multimedia field and was awarded the IEEE Fellow. Later, he was selected as a national leading talent innovation project in 2018. Tian Qi is not only a chair professor or visiting professor at Tsinghua University Center for Neurocognitive Computing, Institute of Computing, Chinese Academy of Sciences, Chinese University of Science and Technology, Zhejiang University, Shanghai Jiaotong University, Xi ‘an Jiaotong University, Nanjing University, xidian university, Dalian University of Technology, Southeast University, etc., but also a professor at the Yangtze River in the Ministry of Education, an overseas evaluation expert of the National Natural Science Foundation of China, and the China Academy of Sciences. At the end of May 2021, Tian Qi was elected as an academician of the International Eurasian Academy of Sciences.
As a top expert in AI academia, coming to the industry has given Tian Qi more opportunities to combine theory with practice and improved his knowledge structure. In his view, although the academia and the industry are different, the "Tao" is interlinked and both are pursuing the development of science and technology, but the academia pays more attention to the pursuit of theoretical research, while the industry pursues practical application. At present, the academic circles advocate the combination of Industry-University-Research, with the purpose of promoting the common development of theory and practice, providing more practical cases for academic development and providing more theoretical guidance for industry.
editorial comment/noteSince the appearance of Alpha Go, the game of Go seems to have jumped out of the circle of simply pursuing victory and defeat, and turned to pursue the real "formal beauty" of Go; In the same way, the existence of artificial intelligence helps people get rid of monotonous and boring work and pursue their own real value; The application of Pangu model will also help enterprises to reduce costs and increase efficiency, so that AI will benefit thousands of industries.