When the AI Open GPT-3 model made its debut in May of 2020, its performance was widely regarded as the literal state of the technique. Capable of generating indiscipient text of artisanal prose, GPT-3 define a new standard in deep learning. But oh what a difference a year does. The researchers of the Beijing Artificial Intelligence Academy announced Tuesday the publication of their own generating learning model, Wu Dao, a mammoth has apparently able to do all that GPT-3 can do, etc.
First, Wu Dao is amazing huge. It was formed on 1.75 billion parameters (essentially the self-selected models of the model), which is a complete outfit ten times larger than the 175 billion GPT-3 were trained on 150 billion larger parameters than Google switches transformers.
In order to form a model on these many parameters and to do it quickly – WU DAO 2.0 has arrived only three months after the release of version 1.0 in March – Baai researchers have first developed an Open Source learning system similar At Google’s mix of experts, nicknamed Fastmoe. This system, which can be used on PyTorch, made it possible to form the model to form both on clusters of conventional supercomputers and GPUs. This has given more FastMoe flexibility than the Google system since FastMoe does not require exclusive equipment like Google TPUs and can be run on off-stage hardware clusters, notwithstanding.
With all that computer power comes a lot of capabilities. Unlike the deepest learning models that perform a single task – copy writing, generate deep fake, recognize faces, win at Go – Wu Dao is multimodal, similar in the theory of Facebook or from Facebook The recently released mother of Google. Baai researchers have demonstrated the capabilities of WU DAO to perform natural language treatment, text generation, image recognition, and image generation at the annual laboratory conference Tuesday. The model can not only write tests, poems and couples in traditional Chinese, it can also generate an ALT text based on a static image and generate almost photorealistic images based on natural language descriptions. Wu Dao has also shown its ability to power virtual idols (with a little help from Microsoft-Spinoff Xiace) and predict 3D protein structures like Alphafold.
“The way to artificial general intelligence is big models and a great computer,” said Baai President, President of Baai, at the Conference Tuesday. “What we are building is a power plant for the future of AI, with Mega Data, Mega Computing Computing Models, we can transform data to power the IA applications of the future.”
