Sberbank has launched the GigaChat neural network in test mode, which generates text and images.

The main feature of the bot is that it works with the Russian language. GigaChat can generate answers to any questions, and the developers note that it is best at creating creative texts.

We talk about how the neural network copes with tasks and how it performs against the main competitor, ChatGPT from OpenAI.

What you need to know about GigaChat

GigaChat is a multimodal model announced by Sber at the end of April 2023. It can answer questions, engage in dialogue, write code, come up with presentation plans, compose poetry, write school essays, and perform other text tasks. The neural network responds both in Russian and English languages.

The Sber website says that the strong point of GigaChat is creativity. It’s most interesting to use neural networks to write advertising text, emails to colleagues, and recipes. At the same time, when answering questions that require facts, it can make mistakes and hallucinate. Developers call these responses “creative.”

The text part of GigaChat works on language models ruGPT-3 and FRED-TP, based on the architecture of researchers from Google. The first one was released in 2021, and the second one in 2023. According to the developers, GigaChat is trained on 18 billion parameters.

GigaChat is also capable of generating images based on a textual description using neural networks ruCLIP and Kandinsky 2.1. The service primarily creates images based on short requests.

How to access GigaChat

“GigaChat is currently in beta testing stage and access is limited. News about invitation distributions are periodically published in a closed Telegram channel. Since the announcement of the invitation to test the bot, around 40,000 users have received it. The authors promised to give access to the neural network to all subscribers by May 28th.”

I received a link for registration on May 18, when it was sent out to subscribers of the Telegram bot @GigaChat_Test_bot. If you received the link, you will be able to use GigaChat from any browser in your personal space on the developers.sber.ru website. To get there, registration through Sber ID or Google mail is required. There are no smartphone apps for GigaChat.

The interface of GigaChat is similar to ChatGPT – it’s a text chat with the ability to create multiple dialogs. During the conversation, the neural network can use information from previous requests to make clarifications and adjust responses.

How to use GigaChat

We tested the neural network in different scenarios.

Simple search queries. Text neural networks are often used instead of search engines to get a quick answer to a simple question, rather than clicking on different links. GigaChat doesn’t always handle this well.

A neural network can well explain basic concepts and notions that are not tied to current events. For example, to explain some physical phenomenon. But GigaChat is inconsistent – for the same query, it can generate either a huge text of several paragraphs or a short phrase in one sentence.

Moreover, with GigaChat, it is necessary to fact-check. But this is true for any language neural network, including ChatGPT. At this stage of development, they occasionally hallucinate – confidently providing completely false information.

We asked GigaChat more narrowly specialized questions: “What is Skyrim”, “Who is Hideo Kojima” and “How old is Elon Musk”.

In all its responses, the neural network made a mistake and began to hallucinate, i.e. to invent information on the fly. The free version of ChatGPT answered similar questions correctly.

Apparently, the neural network has no access to up-to-date information – much like ChatGPT, whose database is limited to 2021. But we don’t know what date GigaChat is limited to. Although it answered some questions about events in 2022 correctly and without distortion.

If you try to ask what day it is today, GigaChat comes up with a new date every time, and it doesn’t correspond to the days of the week.

Code. On the main page of GigaChat there is an example of how a neural network writes code in Python. We asked for a simpler code – an HTML page with a blue color and the inscription “Hello” in the middle and the current date.

The bot generated code without any color, did not center the text, and overall made many gross errors. Free ChatGPT wrote error-free code on the same request.

Image Generation. The advantage of GigaChat, which ChatGPT does not have, is the ability to generate images within the chatbot. In this case, the Kandinsky 2.1 neural network is used, which was also developed by Sber.

The pictures come out good, and they can be edited on the go with more specific requests. Additionally, Kandinsky understands Russian queries better than other neural networks. Its closest competitor, Midjourney, will have to use English.

School problems. It won’t be possible to do homework with the help of GigaChat yet: the neural network poorly conducts calculations and comes to incorrect answers. It incorrectly takes into account the logic of actions and often answers not to the question that was asked.

Moreover, during our tests, we tried to fix the neural network and indicate to it that it should avoid the answer. But it still insisted on its own. Free ChatGPT solved similar tasks correctly.

City walk. We asked to create a route for a walk around Krasnoyarsk: it turned out realistic but rather short, if you ignore a fictional establishment on the embankment. The route can be completed in a few hours.

But with the clarifications, GigaChat did not cope at all: she suggested to eat seven times in different places, although she was asked to add three establishments and expand the walk. In response to the same request, the free ChatGPT created a longer route, but made up the names of the restaurants and two locations.

Script. The developers say that the neural network copes better with creativity than with facts. We asked them to come up with a script for an episode of “SpongeBob SquarePants,” but it seems like GigaChat didn’t take into account that the characters live in an underwater world.

In the end, an absurd script was created that did not correspond to the mood of the show or the characters’ personalities. ChatGPT generated a plot for a similar request, taking into account the lore of the show and even dividing the episode into three acts.

Poems. We asked GigaChat to come up with a poem about an absurd concept in the styles of various Russian poets. The neural network managed only with Mayakovsky, but not on the first try. At first, it simply copied his real poem in full. On the second attempt, it succeeded when the degree of absurdity was increased.

At the request to come up with a poem in the spirit of Pushkin, she came up with her poem “To ***”. And when asked to recreate Brodsky’s style, for some reason she copied a poem by Yesenin. ChatGPT came up with all the poems from scratch for free, but they didn’t always correspond to the poets’ styles, often lacking rhyme.

Recipes. We asked GigaChat how to cook carbonara. She added pancetta to the list of ingredients, but did not mention it in the recipe itself. When asked to clarify this point, the neural network said that pancetta can be added to the finished dish, although it is raw bacon that is always fried. Even after several attempts to correct the recipe, pancetta still did not appear in it.

In addition, GigaChat recommended using both cream and sauce made of eggs, parmesan cheese, and pasta water at the same time. In all known carbonara recipes, either the first or the second option is recommended. The advice to beat the eggs in a blender is also dubious: whipped egg foam will not result in a creamy sauce. At the very least, according to GigaChat’s version, the carbonara would turn out tasteless. ChatGPT offered a classic recipe that involves frying pancetta and does not require the addition of cream.

Role-playing. GigaChat succeeded in portraying the image of a Russian gangster from the 90s, but could not depict the speech mannerisms of the celebrity and pop culture character.

Even after clarifications and additional descriptions from Elon Musk, the neural network was unable to enter the image and generated stylistically neutral text. The free ChatGPT also has difficulty with roles of famous characters.

Conclusions

✅ Quality work with the Russian language. The neural network was trained on Russian-language materials, so the answers look more natural than those of ChatGPT. The results of the latter are often similar to automatic translation from English.

✅ There is “memory”. GigaChat remembers conversation details and can build responses based on information that the user has already provided. Although the neural network remembers the answers, it reacts poorly to corrections. If one doubts the words of GigaChat, the neural network either refuses to correct the answers or changes them to other hallucinations.

✅ Image generation is available. Requests can be made in Russian, and the resulting images are of good quality. There are no similar free services.

✅ Easy to use in Russia. No need to register with a foreign number, search for a foreign payment card to pay for a subscription, or use a VPN.

⛔️ There is no data on current information. It is unknown on which years database the neural network was trained. When GigaChat tries to answer a question about recent events, it mixes real facts with hallucinations.

⛔️ Unpredictable response format. It can write a short message in two sentences or a huge text in five paragraphs. Moreover, the neural network reacts differently to identical requests.

⛔️ The creativity is quite mediocre. The neural network cannot always generate creative texts from scratch or portray a popular character. Apparently, the dataset did not include many popular styles.

Text generation feels outdated. In virtually all tasks, the free ChatGPT, which runs on the GPT-3.5 version, surpasses GigaChat. The neural network’s responses are more reminiscent of GPT-3 generation results.