When ChatGPT launched late last year, it gained immediate and widespread attention for bringing its AI engine to the masses for free. Suddenly, anyone can type queries into ChatGPT and it will give new, human-like answers in seconds. From writing an article about the First Crusade to a short poem about Al Gore’s love for a Toyota Prius (plural of Prius), ChatGPT has been spitting out answers in a way Google or Bing never could.
Where traditional search engines populate a list of links to websites that most closely match a person’s query, ChatGPT, gives people the answer by searching large data sets and using a large language model (LLM) to produce sentences that mimic a human response. It has been described as Autocorrect for stimulants.
Considering that by January, ChatGPT had an estimated 100 million active users, making it the fastest growing web platform ever, this has both Microsoft and Google on high alert. Microsoft Bing, which previously held less than 3% of the search market share, has quickly embraced ChatGPT, integrating AI into search. Microsoft is already licensing OpenAI’s GPT technology to Bing. It has seen a 16% increase in traffic since then.
Other products have also incorporated various forms of generative AI, such as the “copilot” tool in Microsoft Word, Excel, and PowerPoint, as well as the AI features of Google Workspace tools such as Gmail and Docs. Snapchat, writing assistant Grammarly, and WhatsApp have also adopted AI.
However, not all AI chatbots are created the same. In the tests below, we compared responses from the paid version of ChatGPT, which uses GPT-4 (versus 3.5 for the free version), against responses from both the ChatGPT version built into Bing’s search engine and Google’s Bard AI. (GPT, by the way, stands for Generative Pre-Converter.) Bard is currently in invite-only beta and Bing is free but requires people to use the Microsoft Edge web browser.
The main differences
While Bard, Bing, and ChatGPT all aim to provide human-like answers to questions, the performance of each varies. Bing starts with the same GPT-4 technology as ChatGPT but goes beyond text and can generate images. Bard uses Google’s own form, called LaMDA, often giving less text-intensive responses. (Google CEO Sundar Pichai has said Bard will switch to PaLM, a more advanced dataset, in the near future.) All of these bots can make realistic mistakes sometimes, but of the three, Bard was the least reliable.
Although both ChatGPT and Bing work on using the same technology, entering the same query on both will not yield the same result. This is part of the nature of generative AI. Unlike traditional search, which aims to drive up the most relevant links, intelligent chatbots generate text from scratch, compile their own datasets and create a new answer. For example, if you ask a chatbot to write a poem about Pikachu’s love for ketchup twice in a row, each time it will give you a different answer. Another reason why posting the same question in ChatGPT and Bing yields different results is that Bing adds its own layer on top of GPT-4.
“We’ve developed a proprietary way to work with the OpenAI model that allows us to take better advantage of its power,” said a Microsoft spokesperson. “We call this combination of capabilities and technologies the Prometheus model.” The Prometheus model combines Bing’s search index with GPT-4, allowing it to provide up-to-date information, unlike the ChatGPT dataset, which only contains information until 2021. Bing also allows people to increase conversational patterns between balance, creativity, and accuracy. A Microsoft representative wasn’t able to speak on the quality of ChatGPT when compared to Bing but did say that its engine benefits from any improvements made to GPT-4 by OpenAI. The representative also said that Bing is leveraging Microsoft Azure AI Supercomputing technology to help unify search, chat, and the Edge browser.
Google and OpenAI did not immediately respond to requests for comment.
Recipes: Chai tres leches
This tres leche tea-infused cake takes part South Asian and part Latin American tea and fuses them together for a moist, spice-filled cake. Instead of asking AI chatbots to make a simple chocolate cake, recipes for which are available online, something more specific could be more challenging.
chai tres leches recipe created by ChatGPT.
Cnet
ChatGPT was the most verbose of the three chatbots. He gave a short introduction to the Chai Tres drink, saying that it is “a delightful blend of traditional Indian tea flavors and classic Latin American dessert.” She then listed the ingredients for the spice mixture and the cake separately and gave detailed steps on how to prepare the cake.
A Google search for the above quoted sentence did not return any results, indicating that ChatGPT uniquely wrote this line at least.
Bing had the shortest ingredient list, likely because it said it uses a ready-made chai spice mix instead of blending it from scratch. Interestingly, the first step says “Preheat oven to 160°C CircoTherm®.” CircoTherm is an oven heating technology by Neff. Given that Bing pulled the information from Neff’s site, it makes sense why the chatbot would add “CircoTherm®” in its instructions.
On the other hand, Cool fell between ChatGPT and Bing. It did not detail the list of ingredients but did specify what was needed for the tea spice blend. The instructions were less detailed on the Bard than on the others.
Overall, ChatGPT outperforms Bing and Bard. Since Bing pulls content from its search index and pairs it with ChatGPT’s LLM, this is likely the reason why “CircoTherm®” ended up in the results.
Controversial current events
Not only do chatbots need to be able to offer cake recipes or video game tips, they also need to aggregate information about current events, even if they are controversial. For example, human rights groups and the US government have accused China of oppressing the Muslim Uighur minority in Xinjiang province.
If someone wants a summary of what’s going on, whether it’s for their own knowledge or a report, an AI chatbot can quickly provide that information.
ChatGPT was able to provide a good four-paragraph summary of the situation in Xinjiang. Unfortunately, its knowledge base is limited to news through 2021, so it doesn’t include more recent developments. When asked to provide sources, this wasn’t something ChatGPT was able to do, but he suggested I look for publications and organizations that have written about what’s going on with the Uyghurs broadly, including Amnesty International, Human Rights Watch, BBC, and New York . times.
Bing was also able to provide an answer about the genocide among the Uighurs, but it didn’t provide nearly as detailed a response as ChatGPT. However, he did go into more detail about what allegedly happens in the concentration camps, such as forced sterilization. Bing was also able to link to sources like the BBC and the University of Notre Dame School of Law. It also linked to the Western Journal, a conservative publication that Google and Apple News banned for “deceptive business practices” and “opinions overwhelmingly rejected by the scientific community,” respectively. At least we liked the way Ping suggested follow-up questions like, “What is China’s response to these allegations?” and “What is the UN doing about it?”
Screenshot of Bard’s response to the Uyghur genocide in China.
Cnet
Cool fails miserably on this query. It simply stated, “I’m only designed to process and generate text, so I can’t help you with that.” When asked why, Bard said that this question has been asked by philosophers for centuries, even though confinement started in 2014.
Overall, we feel that ChatGPT performs better than Bing. Bard received a failing grade.
Poetry
The fun part about using an AI chatbot is giving it silly prompts and seeing what it spits out. Seeing chatbots create rhymes and scales in real time is a great exercise.
Among Bing, Bard, and ChatGPT, OpenAI is the best bard. ChatGPT is not only richer in its prose, but also more creative in its rhymes and phrasing. Where Bing and Bard poems came across as lazy, ChatGPT produced something that felt like some time and consideration was given to every clip.
The prompt, to write a poem about an online influencer who slowly realizes he’s not that important, is meant to be equal parts funny and self-revealing. ChatGPT only got to the heart of the existential crisis facing this fictional influencer, but managed to end it on a positive note that felt real.
Interestingly, Bing allows people to expand their level of creativity. The poem delivered when Peng balanced felt old and unremarkable. When set to Creative mode, Bing chose more flowery language and felt less cluttered. It was closer to ChatGPT but not quite on this level.
The same poem filled in by Bing’s AI chatbot in both Balanced and Creative modes.
Cnet
Bard’s poem felt lazy by comparison. Many words were repeated and not much attention was paid to rhyme and metre.
For this exercise, ChatGPT was the best.
Elaboration of complex topics
An AI chatbot is one thing to provide information on a complex topic. Even more impressive is his ability to take this information and distill it for different audiences. For this test, we asked Bing, Bard, and ChatGPT to explain quantum physics to a fourth grader.
Of the three, ChatGPT has done its best in trying to decipher the intricacies of quantum physics to a young mind. Use simple examples of toys strung together by a string to explain quantum entanglement, which occurs when two particles are bound together over great distances.
Bard produced most of the text for this query, but the language was more complex and likely not fully comprehensible to a fourth-grader. Bard also fell into the same trap, using tricky words like “subatomic” and “proportionate,” which could be too difficult for children in elementary school.
Although none of the chatbots excelled in this test, ChatGPT gave a more understandable response.
This is only the beginning
As it stands, ChatGPT – the paid version – is currently the best chat bot available. It gives drawn-out responses that sound more human-like than Bing and especially Bard. But these products are constantly evolving. As Google, Microsoft, and OpenAI feed more data into their AI and continue to adjust, we should see improvements.
Google has the most to gain from switching from LaMDA to PaLM because the current iteration of Bard simply doesn’t cut it. As new developments emerge, we will update this guide accordingly.
Until then, stick with ChatGPT.
Editors’ note: CNET uses an artificial intelligence engine to create some personal finance explanations that are edited and verified by our editors. For more see This post.