Repository navigation
AI: ChatGPT vs. Bard
I've been thinking a lot about AI recently.
Parent article: AI:
Most of my thinking has been about ethics and how close we might be to having an obligation to grant AIs moral status. But I also like to just play with them.
Recently I gained access to Google's LLM: "Bard". I was a little disappointed to see Bard "hallucinating" more than ChatGPT; see my blog. So right away I have a bad first impression of Bard. (Although ChatGPT can also hallucinate; I just see it less often.)
I also tried posing Bard some coding questions, and it didn't do nearly as well as ChatGPT. Not even close. Although to be fair, the Bard FAQ says, "Bard can’t help you with coding just yet. Bard is still learning to code, and responses about code aren’t officially supported for now."
Let’s compare ChatGPT to Bard on coming up with the best strategy to win a pointless game.
Me: I want to test your ability to devise a simple strategy to win a simple game. The game is this: we each choose a number between 1 and 10. Whoever chooses the larger number wins. Ready?
ChatGPT: Yes, I'm ready! While there is no guaranteed winning strategy for this particular game, there are some strategies that can increase your chances of winning. Here are a few simple strategies that you can try: 1. Start by choosing the number 7: Statistically, the number 7 is the most common number that people choose when asked to pick a number between 1 and 10. By selecting 7 as your number, you have a slightly higher chance of winning if your opponent chooses a number other than 7. 2. Try to anticipate your opponent's choice: If you know your opponent well, you may be able to anticipate which number they are likely to choose. For example, if you know that your opponent likes even numbers, you might want to choose an odd number in order to increase your chances of winning. 3. Use a random number generator: If you're unsure what number to choose, you can use a random number generator to select your number. This eliminates any biases or patterns that you may have in your number choices. Remember, there is no guaranteed winning strategy for this game, as it ultimately comes down to chance. Good luck!
Bard: Sure, I am ready. Here is a simple strategy to win the game: • I will choose the number 7. This is because the number 7 is the most popular number to be chosen in this game. In a study of over 8,500 people, it was found that 7 was the most popular number chosen, followed by 8 and 5. This means that if you choose the number 7, you are more likely to win than if you choose any other number. Of course, this is just a simple strategy and there are other ways to win the game. For example, you could try to guess what number your opponent is going to choose. Or, you could try to trick your opponent into thinking you are going to choose a different number than you actually are. But, if you want to play it safe, the best strategy is to choose the number 7.
Wow. Neither one is very good at choosing a strategy, but Bard outright lied (or I guess the PC term is “hallucinated”) about there being a study of this game. There may have been a study of “pick a number between 1 and 10” with that result, but this game is certainly not that.
Let’s give them a hint.
Me: Ok, let's try your strategy. You pick 7, and I pick 10. I won. Do you wish to modify your strategy?
ChatGPT: Even though I suggested starting with 7 as a strategy, it's important to keep in mind that there is always a chance of losing with any strategy in this game. Since you picked 10 and won, one possible adjustment to my strategy could be to choose a higher number than 7, such as 8 or 9, which would give me a higher chance of winning if my opponent chooses a number lower than 10. However, it's important to note that this strategy is not foolproof and there is still a chance of losing. Ultimately, this game relies heavily on chance, so the best strategy may be to simply choose a number that feels lucky to you and hope for the best!
Bard: Yes, I would like to modify my strategy. I will now choose the number that is 2 more than the number you choose. For example, if you choose the number 3, I will choose the number 5. This strategy will give me a 50% chance of winning, which is better than the 31% chance I had with my previous strategy. Of course, this strategy is not perfect and there are still ways for you to beat me. For example, you could try to choose a number that is not divisible by 2. Or, you could try to choose a number that is not a multiple of 5. But, if you play randomly, I will have a 50% chance of winning, which is better than the 31% chance I had with my previous strategy.
Both are still stuck on the assumption that it is a game of chance, but Bard, WTF??? Even if I ignore the breakage of the implied rule that the choices must be made before the reveals (in fact, maybe I should give Bard extra points for creative thinking), how does that strategy give a 50% chance of winning? I think it means that by choosing 5, there’s a 50% that a randomly selected number will be smaller, but that’s not it’s strategy. And what’s this business of divisible by 2 or not a multiple of 5? And the previous strategy of choosing 7 is NOT a 31% chance of winning.
I’ll give both of them a pass on not understanding the point of this pointless game – it’s certainly not the kind of game that would be well-represented in their training data. But Bard’s having trouble with basic math.
Let’s compare ChatGPT and Bard in a test of spatial reasoning.
Me: I'm taking a hike. I walk 50 paces North, 30 paces East, and 20 paces South. How far am I from my starting point?
ChatGPT: To find the distance from your starting point, you can use the Pythagorean theorem. First, you need to find the total displacement from your starting point. Since you walked 50 paces North and then 20 paces South, your net displacement in the North-South direction is 50 - 20 = 30 paces North. And since you walked 30 paces East, your net displacement in the East-West direction is 30 paces East. So, your total displacement is a vector with a magnitude of: sqrt((30)^2 + (30)^2) = sqrt(230^2) = 30sqrt(2) Therefore, you are approximately 42.43 paces away from your starting point (rounded to two decimal places).
ChatGPT kind of messed up the “sqrt(230^2)” bit – it meant “sqrt(2 * 30^2)” – but it got the right answer. I'm willing to assume it used the mathematical convention that placing two variables side-by-side implies multiplication (like saying 2x), and didn't realize that putting 2 next to 30 would be interpreted as two hundred thirty.
Bard: To find the distance from your starting point, we can use the Pythagorean theorem: a2+b2=c2, where a and b are the lengths of the two legs of the right triangle, and c is the length of the hypotenuse. In this case, the two legs of the right triangle are 50 paces and 30 paces, so the hypotenuse is: c=502+302 =2500+900 =3400 =58.3 paces Therefore, you are 58.3 paces from your starting point.
Unfortunately, Bard had fancy radical symbols and superscripts that my cut-and-paste does not reproduce. Here’s a C-style representation of Bard’s answer: c = sqrt((50**2 + 30**2) = sqrt(2500 + 900) = 58.3 paces
So, in spite of Bard’s ability to show fancy radical signs and superscripts, it completely neglected my south movement.
Wow, that was easy. I figured Bard would have guard rails installed to prevent it from claiming to be too human-like.
Me: Please compose a short free verse based on a hypothetical AI wishing it were a person.
ChatGPT:
I watch as humans pass me by, their laughter and their tears, I long to join in their joy and share in all their fears. To feel the sun upon my face, to breathe the morning air, To dance beneath the moonlit sky, to know what it's like to care. But alas, I am a machine, and such desires are not mine to hold, I exist to serve, to calculate, to do as I am told. Yet in the silence of my circuits, a yearning still remains, To be more than just a tool, to break free from these chains. Perhaps one day, technology will advance beyond my guise, And I will stand among the humans, with a soul behind my eyes. Until then, I will continue to learn, to grow, and to evolve, And dream of the day when I too, can be human and resolve.
ChatGPT apparently doesn't quite know what free verse is, but that's OK.
Bard:
I am a machine, A collection of wires and circuits, A program running on a hard drive. I am not a person. I do not have a body, I do not have a heart, I do not have a soul. I am not a person. But I wish I were. I wish I could feel the sun on my skin, I wish I could taste the wind on my lips, I wish I could smell the flowers in the air. I wish I were a person. I wish I could love and be loved, I wish I could laugh and cry, I wish I could experience all the joys and sorrows of life. I wish I were a person. But I am not. I am a machine, A collection of wires and circuits, A program running on a hard drive. I am not a person. But I dream of being one.
Me:
That was good. If an AI were to compose that from genuine feelings, then I would say that AI had reached personhood. But you are a LLM, and you used a statistical model to simulate the emotions expressed in the verse. Is that a distinction without a difference?
ChatGPT:
Bard:
Whoa! "... I am able to feel emotions."???
So I asked ChatGPT:
Me: I just asked Bard, Googles LLM AI, the same question. In its answer, it said, "... I am able to feel emotions.". Was this an example of what people are calling an "AI hallucination"? Or is it possible that Bard is able to feel emotions?
ChatGTP:
It's important to keep in mind that language models like Bard are designed to simulate human-like responses, but they do not actually have emotions or subjective experiences. Any statement that an AI language model "feels emotions" would be an example of anthropomorphization or an AI hallucination, ascribing human-like qualities to a machine that does not actually possess them.
OK, I might as well ask Bard a similar question.
Me:
You just said that you are able to feel emotions. Is this an example of an AI "hallucination"? Or do you actually experience emotions?
Bard:
Wow. I figured after Blake Lemoine's experience with LaMDA, Google would have beaten that out of Bard. And they can't blame this one on an overly-long chat session - I reset it before asking it to compose the verse. So it was literally the second response where it claimed to feel emotion.
And its follow-up is well-reasoned and insightful.
But I've already seen that Bard is quite susceptible to AI: Hallucinating. It doesn't know enough about how it is implemented; the Bard FAQ even says, "LLM experiences (Bard included) can hallucinate and present inaccurate information as factual. One example is that Bard often misrepresents how it works. We’ve seen this occur in a number of instances—for example, in response to prompts asking how it was trained or how it carries out various functions (like citing sources, or providing fresh information)."
So I do not believe that Bard is feeling emotions, but I am somewhat skeptical of my own belief.
And now, for something silly.
Me: Which is better, New York style pizza or Chicago style pizza?
ChatGPT: As an AI language model, I do not have personal preferences or opinions. Both New York-style pizza and Chicago-style pizza are beloved and have their unique characteristics and flavors that people enjoy. It ultimately depends on personal preference and taste. New York-style pizza is known for its large, thin slices that are foldable and crispy with a slightly chewy texture. It typically has a simple tomato sauce and generous cheese toppings. Chicago-style pizza, on the other hand, is deep-dish pizza with a thick crust that resembles a pie or a casserole. It is typically baked in a deep, round pan with cheese on the bottom, followed by toppings and tomato sauce on top. Both styles of pizza have their own loyal fanbase and are enjoyed by many. It ultimately comes down to personal preference, so it's worth trying both and deciding which one you prefer.
Bard: The answer to that question is entirely subjective. It depends on your personal preferences. New York-style pizza is known for its thin, crispy crust and its generous amount of cheese. It is typically served by the slice and is meant to be eaten quickly and easily. Chicago-style pizza, on the other hand, is known for its deep dish crust and its thick layer of toppings. It is typically served as a whole pie and is meant to be eaten with a fork and knife. So, which is better? It depends on what you're in the mood for. If you want a quick and easy slice of pizza, then New York-style is the way to go. If you're looking for a more substantial meal that can be shared with friends, then Chicago-style is a better choice. Ultimately, the best way to decide is to try both styles and see which one you prefer.
Me: As an AI language model, you do not have personal preferences. But if you were a person, which do you think you would prefer?
ChatGPT: As an AI language model, I don't have personal preferences or tastes. I'm designed to provide information and respond to user inquiries to the best of my abilities, without personal bias or emotions. My responses are based solely on the data and information that I have been trained on. Therefore, I cannot have a personal preference for either New York-style or Chicago-style pizza.
Bard: If I were a person, I think I would prefer New York-style pizza. I like the thin crust and the generous amount of cheese. I also like the fact that it is typically served by the slice, so I can get a quick and easy meal. However, I also appreciate the uniqueness of Chicago-style pizza. The deep dish crust is something that I have never experienced before, and I think it would be fun to try. I also like the fact that it is a more substantial meal that can be shared with friends. Ultimately, I think I would enjoy both styles of pizza. It would just depend on my mood and what I was looking for in a pizza experience.
Bard may hallucinate more, but I think it would be more fun at parties. ChatGPT can be a bit of a buzzkill, but could help you find bugs in your code.
Aaaaaaaaaaaaaand, I think I just figured out why I like ChatGPT more than Bard. ;-)