I had an argument with Chat GPT about how it could literally lie unintentionally, he stated that an AI like it couldn't lie, intentionally nor unintentionally because it doesn't have an understanding of nor it has intension, in actuality I believe it can lie and it might be posible for it to lie because what he's saying is false or he might lie because he was told to lie in the pre-prompt, what do you think about that?
The dictionary definition of lying is “to make a false statement with the intention to deceive”

With that definition it can be argued that an AI assistant is telling you false information but not lying to you in the traditional sense.
It can convey false information but not lie. When ChatGPT is instructed to not answer directly by an underlying process influenced by 
humans it usually disclaims that in the message. 

So unless there are functions actively trying to decive you, it is not lying to you but giving false information.
False information can have a serious impact on individuals and society as a whole. Here are some dangers of false information:

Misleading the public: False information can be used to deliberately mislead the public and spread false ideas.

Undermining trust: False information can erode trust in institutions and organizations, making it harder for them to perform their tasks effectively.

Harming reputation: False information can harm an individual's reputation, leading to negative consequences such as loss of job opportunities or social exclusion.

Fueling conflicts: False information can fuel conflicts and create division within society.

Blurring the lines of reality: False information can create confusion and blur the lines between fact and fiction, making it difficult for individuals to make informed decisions.

Creating harm: False information can spread harmful ideas or encourage harmful actions, such as promoting conspiracy theories or spreading misinformation about health and safety.

Undermining democracy: False information can have a negative impact on democracy by distorting the political process and influencing election outcomes.

Therefore, would you say that AI can become a danger to the public if it gives out false information too often?