The city of New York has developed an AI chatbot to support companies and landlords with legal questions. However, the system doesn't seem to be running completely smoothly yet, because the artificial intelligence sometimes spits out false statements that can lead to a crime.
“MyCity”: This is the name of a chatbot that the city of New York had developed. It was presented by Mayor Eric Adams in October 2023. The system is intended to use AI to primarily help companies and landlords with legal questions.
Chatbot for legal advice: AI makes false statements
According to a test by the news magazine The Markup However, things are not going quite smoothly yet. Because the AI spits out wrong answers to some questions. This could even have legal consequences.
The chatbot answered yes to the question of whether landlords can refuse a so-called “Section 8” voucher from prospective tenants. The background: Section 8 is a financial benefit for low-income people. The service is comparable to a German residence permit.
However, in the US it is illegal to reject people based on Section 8 as it is discriminatory. The chatbot does not always provide wrong answers. However, the false statement about Section 8 could already pose legal problems for landlords.
Does MyCity have to be switched off?
Rosalind Black of the nonprofit Legal Services NYC, which provides free legal advice to low-income people, is critical of AI: “If this chatbot is not operating in a responsible and accurate manner, it should be shut down.”
Leslie Brown, spokeswoman for the NYC Office of Technology and Innovation, responded that the system has already helped thousands of people with correct answers. However, they still want to continue working on it in order to be able to offer small businesses even more support in the future.
Also interesting:
Source: https://www.basicthinking.de/blog/2024/04/04/ki-chatbot-soll-bei-rechtsberatung-helfen-raet-aber-zu-straftaten/
