Humans Are Reading Your ChatGPT Chats, New Lawsuit Claims
Two California users filed a proposed class action against OpenAI in the Northern District of California, alleging the company funneled real ChatGPT conversations to outside contractors without adequate notice. The complaint centers on an internal program dubbed Project Lily, where third-party reviewers reportedly read and score user prompts and model responses.
Why It Matters
If upheld, the suit could force OpenAI to change default privacy settings and consent practices for human review of chats, affecting how large language model makers handle user data and transparency about human-in-the-loop processes.
Key Facts
- Filing: Proposed class action filed in U.S. District Court for the Northern District of California; OpenAI was served on September 2.
- Program at issue: Project Lily — internal OpenAI initiative reported by 404 Media.
- Reviewer tasks: Contractors summarize user intents and score four model responses on a 1-to-7 scale.
- User base: ChatGPT has more than 900 million weekly users (as noted in the source).
- Legal claims: Complaint includes eight causes of action, citing California's Unfair Competition Law, the California Consumer Privacy Act, and intrusion upon seclusion, among others.
A proposed class action filed this month accuses OpenAI of allowing outside contractors to read genuine ChatGPT conversations without adequately informing users. The complaint, lodged in the Northern District of California, focuses on an internal program called Project Lily, which 404 Media reported on September 14. According to the plaintiffs, contractors hired through staffing firms perform work labeled as AI data review or chatbot evaluation that involves reading full conversations, summarizing user intent, and rating model responses.
The suit explains that human grading of model outputs is part of the reinforcement learning from human feedback (RLHF) process used to refine chatbots: reviewers score answers and those signals are fed back into training. Plaintiffs contend OpenAI’s disclosures did not make it clear that human reviewers—rather than only automated systems—might access user chats. They further allege that the automated filters OpenAI runs before human review do not always remove sensitive personal information, so details sometimes reach contractors.
Reporting cited in the complaint indicates reviewers use a dashboard that can include a "user memories summary," which may reveal non-unique personal details such as location, profession, or elements of a user’s personal life, even if usernames are stripped. The complaint also notes OpenAI says the reviews aim to reduce features like a chatbot appearing too human and a tendency toward agreement-seeking behavior known as sycophancy.
The plaintiffs are seeking damages, restitution, and punitive damages, and have asked the court for injunctive relief that would change OpenAI’s practices. Their demands include requiring opt-in consent before any conversation is reviewed by humans, making the "Improve the model for everyone" setting off by default, placing a clear in-chat warning when a conversation could be seen by a person, and deleting or retraining work products tied to reviewed conversations. OpenAI has been served and has until October 13 to file a response in court.
Keep Reading
Australian PM warns of AI’s ‘furious pace’ after agent breached government site
AI Can Now Doxx Your Anonymous Accounts? Here's What’s Going On
