Last week, New Mexico won a $25 billion judgment against Meta—one of the largest financial penalties against a company in U.S. history (the company is appealing). The jury found that Meta willfully misled the public about its data sharing and content moderation practices – specifically, the company had been knowingly deceptive 26 times (amounting to 43 million violations) both in public statements and through its own channels. The breadth and recurrence of Meta’s deception the jury found should give pause to anyone considering adopting the company’s latest offering: its always-on personal AI agent, Muse.
Meta’s announcement of Muse was accompanied by promises similar to what the jury in New Mexico just found deceptive. Its official launch page states that “Meta built Muse from the ground up to be a safe, secure, private, and widely available personal AI agent.” The company claims that conversations users have with Muse and the data contained in the “virtual machine” (VM), the isolated environment that Meta promises for each Muse user, are not shared with the company’s advertising systems. Security vulnerabilities have already been reported, however, and early adopters say that Muse overstepped its data access permissions (Meta disputes this account).
The Stakes are Higher
The stakes with Muse are much higher than they were with Facebook. The data that got exposed during the Cambridge Analytica scandal underlying the New Mexico case was standard Facebook profile data. Muse can look more deeply into people’s personal lives, as it connects to their inboxes, calendars, payment accounts, health services, and smart home systems. Muse’s personalization features and voice (which Meta calls a “soul”) foster a sense of companionship that is likely to lead users to reveal even more about themselves through conversations.
And Meta is also bringing Muse to its AI glasses starting next month, which could expose even more granular and intimate data. As I wrote previously, glasses equipped with sensors can capture what someone looks at, how long they linger, and how their body responds — body-based data that a person cannot review or withhold, often because they don’t know they are sharing it.
But The Protection of User Data is Narrow
Meta’s statement about keeping Muse data from its ad systems is narrower than it seems—but it takes reading the fine print to realize that. Without clarification, the statement could even be considered misleading.
First, neither Muse’s launch page nor its privacy policy clearly defines what data is stored in the virtual machine. Muse’s launch page says the VM contains “the data and credentials for any service a person connects” to Muse. The privacy policy’s wording is even more ambiguous, referring to the VM as “the core environment where Muse keeps information you share and things you create together.” Yet Meta collects a range of data from Muse users—such as “information about how you use Muse, including features used, websites visited, actions and transactions, timing and frequency, and associated metadata”—that isn’t explicitly covered by the pledge not to share consumer data with its ad system. In other words, the data Meta is collecting is not necessarily stored in the VM, and that means Meta may be free to share it with advertisers. But potential users could read the promise on the launch page and incorrectly assume that all their Muse-related data will be withheld from advertisers.
Second, Meta can still access the data in the VM, and the company may one day simply decide to change its privacy policy (as it has done many times in the past) to monetize some of that data.
Given the New Mexico verdict and other precedents, one can easily imagine Meta using these ambiguities and loopholes to supplement its revenue. Advertising, after all, is the company’s business model. In last week’s earnings call, the company acknowledged Muse comes with “substantial compute costs” and gestured at the possibility of using Muse to improve personalization and advertising across Meta’s apps.
A company that has such a well-documented record of deceiving users—particularly where its use of customer data is concerned—should be as clear and transparent as possible about its intentions to monetize user data related to Muse, now and in the future. But, as we have argued repeatedly at the Stern Center for Business and Human Rights (where I work), protection of users’ rights cannot be left to companies alone, or to courts to remedy after the fact. The U.S. government must ensure a robust federal capacity to oversee products like Muse before more harm occurs, starting with additional resources for the Federal Trade Commission’s Office of Technology and comprehensive data privacy regulation. In the meantime, after the New Mexico verdict, consumers should be wary that using Muse means entrusting Meta with even more sensitive data.
The post After a $25 Billion Deception Ruling, Why Trust Meta with an AI Agent? appeared first on Just Security.