The growing problem of AI scams

ArsTechnica has a follow-up to a Wall Street Journal piece about polymarket’s false advertising. Essentially, the old saw of “on the Internet, nobody knows you’re a dog” has become a business model for a new kind of snake oil.

“In its push to draw users to its unregulated platform, Polymarket has flooded social media with videos like [George] Makihara’s, which appear genuine at first glance,” the article said. “In reality, Polymarket built near-perfect copies of its website, then instructed creators to make simulated trades on those dummy sites and hide that they were being paid by Polymarket.”

Makihara, a college student, posted a video in January “that showed him winning $100,000 on a wager that President Trump would publicly say the word ‘McDonald’s’ that month.” But trade data showed that no one on Polymarket won such a bet in January, according to the Journal. This was one of 145 bets that Makihara appeared to place on Polymarket between January and May, but all of those bets were fake, the article said.

The social media influencers who did “advertising” for polymarket faked winning bets which could not possibly have been won in reality. These prediction markets are essentially scams built around insider knowledge. But, they are getting away with these scams because there is no accountability for their false advertising. Even Facebook now wants in on this ‘free money’ business. Unfortunately, everyday folk are providing the corporate profits.

This prediction market scam is, in general, a sign of the growing influence of generative AI in the economy.

There is a great deal of angst in the legal profession about the use of generative AI and LLM (language learning models) for performing legal analysis. Professional standards, however, remain in place as this example and this other example demonstrate.

Note: Whether current legal standards will be applied to generative AI products and actors remains an open question.

I worry that the scams being pushed by prediction markets are only the start of an approaching tidal wave. There are larger and more pervasive problems concerning the use of AI apparently coming our way: namely how AI can enable and empower fraud and crime.

• Identity theft/falsification is likely going to be a major source of legal work (both in preventing and in dealing with the consequences). See Sextortion is just the start, and Deepfake scammer walks off with $25 million in first-of-its-kind AI heist.
• Ransomware attacks are only going to become more proficient and more common.
• As the models become more powerful and pervasive, hacking of computers and computer services will only become more common and prevalent. A spiraling escalation in vulnerabilities and security measures will follow.
• Datasets available from breaches are growing ever larger, with the rewards for successful hacking ever greater from a world wide network of bad actors. See Massive breach spills credentials for thousands of sensitive networks, My SSN was exposed in a breach at Columbia—a school I have no connection with, One Million Passports Leaked Online, and ShinyHunters Leaks Madison Square Garden Surveillance Records. Reliance on any one vendor for cloud computing/AI will likely be impossible to defend both literally from hacking and from any reasonable understanding of the dangers growing from AI hacking models let alone poorly engineered software in the first place. Without taking actions to hide or confuse your own identity, using your real/actual credentials to identify yourself will become riskier and riskier. See this example of the Tea women’s dating app — where women’s confidential, identifying info was spilled out onto the Internet because of atrocious app security.
• Use of agentic AI by govt agencies for the sake of efficiency will likely be hidden from the public and allow for more ID theft by those attuned to AI (by hacking these agencies) and administrative sludge for real people who will struggle with the AI models when trying to gain access to govt programs. Techtonic Justice has a few examples.
• Corporate and govt use of AI will also be problematic because of the loss of civil liberties. For example: Papa Johns Surveillance-Based Advertising, The Realities of AI Video Surveillance, Cybersecurity Mission Creep, Flock Cameras Can Surveil Cars Without License Plates, Flock and Ring Are Champions of the Privatized Surveillance State, and World Cup Propels Surveillance To New Heights.

It is on these issues where legal reform and action is desperately needed. Regulation and oversight cannot be ignored. So, the fiduciary framework in law that already exists needs to be brought to the fore and applied to the companies and govt agencies currently pushing generative AI models. See also Protecting Privacy in an AI Era (with comments that detail massive security and privacy holes that currently exist when “controlling” personal data rather than holding companies and government accountable as fiduciaries, though problems on that front are noted as well).

In the meantime, there are additional concerns that everyone within the legal procession should be tracking and aware of:

• Pro se use of AI in drafting documents (biases will prevail without any legal judgment to check or limit those biases). AI will be needed then to identify and review AI (similar to professors needing AI to review use of AI in students’ papers). Or, in-person argument and presentation will replace written documents. See Suspecting AI cheating, Ivy League prof ordered an in-person final; scores fell 50%.
• Models are trained on the quantity of available data and so will be biased towards the most common topics and issues (example: every dispute processed through an AI-model will have a due process component because due process text is so common on the Internet and in legal databases).
• Biases used to create the models are inherent and unknown, tied to company trade secrets.
• The expense of LLM is only going to get worse: the use and spread of LLM models has largely been subsidized by investors who hope for monopoly-level profits later on despite the inherent inefficiency of this technology. So, current users of LLM are not paying the actual costs of tokens, energy, and hardware needed for their LLM queries (AI processing in outer space is probably not that cost effective). Jumping on the AI bandwagon is likely going to lead to spiraling costs as the subsidies for AI use decline and eventually disappear. Those locked in to AI will either have to pay those increased costs or reverse their switch to AI.