Technology

Waiter, I'll take today's "unsurprising but alarming" development.

Indeed, GNOME has been experiencing issues since a last November; as a temporary solution they had rate-limited non-logged in users from seeing merge requests and commits, which obviously also caused issues for real human guests.

The solution the eventually settled to was switching to Anubis. This is a page that presents a challenge to the browser, which then has to spend time doing some math and presenting the solution back to the server. If it's right, you get access to the website.

According to the developer, this project is "a bit of a nuclear response, but AI scraper bots scraping so aggressively have forced my hand. I hate that I have to do this, but this is what we get for the modern Internet because bots don't conform to standards like robots.txt, even when they claim to".

• Source 1.
• Source 2.
• Source 3.

TechCrunch has a new owner, again. Yahoo has sold the tech news site to the private equity firm Regent for an undisclosed sum, according to an announcement on Friday.

Regent is the same company that snapped up Foundry, the firm behind outlets like PCWorld, Macworld, and TechAdvisor on Thursday. Founded in 2005, TechCrunchhas experienced many shakeups in ownership after AOL acquired the site in 2010.

By now, it should be pretty clear that this is no coincidence. AI scrapers are getting more and more aggressive, and - since FOSS software relies on public collaboration, whereas private companies don't have that requirement - this is putting some extra burden on Open Source communities.

So let's try to get more details – going back to Drew's blogpost. According to Drew, LLM crawlers don't respect robots.txt requirements and include expensive endpoints like git blame, every page of every git log, and every commit in your repository. They do so using random User-Agents from tens of thousands of IP addresses, each one making no more than one HTTP request, trying to blend in with user traffic.

Due to this, it's hard to come off with a good set of mitigations. Drew says that several high-priority tasks have been delayed for weeks or months due to these interruptions, users have been occasionally affected (because it's hard to distinguish bots and humans), and - of course - this causes occasional outages of SourceHut.

Drew here does not distinguish between which AI companies are more or less respectful of robots.txt files, or more accurate in their user agent reporting; we'll be able to look more into that later.

Finally, Drew points out that this is not some isolated issue. He says,

All of my sysadmin friends are dealing with the same problems, [and] every time I sit down for beers or dinner to socialize with sysadmin friends it's not long before we're complaining about the bots. [...] The desperation in these conversations is palpable.

On one hand I think people shouldn't use Discord anyway, but also, we need to figure out how to make the services we use sustainable.

Source.

This article is part of a bigger newsletter.

Archive: https://archive.is/2025.03.20-181313/https://www.bloomberg.com/news/newsletters/2025-03-20/porn-on-spotify-is-infiltrating-the-streaming-service-s-top-podcast-charts

These programs don’t contain video but are sexually explicit with show or episode titles often marked by the creator as “NSFW,” or “not safe for work.” The creators sometimes instruct their fans to rate the content poorly on purpose so that Spotify doesn’t detect it. They also sometimes ask listeners not to report the shows if they don’t like what they’re hearing or seeing. When I reached out, Spotify also removed these programs for violating the platform’s terms of use.

Sexually explicit material has persisted on Spotify for years, but the issue resurfaced in December when a Reddit user noticed the service’s algorithm recommending porn. Some users on the videos I spotted this week also commented. Why, they wondered, were they being served this content when searching for music?

Archive: https://archive.is/2025.03.20-132521/https://www.ft.com/content/3fd3a717-2fbf-42ef-bb08-5baecdeb1985

Nvidia will spend hundreds of billions of dollars on chips and other electronics manufactured in the US over the next four years, its chief executive has said, as the company tilts its supply chain back from Asia in the face of Donald Trump’s tariff threats.

The huge spending projection from the world’s most valuable semiconductor group follows multibillion-dollar US investment plans announced by other technology companies including Apple, as the impact of Trump’s “America First” trade policies ripples through the global economy.

“Overall, we will procure, over the course of the next four years, probably half a trillion dollars worth of electronics in total,” Jensen Huang, Nvidia’s chief executive and co-founder, told the Financial Times. “And I think we can easily see ourselves manufacturing several hundred billion of it here in the US.”

Archive: https://archive.is/2025.03.20-102147/https://www.bloomberg.com/news/features/2025-03-20/are-ai-monopolies-here-to-stay-nvidia-and-the-future-of-ai-chips

Every time you type a question into ChatGPT, you are, probably without knowing it, making several monopolies richer. 

Actually, it’s no different if you use one of ChatGPT’s many competitors. Nearly all of them use chips from Nvidia Corp., which sells around 92% of the particular components — called artificial intelligence accelerators — that make chatbots function. Nvidia relies on a trio of partners to produce its semiconductors: South Korea’s SK Hynix Inc., Taiwan Semiconductor Manufacturing Co. and ASML Holding NV of the Netherlands. Each supplier has a market position almost as fortified as Nvidia’s, or even more so. 

In many industries, that kind of dominance might have antitrust watchdogs threatening a breakup. In technology, it’s long been accepted that important innovations can lead to companies dominating their markets and then staying on top for years by exploiting the laws of scale. It happened with mainframe and personal computers, web browsers, search engines, social networks and mobile software.

White House Press Secretary Karoline Leavitt stated the system was implemented "to enhance Wi-Fi connectivity on the complex," citing poor cellular service in some areas and overloaded Wi-Fi networks. ... instead the outlet writes that the White House is having its Starlink service piped from a government data center miles from the compound.