this post was submitted on 20 Jan 2024
116 points (100.0% liked)

Futurology

40 readers
15 users here now

founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] Endorkend@kbin.social 2 points 10 months ago

No need to invent.

That's how originally search engines, including Google, Yahoo and all the other big ones worked.

You didn't get indexed by default.

You either got indexed by being submitted or by being referenced often by one or more well represented sites.

It's only later in the game they started crawling everything.