this post was submitted on 20 Jan 2024
116 points (100.0% liked)
Futurology
40 readers
15 users here now
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
No need to invent.
That's how originally search engines, including Google, Yahoo and all the other big ones worked.
You didn't get indexed by default.
You either got indexed by being submitted or by being referenced often by one or more well represented sites.
It's only later in the game they started crawling everything.