this post was submitted on 05 Jun 2023
156 points (100.0% liked)

Lemmy

496 readers
1 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 4 years ago
MODERATORS
nutomic@lemmy.ml
156
How much money does Lemmy.ml need to temporarily boost their servers? (lemmy.ml)
submitted 1 year ago by OsrsNeedsF2P@lemmy.ml to c/lemmy@lemmy.ml
112 comments fedilink hide all child comments
 

With forewarning about a huge influx of users, you know Lemmy.ml will go down. Even if people go to https://join-lemmy.org/instances and disperse among the great instances there, the servers will go down.

Ruqqus had this issue too. Every time there was a mass exodus from Reddit, Ruqqus would go down, and hardly reap the rewards.

Even if it's not sustainable, just for one month, I'd like to see Lemmy.ml drastically boost their server power. If we can raise money as a community, what kind of server could we get for 100$? 500$? 1,000$?

you are viewing a single comment's thread
view the rest of the comments
[–] poVoq@slrpnk.net 22 points 1 year ago (4 children)

There are some SQL database optimisations being discussed right now and apparently the picture resizing on upload can be quite CPU heavy.

[–] itsmikeyd@lemmy.ml 41 points 1 year ago (2 children)

SQL dev here. Happy to help if you can point me in the direction of said conversation. My expertise is more in ETL processes for building DWs and migrating systems, but maybe I can help?

[–] poVoq@slrpnk.net 24 points 1 year ago

this seems to be the relevant issue: https://github.com/LemmyNet/lemmy/issues/2877

[–] veroxii@lemmy.world 20 points 1 year ago

I've been helping on the SQL github issue. And I think the biggest performance boost would be to separate the application and postgresql onto different servers. Maybe even use a hosted postgresql temporarily, so you can scale the db at the press of a button. The app itself appears to be negligible in terms of requirements (except the picture resizing - which can also be offloaded).

But running a dedicated db on a dedicated server - as close to the bare metal as possible give by far the best performance. And increase it for more connections. Our production database at my data analytics startup runs a postgresql instance on an i9 server with 16 cores, 128GB RAM, and a fast SSD. We have 50 connections set up, and the run pgbouncer to allow up to 500 connections to share those 50. And it seamlessly runs heavy reporting and dashboards for more than 500 business customers with billions of rows of data. And costs us less than US$200pm at https://www.tailormadeservers.com/.

[–] MDKAOD@lemmy.ml 7 points 1 year ago* (last edited 1 year ago) (1 children)

apparently the picture resizing on upload can be quite CPU heavy

This suggestion probably won't help with hosted VPS, but lib nvJPEG pushes crazy theoretical numbers for image resizing.

Maybe this could be worth investigating?

[–] poVoq@slrpnk.net 7 points 1 year ago* (last edited 1 year ago)

Probably not, but it does mention a more general CUDA based solution that might be interesting to add to Pictrs. I could for example move my Pictrs instance onto a server that does have an older Nvidia GPU to accelerate stuff (to use for Libretranslate and some other less demanding ML stuff).

Edit: Ok looks like the resizing is anyways only supported on Pictrs 0.4.x which most Lemmy instances are not using yet. However this seems to use regular ImageMagick in the background, so chances are quite high that it can be made to work with OpenCL: https://imagemagick.org/script/opencl.php

load more comments (1 replies)