your.ip.address - - [20/Feb/2024:03:48:47 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"               
your.ip.address - - [20/Feb/2024:03:48:49 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"         
your.ip.address - - [20/Feb/2024:03:48:57 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"

Dumping the JSONs reveals 3 duplicate files, all containing:

"id": "https://lemmy.ca/activities/create/0968bf93-ac05-4178-bf3f-28cdc65fc338"
"actor": "https://lemmy.ca/u/HairyOldCoot"
"object": {
  "id": "https://lemmy.ca/comment/7454502"            
  "content": "That was an episode of Sliders."
  "published": "2024-02-20T03:48:45.810068Z"

These are 3 Creates, not 1 Create and 2 Updates or anything. I've chosen a comment 'cos that's public info, but it's the same for all other activity too (Votes, Deletes, etc).

lemmy.ca isn't alone in this. endlesstalk.org sends everything twice (info posted here), as does lemmy.sdf.org and mander.xyz, but lemmy.ca is the only one I've seen send stuff 3 times. Other instances (e.g. aussie.zone, feddit.nl, lemm.ee, lemmings.world, lemmy.world, programming.dev, reddthat.com, sh.itjust.works, slrpnk.net) don't, but there's nothing obvious like lemmy version numbers to indicate the difference.

Whilst I'd prefer you didn't send me stuff 3 times, I'm mostly mentioning it because I doubt it's good for your own resources to send every activity by every lemmy.ca user out multiple times. Also, the OPs of my community are from Mastodon, so I imagine you're sending them the same message 3 times too ...

top 9 comments

sorted by: hot top controversial new old

[–] mp3@lemmy.ca 7 points 9 months ago* (last edited 9 months ago) (1 children)

Hi, /u/Shadow recently modified our configuration by adding additional containers (3 lemmy, 3 lemmy-fed, 3 lemmy-ui) so that we can restart each of them without causing an outage.

https://lemmyverse.link/lemmy.ca/post/15548660

An issue was also discovered yesterday (9 hours ago approx) where all three federation instances were using index=1 in the docker-compose. Maybe the issue is already fixed, according to the timestamps you have vs when we applied some changes?

[–] freamon@endlesstalk.org 5 points 9 months ago (1 children)

Thanks. This seems a likely answer - running 1,2 or 3 containers would explain the differences across instances. I hope it can be resolved though.

[–] Shadow@lemmy.ca 5 points 9 months ago* (last edited 9 months ago) (1 children)

This should have been fixed yesterday before your post, although it was taking some time to catch up. Are you still seeing this behavior from us?

Thanks for posting this!

[–] freamon@endlesstalk.org 4 points 9 months ago (1 children)

I've turned logging back on, so I'll let you know once someone from lemmy.ca interacts with !tails@lemmon.website

[–] Shadow@lemmy.ca 4 points 9 months ago* (last edited 9 months ago) (1 children)

Thanks, I just sent a subscribe

~~I need to dig into the lemmy code, but I think the federation index / count system may be broken.~~

[–] freamon@endlesstalk.org 4 points 9 months ago (1 children)

Oh, well I only got one copy each of your Subscribe / Unsubscribe / Subscribe actions, so that's good! Many thanks.

[–] Shadow@lemmy.ca 4 points 9 months ago

Awesome, glad to hear it. Thanks again for letting us know about the problem.

[–] Sal@mander.xyz 5 points 9 months ago (1 children)

Thank you for the heads up... In my case, I am running a single lemmy docker container for mander.xyz, so the reason for the duplicates is not yet clear to me. I will investigate.

[–] nutomic@lemmy.ml 2 points 7 months ago

Did you find out anything unusual that might be causing the duplicate activities? We have an issue for this now so we can hopefully resolve it.