Matrix literally syncs the entire data/metadata history to all other servers where someone pops in; chat is meant to have an ephemeral aspect to it. The whole network is de facto centralized on Matrix.org or the servers they host for others which means one org has access to almost everything—like the issue with Signal.
What’s scary to me is how expensive it is to run this eventual consistency model, which should not be a protocol requirement for this style of communication. It sucks so much RAM, so much storage, so wasteful—which causes medium-sized servers to shutdown on maintenance costs alone which causes more users to leave for the Matrix.org. These are not the characteristics of a revolutionary protocol—revolutionary is users & collectives to reasonably be self-hosting this stuff for their privacy & autonomy.