Data Engineering

Hi all,

I was recently reading about the Unified Star Schema and the Puppini Bridge. I’m curious whether anyone here has experience with it and what their thoughts are.

TIA

I plan to run a few tests to determine if Kafka is suitable for a certain usecase I have in mind.

My idea is to run a local cluster of Kafka servers (either VMs or containers), produce/consume a series of messages, observe a bunch of metrics (Prometheus & Grafana) and custom business logic outcomes.

What are some good tools to record and visualise the internals of Kafka cluster?

I'm looking for things like consumer lag, topic replication, possibly tracing messages, ...

Originally posted on https://mastodon.social/@bahmanm/110662538718523380

Hey there community!

Does anyone have any resources they could share relating to Data Vault 2.0, specifically the joining of SAL and PIT tables? The two main books on the architecture are very sparse on this area, which I would have thought would be a fairly key component for any mid-to-large organisation.

What needs to be added for 2023

Fellow data engineers, looking forward to your contribution/participation to the communiy. If you want to help in managing the community, get in touch to join the team