Data Engineering

4 readers

1 users here now

News and discussion on Data Engineering topics

founded 1 year ago

MODERATORS

submitted 1 year ago by ephemeral404@lemmy.ml to c/dataengineering@lemmy.ml

4 comments fedilink hide all child comments

What needs to be added for 2023

you are viewing a single comment's thread
view the rest of the comments

[–] nydas@lemmy.ml 1 points 1 year ago (1 children)

Isn’t Apache Arrow a table format rather than a batch processing tool?

You could add SQLMesh as an alternative to dbt.

[–] ephemeral404@lemmy.ml 1 points 1 year ago

thanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html