Likewise, we all have our techniques for accomplishing this! This talk describes the Write-Audit-Publish pattern that has worked tremendously well for data at massive scale, enabled through Apache Iceberg, the open table format.
Sam-Redai-The-Write-Audit-Publish-Pattern-via-Apache-Iceberg