Lambda architecture

Lambda architecture is a data architecture pattern that combines batch and stream processing systems to enable real-time and batch processing of large volumes of data, providing a fault-tolerant and scalable way to process and analyze data in near-real-time. It is based on three main components: the batch layer, the speed layer, and the serving layer. The batch layer is used to process the data in batch, providing accurate results. The speed (or realtime) layer is used to process the data in realtime, trading accuracy for latency. Finally, the serving layer is responsible for responding to queries, by aggregating the results from the batch and speed layers. The lambda architecture can result in increased complexity and maintenance costs due to the need for separate batch and stream processing systems and potential inconsistencies between real-time and batch views of data.