What is Lazy Evaluation?
Lazy Evaluation is an evaluation strategy that delays the evaluation of an expression until its value is needed. In Spark, Lazy Evaluation means that you can apply as many transformations as you want, but spark will execute these processes until an action is call.
Spark’s Catalyst Optimizer
When Spark perform transformations, It will store them in a Directed Acyclic Graph (DAG) and constructed it. Spark’s catalyst optimizer will perform a set of rule-based and cost-based optimizations to determine a logical and then physical plan of execution.
What are the advantages of using Lazy Evaluation?
Spark’s Catalyst optimizer will group operations together, reducing the number of passes on data and improving performance. One more advantage of the catalyst optimizer is that, often, values that won’t be used for the final result, will simply not be computed. To research more check out this blog post