Issue
- Run a spark job and save RDD checkpoint to S3.
- Spark job failed intermittently with below error:
org.apache.spark.SparkException: Checkpoint RDD has a different number of partitions from original RDD. Original RDD [ID: xxx, num of partitions: 6]; Checkpoint RDD [ID: xxx, num of partitions: 5].
继续阅读“Spark RDD checkpoint on S3 exits with exception intermittently”