Issue
– Run a spark job and save RDD checkpoint to S3.
– Spark job failed intermittently with below error:
[cc lang=”text”]
org.apache.spark.SparkException: Checkpoint RDD has a different number of partitions from original RDD. Original RDD [ID: xxx, num of partitions: 6]; Checkpoint RDD [ID: xxx, num of partitions: 5].
[/cc]
（更多…）