In Apache Spark, how does spark-submit.sh work with different modes and different cluster managers? Specifically:
In local deployment mode,
- does spark-submit.sh skip calling any cluster manager?
- Is it correct that there is no need to install a cluster manager on the local machine?
In client or cluster deployment mode,
Does spark-submit.sh work with different cluster managers (Spark standalone, YARN, Mesos, Kubernetes)? Do different cluster managers have different interfaces, and spark-submit.sh has to invoke them in different ways?
Does spark-submit.sh appear to programmers the same interface except
--master? option--masterof spark-submit.sh is used to specify a cluster manager.