Is there a way to query AWS Athena with PySpark from Jupyter Notebook?
19:08 15 Jan 2020

I am able to query SQL database without any problem except for Athena AWS. There seems to not be any connection string that works for it. I tried to follow spark Athena connector but it requires AWS Glue. I want to be able to query Athena AWS from Jupyter Notebook with a JDBC connector that I downloaded from Amazon AWS.

Here is an example of PyAthena that works well with Pandas but not with PySpark. https://pypi.org/project/PyAthena/

python amazon-web-services apache-spark jupyter-notebook