Table of Contents
Workflow Overview
Programmatically download Insights Ground Displacement sample data (Kings County) via AWS Data Exchange with Python.
Requirements
Subscriptions
- Amazon Web Services (AWS)
- California InSAR Displacement – Kings County Sample
Libraries and Scripts
- Python 3.x
Subscribing to CATALYST Insights free sample data on ADX
- Register for an AWS Account if you do not already have one.
- Please note, a credit card is required to sign up. As you will note upon retrieving the dataset, however, is that it costs $0.
- Sign into your AWS account.
- In the main search bar, type insar.
- Under Marketplace, find the result titled California InSAR Displacement – King County Sample.
- Click Continue to subscribe button in the top right of your AWS console.
- An AWS Data Exchange window opens. Scroll to the bottom and click Subscribe.
Note: Your subscription may take a few minutes to set up. You will be notified when your subscription is ready.
Reviewing and accessing your entitled data in browser (optional)
- Click on View Subscription.
- Scroll down to the Entitled data sets section.
- Click on California InSAR Deformation – King County Sample.
- In the Products tree-list, select the latest revision.
- Click on the checkbox of the Asset you wish to download:
- e.g. Kings_County_Sample_20201105_To_20210523.csv.
- Select the Export action you wish:
- e.g. Download selected assets, or Export selected assets to S3.
Accessing your data programmatically with Python
- Download requirements.txt and download-entitled-assets.py from GitHub.
- To download the files, you must go to the parent folder by clicking Code in the top-left corner. Alternatively, you can simply copy the text of the files into a text editor and save them with the file names listed above.
- Open the download-entitled-assets.py file in a text editor.
- On line 11, change region_name to ‘us-east-2’.
dx = boto3.client('dataexchange', region_name='us-east-2')
- Open a command console (Miniconda3 was used for this demo).
- Type pip install -r requirements.txt
- Set AWS Access Key and Secret (learn how to get your AWS Access Key ID and Secret).
- set AWS_ACCESS_KEY_ID=
- set AWS_SECRET_ACCESS_KEY=
- Run the python script you downloaded from GitHub, download-entiled-assets.py.
#install requirements.txt local file
pip install -r requirements.txt
#Set AWS acces key ID and Secret (export command for unix)
set AWS_ACCESS_KEY_ID=<your-access-key-here>
set AWS_SECRET_ACCESS_KEY=<your-secret-here>
#Run the download-entitled-assets.py script you downloaded from github
download-entitled-assets.py
- Your download will now be accessible in your root folder.
- The data is downloaded as a .CSV file in a new folder named California-InSAR-Deformation-Kings-County-Sample.
- The data columns are structured as follows: Geopoint, Capture Time, Movement, and Velocity.
