Atlas: How to Deploy and Set Up Atlas 1.5

It's your first time deploying your shiny new Atlas 1.5

This guide will outline the steps to deploy and configure Atlas.

Prerequisites

  • A command-line environment to run CLI commands
  • Ability to securely receive API keys from Ethyca
  • Ability to run the docker commands and launch docker images
  • Access to a test subject’s email for the purposes of requesting data from the Privacy Center, validating Multi-Factor Authentication (MFA), and verifying the final data
  • Administrative access to customer’s Control Panel to approve the test subject’s DSR request

Atlas Resource Requirements

The machine running Atlas should have at least the following specifications:

ResourceMinimum Size
RAM4GiB
Storage1GiB

Virtual Specifications:

ResourceMinimum Size
AWS EC2t2.medium

Deploying Atlas

Atlas is provided by Ethyca as a Docker image. The instructions below contain details on how to deploy the image to your infrastructure including configuration parameters and system requirements.

Prior to deployment, your Customer Success Manager will provide you with the API key designated to allow your Atlas deployment to communicate with Ethyca Services. This will be mapped in the configuration file to the atlas.ethyca.api.key property. Additionally, your Customer Success Manager will provide you with a "Unique Organization Name", which will also be used in the configuration file, under the atlas.organization property.

Step 1. Download Atlas from Docker

Use the following command to pull down the latest image from the public Ethyca Atlas docker repository:

docker pull ethyca/atlas

Step 2. Create a table in a database reachable by Atlas (only if you did not previously have Atlas 1.0+ installed)

In order for Atlas 1.5 to store your database credentials locally and to support future versions of Atlas, we do require a small table to be created in one of your SQL databases.

CREATE TABLE ethyca_atlas_connection ( 
id VARCHAR(50) PRIMARY KEY, 
name VARCHAR(50) NOT NULL, 
parameters TEXT NOT NULL, 
CONSTRAINT ethyca_atlas_connection_unq UNIQUE (name) 
);

Make sure to have the connection string to this database handy for the next step!

Step 3. Deploy your Atlas instance

docker run -it \
-e atlas.organization=<ETHYCA-PROVIDED ORGANIZATION NAME> \
-e atlas.ethyca.api.key=<ETHYCA-PROVIDED API KEY> \
-e atlas.ethyca.url=https://api.ethyca.com \
-e atlas.name=<NAME OF YOUR INSTANCE> \
-e atlas.connection.db.jdbc.url=<CONNECTION STRING TO LOCAL DATABASE> \
-e atlas.connection.db.jdbc.username=<LOCAL DB USERNAME> \
-e atlas.connection.db.jdbc.password=<LOCAL DB PASSWORD> \
-e atlas.schedule.cron='0 0 * ? * * *' \
-e atlas.threads=10 \
-e atlas.log.max.files = 28 
-e atlas.log.max.dir.size = 2G
-p 8080:8080 \
--name atlas-container ethyca/atlas:latest

Here is a breakdown of the parameters used for deployment:

Parameter Name

Description

atlas.organization

The unique organization name your Customer Success Manager provided to you previously, which facilitates communication between Atlas and Ethyca Services

atlas.ethyca.api.key

The API key your Customer Success Manager provided to you, which facilitates communication between Atlas and Ethyca Services

atlas.ethyca.url

The URL for Ethyca Services

atlas.name

A name for your Atlas instance - we suggest it be unique to your infrastructure

atlas.connection.db.jdbc.url

Connection string to the database containing the ethyca_atlas_connection table you created previously

atlas.connection.db.jdbc.username

Local database username

atlas.connection.db.jdbc.password

Local database password

atlas.schedule.cron

CRON scheduler for the frequency by which Atlas will poll Ethyca’s services for pending DSRs

atlas.threads

Maximum number of concurrent subject requests that can run in Atlas simultaneously

atlas.log.max.files

Maximum number of files that Atlas will store (located in /ethyca/logs)

atlas.log.max.dir.size

Maximum directory size for /ethyca/logs, specified in positive integer + size format. Examples include 100M, 2G

atlas.use.ssl (optional)

Sets a secure flag in the cookie for the Atlas UI. Accepted Values are true and false. False is the default value if not explicitly set.

atlas.hook.url (optional)

URL for Atlas to call if webhooks are configured. Webhooks are not invoked unless this is present.

atlas.hook.response.wait.time.minutes (optional)

The time Atlas will wait for a response after calling the webhook. Defaults to 10 minutes if not set.

Step 4. Safe-List or Open Outbound communication to Ethyca-hosted web services at api.ethyca.com


Configuring Atlas 1.5

To confirm successful deployment, you can enter the URL (or IP address) that you deployed Atlas to into a browser on whatever host your deployment is located at and then proceed to the next steps.

Step 1. Log in to Atlas

Using your Ethyca Control Panel credentials, you can log into your Atlas UI.

24002400

Pass the 2FA email code challenge and you're authenticated.

Step 2: Create a new database connection

  1. Press "Setup"
  2. Select the type of database for the connection
  3. Enter the connection details in the following fields:
  • Name: this is a memorable label for your database connection
  • URL: this is the url that Atlas will be able to access your database at.
  • Username and Password: these are the credentials for a database user that we recommend you provision specifically for Ethyca's Atlas. Please ensure that these have READ and WRITE access to the database and that the container where Atlas is deployed is able to access the database (ie: IP permissions list, security policy, etc)
  • Can Access: If selected, Atlas will only SELECT from your database.
  • Can Erase: If selected, Atlas will only UPDATE your database - never DELETE nor DROP
  1. Press Save Connection. Atlas will test the DB connection for you instantly. On failure, a banner will appear stating that the connection test failed and to check your credentials. Upon successful connection, the database will be added and you'll be taken to the Connections Listing page.

Database Type

URL Prefix

MongoDB

mongodb+srv://

Snowflake

jdbc:snowflake://

Databricks

jdbc:spark://

Databricks 7.3

jdbc:spark:// and suffix ;UID=token;PWD=<yourpassword>

Redshift

jdbc:redshift://

MySQL

jdbc:mysql://

PostgreSQL

jdbc:postgresql://

Microsoft SQL Server

jdbc:sqlserver://

Maria DB

jdbc:mariadb://

GCP BigQuery

connection string should contain a local link to your GCP credential file: jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=<YOURPROJECT>;DefaultDataset=<DBNAME>;OAuthType=0;OAuthServiceAcctEmail=

DynamoDB

connection credentials are configured using AWS IAM credentials, where you will need an Access Key ID and Secret Access Key. You can view instructions on how to generate this here.

Step 3: Select how you'd like to map your PII

24002400

In Atlas 1.5, we can connect to your database schema so you can identify where PII lives within it. This can be accomplished in one of two ways:

  1. Your Ethyca Customer Success Manager provides a .yaml file, also referred to as an Ethyca "Structure File", for each database connection you have configured. By pressing "UPLOAD YAML", you can upload this file here.
  2. To kick off the mapping activity without a .yaml file, press "UPDATE PII MAPPING" and you'll be taken to the mapping activity.

Step 4: Review the mapping for your PII

Now that you've connected a database accessible to Atlas, we can start identifying where PII lives in the database. The mapping activity will load your database schema (both tables and their requisite columns), so you can begin to identify which columns individually contain personally identifiable information table-by-table.

To begin, follow these 3 steps:

1. Select a table from the left-side listing. You'll see the listing of tables (A) that are accessible from the database you've just connected (B).

24002400
  1. Review the columns that populate in the center of the page. We will step through these row-by-row, indicating which of the detected columns contain PII, and for any PII that's there, classifying it and applying a label, masking strategy, and relationship linkage, where necessary.
24002400
  1. Remember to save!

Validating Your Deployment and Setup

Your Ethyca Customer Success team will be here every step of the way to ensure that your infrastructure environment is ready for Atlas and DSR-S. However, you can also validate your setup with a couple basic checks, such as:

  • accessing the host/IP on port 8080 where Atlas is deployed from within the container of VPC
  • confirming that the docker image is running locally

Validating your deployment

On startup, when your deployment initializes, Atlas will verify all connection configurations and structure files. Upon successful deployment, your deployed Atlas will register its status with Ethyca's web services. When Atlas is fully deployed, you'll see the following application log line:

INFO [DD-MM-YY HH:MM:SS] com.ethyca.atlas.job.DsrJob : DSR job started

If the startup fails, then there may be a problem with the connection or structure files. Contact your Ethyca Customer Success Manager for additional details.

If the startup succeeds, then your deployment of Atlas has successfully registered with Ethyca and you’re all ready to go!

When a subject request has been approved from your Ethyca Control Panel, you will see an Atlas application log line confirming the request has been accepted by Atlas and then executed on. Each step of the query execution is logged, and when it's finally completed, it will log that the request has completed its work in Atlas, like this:

Validating your Atlas setup with end to end testing

  1. First, follow the instructions in our guide to testing a Download My Data request.
  2. After a Download request has been tested, follow our guide to testing an Erasure request..

Support

Please contact [email protected] or your dedicated Customer Success Manager if you require any technical assistance for the installation or configuration of Ethyca's Atlas.