Local development

CipherStash supports local development and testing. However, it is important to note that when you activate the CipherStash integration, you will need to be connected to the Internet for your database integration (such as libpq) to function. In order to develop and test locally without the Internet, you will have to turn off the CipherStash integration.

This how to guide covers:

• Prerequisites
• How to sign in to CipherStash using Stash CLI
• How to create a dataset and client for local development
  • Environment variables
• Switching clients
• Switching datasets
• How to set up and configure for tests
  • Uploading the dataset configuration
  • Node.js with Sequelize
  • Ruby on Rails
• Running tests
  • Test in the plaintext-duplicate mode
  • Test in the encrypted-duplicate mode
  • Test in the encrypted mode
• Viewing the decryption logs
• Debugging
• Drop the plaintext columns

Prerequisites

Before starting, please ensure that you:

1. Have already created a CipherStash account (you can do this with the stash signup command)
2. Have already installed Stash CLI in your local development environment
3. Are connected to the Internet

And either:

• Have a Node.js and Sequelize application with a test suite that uses PostgreSQL 14 or above. Please refer to the Getting started guide with Sequelize for details.
• Have a Ruby on Rails application with a test suite that uses bundler and PostgreSQL 14 or above. Please refer to the Getting started guide with Rails for details.

How to sign in to CipherStash using Stash CLI

After you have created a CipherStash account and installed the Stash CLI tool, you can sign in to your account from the CLI tool.

To sign in, run this command from your shell:

stash login

Your web browser will open displaying a short code. Stash CLI should also show you the same code like this:

Logging in to console

### ACTION REQUIRED ###

Visit https://auth.cipherstash.com/activate?user_code=TBJG-QJRX to complete authentication by following the below steps:

1. Verify that this code matches the code in your browser

             +---------------------+
             |                     |
             |      TBJG-QJRX      |
             |                     |
             +---------------------+

2. If the codes match, click on the confirm button in the browser

Waiting for authentication...

Please check that they match, and click on “Confirm” if they do. At this point your may be asked to sign into your CipherStash account. If this happens, please sign in. After doing so, your browser should show a message telling you you can head back to your terminal and continue. You can close this browser window now.

Your terminal should show you a message like this:

Login succeeded!

How to create a dataset and client for local development

Once logged in, you are ready to create a dataset.

To create a dataset, please run this command from your terminal:

stash datasets create <DATASET NAME> --description <DATASET DESCRIPTION>

Please replace <DATASET NAME> with a name of your choice and <DATASET DESCRIPTION> with something that describes the dataset.

If successful, you should see something like this in your terminal:

Dataset created:
ID         : 32bf6da0-508e-4f74-832a-a8694debcc38
Name       : dataset-for-documentation
Description: dataset created for documentation writing

Now you should make a client. To do so, please run this command in your terminal:

stash clients create --dataset-id 32bf6da0-508e-4f74-832a-a8694debcc38 'test clients for documentation'

Upon successful creation of clients, you should see messages like these:

#################################################
#                                               #
#  Copy and store these credentials securely.   #
#                                               #
#  THIS IS THE LAST TIME YOU WILL SEE THE KEY.  #
#                                               #
#################################################


Client ID          : b2a4f565-b4f6-4914-a0...

Client Key [hex]   : a4627031a16b7065726d7...

Securely store those values somewhere, as you will need them in the following steps.

Environment variables

The following sections assume that CS_CLIENT_ID and CS_CLIENT_KEY are set in your environment variables. Stash CLI will use those environment variables as your client ID and client key. You can also use --client-id=<CLIENT_ID> and --client-key=<CLIENT_KEY> options instead if you prefer.

Please ensure those variables are exported in your environment but remember to use your own client ID and client key from the previous step:

export CS_CLIENT_ID=b2a4f565-b4f6-4914-a0...
export CS_CLIENT_KEY=a4627031a16b7065726d7...

If you are using direnv, please put the above in your .envrc in your project root directory and run:

direnv allow

Switching clients

In case you need to switch your client, you can create a new client and start using it:

1. Create a new client for the dataset (unless you already have done so)
2. Set CS_CLIENT_ID and CS_CLIENT_KEY environment variables
3. Optionally, use stash clients revoke to revoke the old client ID and key

Please refer to this section for how to create a new client.

Switching datasets

You might have different datasets you want to switch between. To switch from one dataset to another, you will have to:

1. Create and configure a new data set (unless you already have done so)
2. Create a new client for the dataset (unless you already have done so)
3. Set CS_CLIENT_ID and CS_CLIENT_KEY environment variables

For creating the dataset and client, please refer to the this section above.

How to set up and configure for tests

Once you have the client ID and client key, you can configure your app to use them. This section only covers basics of how to enable encryption with minimal configuration. For this section, we assume you have a simple application that only has a database table called users which has a name column. For a more complete example, please refer to our Getting started guide with Sequelize or the Getting started guide with Rails.

Uploading the dataset configuration

Before going into the specifics for Sequelize or Rails, your dataset configuration must be uploaded.

If you already have your dataset configuration file, please use it. If you do not, you can modify the following to suit your app and save it as dataset.yml. This is a minimal example that assumes you have an app that has a users table which has a field called name, but you should be able to modify it for your existing application. This example config specifies applying encryption for the name field of the users table, and an index for looking up by exact values:

tables:
  - path: users
    fields:
      - name: name
        in_place: false
        cast_type: utf8-str
        mode: plaintext-duplicate
        indexes:
          - version: 1
            kind: unique

Assuming your dataset configuration file is called dataset.yml, please run:

stash datasets config upload --file dataset.yml

After successfully uploading your configuration file, you should see something like the following, which may be different if you used your own configuration file:

Uploaded dataset_config:
Table Name: users
┌────────────┬───────────┬────────────┬───────────────────────────────┬───────────────────────┐
│ Field Name │ Cast Type │ Operations │ Source Columns                │ Index Columns         │
├────────────┼───────────┼────────────┼───────────────────────────────┼───────────────────────┤
│ name       │ Utf8Str   │ ==         │ Column: name                  │ Column: __name_unique │
│            │           │            │ └─ Plaintext: [Reads, Writes] │ ├─ Version: 1         │
│            │           │            │                               │ ├─ Operations: ==     │
│            │           │            │ Column: __name_encrypted      │ └─ Index Type: Unique │
│            │           │            │ └─ Encrypted: [Writes]        │                       │
│            │           │            │                               │                       │
│            │           │            │ Mode: PlaintextDuplicate      │                       │
│            │           │            │                               │                       │
└────────────┴───────────┴────────────┴───────────────────────────────┴───────────────────────┘

Node.js with Sequelize

If you are using Node.js with Sequelize, you have to take the following steps for local development with CipherStash integration with Sequelize:

1. Add the CipherStash driver npm packages
2. Add and apply migrations
3. Save encrypted data

Add the CipherStash driver npm packages

CipherStash packages can be installed with these commands:

npm add @cipherstash/libpq
npm add pg-native@npm:@cipherstash/pg-native

Add and apply migrations

CipherStash needs custom database types and functions to operate. Please refer to the Getting Started Guide for how this is done.

Once you have added the extensions, you will need to add encrypted columns and index columns (such as __name_encrypted, __name_unique and an index on __name_unique). Please refer to the Getting Started Guide for an example.

Save encrypted data

Once you have added the columns for the encrypted data, you will have to populate them. Please refer to the Getting Started Guide for how this is done.

Ruby on Rails

If you are using Ruby on Rails, you have to take the following steps for local development with CipherStash integration with Rails:

1. Update the Gemfile and local gems
2. Update config/database.yml
3. Add and apply migrations
4. Update the Active Record models

Update the Gemfile and local gems

Now you can replace the pg gem with activerecord-cipherstash-pg-adapter. Here, we are commenting out the line for pg and adding the CipherStash gem:

# gem 'pg'
gem 'activerecord-cipherstash-pg-adapter'

After updating the Gemfile, please run:

bundle install

Update config/database.yml

With the activerecord-cipherstash-pg-adapter gem, you also need to update the database configuration. Please replace the postgresql adapter with postgres_cipherstash in config/database.yml:

  # adapter: postgresql
  adapter: postgres_cipherstash

Add and apply migrations

CipherStash needs its database extensions to operate. Please refer to the Getting Started Guide for an example of how to do it.

Once you have added the extensions, you will need to add encrypted columns and index columns (such as __name_encrypted, __name_unique and an index on __name_unique). Please refer to the Getting Started Guide for an example.

If you have any existing data, you should also run the task to populate existing rows like so:

rake cipherstash:migrate[User]

Update the Active Record models

In all model classes where you have added encrypted columns, please add:

self.ignored_columns = %w[wildcardoperatorfix]

For example, the User model example may look like this:

class Patient < ApplicationRecord
	self.ignored_columns += %w[wildcardoperatorfix]
end

Running tests

For both Sequelize and Rails applications, you can test your application manually, or use scripted tests if you have them. In any of the following modes, you should be able to either manually test your application or run your scripted tests (such as rake spec or npm test) and it should not affect the results, although it may affect performance. Please refer to your application’s documentation on exactly how you should do this.

Test in the plaintext-duplicate mode

If you set mode: plaintext-duplicate in dataset.yml and uploaded it, your application is configured to run in the encrypted-duplicate mode. In this mode, both plaintext and encrypted columns are written to, but plaintext columns are used for reading.

Test in the encrypted-duplicate mode

In the encrypted-duplicate mode, both ciphertext and plaintext columns are written but the ciphertext is be used for reading.

If you switch to this mode, your test results should still not be affected. To do this, please edit your dataset configuration file (dataset.yml) and replace all instances of plaintext-duplicate to encrypted-duplicate and re-upload your dataset configuration file (stash datasets config upload --file dataset.yml).

Test in the encrypted mode

In the encrypted mode, only the encrypted columns are written and read. The plaintext column will be left untouched for both new and existing records.

If you wish to switch to this mode, please replace all instances of encrypted-duplicate to encrypted in your dataset configuration file and re-upload (stash datasets config upload --file dataset.yml).

Viewing the decryption logs

To check that your application is utilising the CipherStash integration, you can view decryption logs with the following command:

tail -F ~/.cipherstash/*/decryptions.log

Debugging

If you see unexpected behaviour or would like to check how the queries are mapped, you can turn set CS_DEBUG=1 in your environment:

export CS_DEBUG=1

This will make your program print debug output to its STDOUT.

Drop the plaintext columns

Once you have checked your application runs fine, you can drop your plaintext columns. To do this, please use Rails migrations to drop those columns. You can refer to the Getting started guide for Sequelize or Rails for an example.