esec: Encrypted Secrets Management for Go

Esec is a Go library and CLI tool for managing encrypted secrets using environment files (.env), JSON (.ejson), and other formats. It allows developers to securely store and retrieve sensitive configurations using public-key cryptography.

It draws heavy inspiration from the EJSON project and aims to provide a similar experience for Go developers. A large part of the crypto related code and the file format handling is also inspired by or directly taken from the EJSON project.

Main differences are that esec is more opinionated on the file naming conventions and the key lookup process. Ejson writes the keys to a local dir in the format of keydir/<public-key>/<private-key> and then looks up the keys from there. esec uses environment variables and a .esec-keyring file for key lookup.

Features

✅ Secure secrets storage using public/private key encryption.
✅ Support for multiple formats (.env, .ejson).
✅ Decryption of secrets in embedded or external vaults.
✅ CLI tool for encryption & decryption.
✅ Run commands with decrypted environment variables.
✅ Flexible environment & key management via keyring file.
✅ Detailed debug logging.
✅ Integrates easily into Go applications.

Todo

• Add support for more file formats (.eyaml, .etoml).
• Add run command to decrypt and execute commands with environment variables.
• Enhance keyring detection with ESEC_ACTIVE_KEY and ESEC_ACTIVE_ENVIRONMENT.
• Add structured debug logging.
• Add more test coverage.
• Standardize the package API.

---

Installation

Using Go Modules

go get github.com/mscno/esec

CLI Installation

go install github.com/mscno/esec/cmd/esec@latest

Using the CLI

Generate Keys

esec keygen

Output:

Public Key:
e50e7c0086bfac43263dc087dc9a0118d3b567d26a87c22876690bca8b50c00c
Private Key:
dfe357ede9f3b42b34ac1fca814a27a99f610e4fde361d09b78adcc659b88b79

Encrypt Secrets

esec encrypt .ejson.dev
# or with an environment
esec encrypt dev

Decrypt Secrets

esec decrypt .ejson.dev
# or with an environment
esec decrypt dev

Run Commands with Decrypted Secrets

The run command allows you to decrypt a secrets file, set the values as environment variables, and execute a command in that environment:

esec run dev -- myapp serve
# equivalent to 
esec run .ejson.dev -- myapp serve

This will decrypt .ejson.dev, set all environment variables found in the file, and then run myapp serve with those variables available.

or with dotenv files

esec run production -f .env -- myapp serve
# equivalent to 
esec run .env.production -- myapp serve

This will decrypt .env.production, set all environment variables found in the file, and then run myapp serve with those variables available.

Debug Mode

Use the --debug flag to enable detailed logging:

esec --debug decrypt dev

ESEC Naming Convention and Key Lookup Process

Naming Convention

The esec library follows a structured naming convention for environment-based encryption files. This convention ensures consistency when encrypting and decrypting secrets across multiple formats.

File Naming Conventions

File Type	Naming Pattern	Naming with Environment (dev)	Description
Encrypted Environment Files	.env	.env.ejson.dev	An encrypted .env file.
Encrypted JSON Files	.ejson	.ejson.dev	A JSON file with encrypted fields.
Encrypted YAML Files	.eyaml	.eyaml.dev	A YAML file with encrypted fields.
Encrypted TOML Files	.etoml	.etoml.dev	A TOML file with encrypted fields.

---

Standard vs. Environment-Specific Naming

What Happens if No Environment is Provided?

If no environment name is provided when running an encryption or decryption command, esec assumes the standard environment. This means:

• The filename will not include an environment suffix.
• The key lookup process will only use the default private key (ESEC_PRIVATE_KEY).

Example Behavior

Command	Resolves To	Private Key Lookup
esec encrypt	.ejson	ESEC_PRIVATE_KEY
esec encrypt dev	.ejson.dev	ESEC_PRIVATE_KEY_DEV
esec decrypt	.ejson	ESEC_PRIVATE_KEY
esec decrypt dev	.ejson.dev	ESEC_PRIVATE_KEY_DEV

If no specific environment is provided, esec defaults to using the blank/standard environment, meaning:

1. For encryption, it encrypts to .ejson.
2. For decryption, it tries to find ESEC_PRIVATE_KEY to decrypt .ejson.

---

How File Naming Works in Encryption & Decryption

1. esec encrypt dev vs. esec encrypt .ejson.dev

These two commands produce the same output because dev is treated as an environment name, and the system automatically constructs the corresponding filename.

esec encrypt dev

esec encrypt .ejson.dev

---

Why? • When you pass an environment name, esec automatically generates the corresponding file name using the .ejson format. • The default format for encrypted environment files is .ejson, so dev becomes .ejson.dev.

2. How the Private Key is Located

When decrypting an encrypted file, esec looks for the private key in the following order:

Step 1: Search for Environment Variables

esec first checks if the private key is set in environment variables:

export ESEC_PRIVATE_KEY=your-private-key
export ESEC_PRIVATE_KEY_DEV=your-private-key
export ESEC_PRIVATE_KEY_PROD=your-private-key

If the input file is .ejson.dev, esec searches for:

1. ESEC_PRIVATE_KEY_DEV

If the file is .ejson.prod, esec searches for:

1. ESEC_PRIVATE_KEY_PROD

If a matching private key is found, it is used for decryption.

Step 2: Check .esec-keyring File

If the private key is not found in environment variables, esec looks for a .esec-keyring file in the current directory.

Example .esec-keyring file:

# .esec-keyring file
ESEC_PRIVATE_KEY_DEV=your-private-key
ESEC_PRIVATE_KEY_PROD=your-private-key

# Optional - explicitly specify which environment to use when decrypted from embeded file
ESEC_ACTIVE_ENVIRONMENT=dev

# Alternative - specify which key to use
# ESEC_ACTIVE_KEY=ESEC_PRIVATE_KEY_DEV

If esec is decrypting .ejson.dev, it searches for ESEC_PRIVATE_KEY_DEV inside .esec-keyring.

The .esec-keyring file supports two special variables to manage which environment to use:

1. ESEC_ACTIVE_ENVIRONMENT - Directly specifies the environment name (e.g., "dev", "prod")
2. ESEC_ACTIVE_KEY - Specifies which key to use (e.g., "ESEC_PRIVATE_KEY_DEV")

If neither of these special variables is set and multiple private keys are found in the keyring file, esec will look for one that matches the file being decrypted.

If a matching key is found, it is used for decryption.

esec File Formats

esec JSON (.ejson)

The esec JSON document format is simple, but there are a few points to be aware of:

• It's just JSON.
• The file must have a ESEC_PUBLIC_KEY field at the top level.
• There must be a key at the top level named _public_key, whose value is a 32-byte hex-encoded (i.e. 64 ASCII byte) public key as generated by ejson keygen.
• Any string literal that isn't an object key will be encrypted by default (ie. in {"a": "b"}, "b" will be encrypted, but "a" will not.
• Numbers, booleans, and nulls aren't encrypted.
• If a key begins with an underscore, its corresponding value will not be encrypted. This is used to prevent the _public_key field from being encrypted, and is useful for implementing metadata schemes.
• Underscores do not propagate downward. For example, in {"_a": {"b": "c"}}, "c" will be encrypted.

Example:

{
  "_ESEC_PUBLIC_KEY": "493ffcfba776a045fba526acb0baff44c9639b98b9f27123cca67c808d4e171d",
  "MY_VALUE": "hello",
  "MY_NUMBER": 123,
    "MY_ARRAY": [
        "hello",
        "world"],
  "MY_OBJECT": {
      "hello": "world"
  },
  "_metadata": {
    "_created_at": "2021-10-10T10:10:10Z"
  }
}

Encrypted Example:

{
  "_ESEC_PUBLIC_KEY": "493ffcfba776a045fba526acb0baff44c9639b98b9f27123cca67c808d4e171d",
  "MY_VALUE": "ESEC[1:HMvqzjm4wFgQzL0qo6fDsgfiS1e7y1knsTvgskUEvRo=:gwjm0ng6DE3FlL8F617cRMb8cBeJ2v1b:KryYDmzxT0OxjuLlIgZHx73DhNvE]",
  "MY_NUMBER": 123,
    "MY_ARRAY": [
        "ESEC[1:HMvqzjm4wFgQzL0qo6fDsgfiS1e7y1knsTvgskUEvRo=:05gVhGzlZ+uAkDhUQkF/Ek8ketC9ta9f:bxHz36i/Etrl3BSGwCw5CmNix89t]",
        "ESEC[1:HMvqzjm4wFgQzL0qo6fDsgfiS1e7y1knsTvgskUEvRo=:grLDhSld77JjdwXfSrooIp1Trl0/4/5u:iz+qKEJgk4+xD0f04VRInoL3AJOH]"],
  "MY_OBJECT": {
      "hello": "ESEC[1:HMvqzjm4wFgQzL0qo6fDsgfiS1e7y1knsTvgskUEvRo=:3Zcx6Quy0mj5MdUDJduNKGgPDqBOLHYB:s9/u1dhQtYoeWGymnZlWogT8UnMR]"
  },
  "_metadata": {
      "_created_at": "2021-10-10T10:10:10Z"
  }
}

esec Dotenv (.env)

The esec dotenv format is a simple extension of the standard dotenv format. It allows you to encrypt values in your .env files.

• The file must have a ESEC_PUBLIC_KEY field.
• It's just a standard dotenv file.
• Encrypted values are prefixed with ESEC[ and suffixed with ].
• The key is not encrypted, only the value.
• Comments are not encrypted.
• Blank lines are not encrypted.
• The ESEC_PUBLIC_KEY field is not encrypted.

Example:

# Some comment

ESEC_PUBLIC_KEY=493ffcfba776a045fba526acb0baff44c9639b98b9f27123cca67c808d4e171d #secret

MY_VALUE=some_secret_value

Encrypted Example:

# Some comment

ESEC_PUBLIC_KEY=493ffcfba776a045fba526acb0baff44c9639b98b9f27123cca67c808d4e171d #secret

MY_VALUE=ESEC[1:uFOJzedrCFCn2wBvZJT+5hG/nFY6pDPJ3cP6E2OxHTQ=:dMlog4zL55ar0O2szkZWYPZUWgA5ypRv:CPOF3sboowCHClcvE7hidYh/9PzX]

Integrating with Go

1. Encrypting Secrets in Go

package main

import (
	"bytes"
	"fmt"
	"github.com/mscno/esec"
)

func main() {
	// Sample secret data
	data := []byte(`{"_ESEC_PUBLIC_KEY": "8d8647e2eeb6...", "api_key": "supersecret"}`)
	
	// Encrypt the data
	var output bytes.Buffer
	_, err := esec.Encrypt(bytes.NewReader(data), &output, esec.Ejson)
	if err != nil {
		panic(err)
	}

	fmt.Println("Encrypted Output:", output.String())
}

2. Decrypting Secrets in Go

package main

import (
	"bytes"
	"fmt"
	"github.com/mscno/esec"
	"os"
)

func main() {
	// Set private key as an env variable
	os.Setenv("ESEC_PRIVATE_KEY", "c5caa31a5b8cb...")

	// Encrypted JSON string
	encryptedData := `{"_ESEC_PUBLIC_KEY": "8d8647e2eeb6...", "api_key": "ESEC[...]"}`
	
	// Decrypt the data
	var output bytes.Buffer
	_, err := esec.Decrypt(bytes.NewReader([]byte(encryptedData)), &output, "", esec.Ejson, "", "")
	if err != nil {
		panic(err)
	}

	fmt.Println("Decrypted Output:", output.String())
}

3. Decrypting from an Embedded Vault

These examples demonstrate how to decrypt secrets from an embedded vault using the embed package.

Basic Example

This example assumes that the vault contains an encrypted file named .ejson.

package main

import (
	"embed"
	"fmt"
	"github.com/mscno/esec"
	"os"
)

//go:embed secrets/*
var vault embed.FS

func main() {
	// Set private key
	os.Setenv("ESEC_PRIVATE_KEY", "c5caa31a5b8cb2...")

	// Decrypt from embedded vault
	data, err := esec.DecryptFromVault(vault, "", esec.Ejson)
	if err != nil {
		panic(err)
	}

	fmt.Println("Decrypted Vault Data:", string(data))
}

Environment-specific Example

This example assumes that the vault contains an encrypted file named .ejson.prod.

package main

import (
	"embed"
	"fmt"
	"github.com/mscno/esec"
	"os"
)

//go:embed secrets/*
var vault embed.FS

func main() {
	// Set private key
	os.Setenv("ESEC_PRIVATE_KEY_PROD", "c5caa31a5b8cb2...")

	// Decrypt from embedded vault
	data, err := esec.DecryptFromVault(vault, "", esec.Ejson)
	if err != nil {
		panic(err)
	}

	fmt.Println("Decrypted Vault Data:", string(data))
}

Using the Config-based API

For more control, you can use the enhanced config-based API:

package main

import (
	"embed"
	"fmt"
	"github.com/mscno/esec"
	"log/slog"
	"os"
)

//go:embed secrets/*
var vault embed.FS

func main() {
	// Set up a logger
	logger := slog.New(slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{
		Level: slog.LevelDebug,
	}))

	// Create a configuration
	config := esec.DecryptFromEmbedConfig{
		EnvName: "dev",                    // Explicitly set environment
		Format:  esec.FileFormatEjson,     // Specify format
		Logger:  logger,                   // Add logging
		Keydir:  "/path/to/keyring/dir",   // Optional keyring directory
	}

	// Decrypt using the config
	data, err := esec.DecryptFromEmbedFSWithConfig(vault, config)
	if err != nil {
		panic(err)
	}

	fmt.Println("Decrypted Vault Data:", string(data))
}

4. Using the Run Command

The Run command allows you to decrypt a secrets file, set environment variables, and run a command:

package main

import (
	"fmt"
	"github.com/mscno/esec/cmd/esec/commands"
	"os"
)

func main() {
	// This would typically be called from your main CLI entrypoint
	commands.Execute("1.0.0")
	
	// Example CLI usage:
	// esec run dev -- myapp serve
	
	// This will:
	// 1. Decrypt the .ejson.dev file
	// 2. Set all variables as environment variables
	// 3. Run the specified command with those variables
}

Why Use esec?

• ✅ Simple & Secure
• ✅ Supports Multiple Formats
• ✅ Works with Embedded & External Secrets
• ✅ CLI & Go Integration