Manage app credentials with AWS Secrets Manager

Introduction

Applications often make use of external resources or services, such as databases and third party APIs, which require credentials in order to be interacted with. The simplest option is to store credentials in the source code - in a .NET app this will typically be in an appsettings.json file - however there are security and convenience implications associated with these methods. In this article, we'll delve into various conventions for handling credentials, including a discussion about the associated problems and pitfalls, then move onto the new best practise of abstracting these credentials ("secrets") to a cloud service, which in this example will be AWS Secrets Manager.

Assumptions

• Moderate familiarity with .NET
• Access to an AWS account with administrator privileges
• A basic understanding of how credentials can be stored and used by an application

Application credentials and their purpose

Many applications built today make use of a microservice-style architecture, with resources and services abstracted away from a single monolith codebase hosted on a single piece of infrastructure, and often making use of third party services in place of custom code.

If you're in the process of breaking up your monolith application into smaller services, for example, by moving one of the APIs so that it can be managed separately, whether that be simply in a separate repository to the main codebase, or deployed to a different type of infrastructure, those two pieces of your application still need to interact with each other securely. In this case, an API key would be used by the client application to authenticate and authorise requests on the remote server, and this API key needs to be kept safe, and away from prying eyes to prevent misuse.

Similarly, your application may make use of a database to manage things like customer or product data, and the connection string used by the app includes not only the location of the server and database name, but typically also a set of user credentials which grant permissions to query, create, and manipulate data.

"ConnectionStrings": {
    "MyApp": "Server=127.0.0.1;Port=5432;Database=MyDatabase;User Id=app.username;Password=password;"
}

Managing application secrets the old way

Anyone who has worked in software development for a few years will have likely seen a variety of methods employed for storing and using application credentials. Some methods are chosen for convenience and may forgo more secure options due to low risk, or because the application is just in the concept phase, whereas other methods are required, often by law or industry regulations, to help ensure data security and integrity.

Hardcoded credentials in the source code

The simplest, and let's not beat about the bush here, worst approach to managing credentials, is to hardcode credentials into whatever piece of source code needs it. This might be written into a class which interacts with a database or external API, and stores the connection string or API key in a private field, for example:

private readonly string _apiKey = "ad8262fa-b50c-4a1c-9c43-e541a1aa1e95"

This method presents two immediately obvious issues:

1. Credentials are stored in plain text: anyone with access to the source code can see the credentials (e.g. developers), and they will also be persisted in source control history, so anyone with access to the remote repository will be also able to view them
2. Credentials can't be changed easily: if a password or secret needs to be changed, either as part of a secret rotation security policy, or is accidentally leaked and needs to be changed immediately, the application will fail until the source code is updated and the application can be redeployed with the updated credentials.

Using environment variables on the server

This option abstracts secrets from the source code entirely by requiring variables to be set on the destination server directly prior to deployment. Applications typically retrieve environment values by key (e.g. MYAPP_API_KEY) during the startup process, then store them in memory for later use.

An immediate benefit of this method is that code only references the key which stores the secret, so the secret can be easily be updated without needing changes to the application code. Although this method is certainly better than hardcoded strings, there are still a few noteworthy limitations:

1. Credentials are still stored in plain text: environment variables aren't encrypted, so anyone with access to the server can see the credentials if they know where to look; this may include system admins, DevOps engineers, or developers who periodically connect to the server to perform maintenance or check logs files etc.
2. Managing environment variables isn't straightforward: variables may need to be added and updated manually by a developer or system administrator, and ideally the secrets would also need to be stored elsewhere to ensure they're not lost if there's a system failure, or the variables are mistakenly changed or lost.

Using environment-specific configuration files

This option is very common in modern applications, any many templates for new applications come with settings files pre-configured. For example, .NET applications have appsettings.json files configured in the Program or Startup classes, including provisions for local development with the appsettings.Development.json file. Likewise, developers from the JavaScript ecosystem (e.g. React, Node) may be familiar with the dotenv package, which uses file names such as .env.local which function in a similar manner.

A key benefit of these settings files, is that they allow credentials to be stored and split by environment. Development settings can remain on the local development machine and don't need to be committed to source control, and production files can be moved around and stored elsewhere and don't ever need to be seen by developers, minimising the risk of leaking sensitive data, or accidentally committing sensitive data to source control.

Similar to the previous solutions, there are still some downsides to take into consideration, such as:

1. Settings file storage/maintenance: appsettings.json files, especially for production, need to be configured to deploy independently from the application, typically as part of a CI/CD pipeline. These files or individual settings still need to be stored somewhere before they can be used
2. Credentials are still stored in plain text: a recurring point, appsettings aren't encrypted as they're used by the application as raw strings, as such, anyone involved in deployment can view plain text credentials. If you use a CI/CD pipeline to manage deployments, credentials will be stored in plain text on the deployment service (e.g. Octopus, TeamCity etc.).

Manage secrets using a secure cloud service

Cloud services such as AWS Secrets Manager and Azure Key Vault allow users to store secrets in a central location, and retrieve them by key when needed. The benefits of these services make the small amount of extra configuration and cost well worth the effort, for example:

1. Encryption: secrets are encrypted on these services, and can't be read as plain text without the encryption key
2. Convenience: being in a centralised location allows secrets to managed independently from the application. If you need to change the secret, it just needs to be updated in a single place, and any apps which use the secret will retrieve the relevant version. If you use the CLI or SDK, these can also be changed programmatically
3. Additional features: along with storage and retrieval, you can typically also configure automatic key rotation and replication, along with monitoring features for auditing usage
4. Cheap: although the other solutions are free, with the potential exception of a CI/CD service, cloud secret storage services are very cheap, with the costs being essentially negligible alongside even the smallest hosting bill - see AWS Secrets Manager pricing, and Azure Key Vault pricing for up to date details.

We will now look at how to store an API key in AWS Secrets Manager, and retrieve it for use in our application using the AWS SDK.

Note: The IAM role attached to your console or CLI user will need the secretsmanager:CreateSecret and secretsmanager:GetSecretValue permissions in order to perform the following actions.

Add a secret using the AWS console

Login into the console and navigate to the Secrets Manager service, then click Store a new secret. We're storing an API key, so select the "Other type of secret" option, add your key(s) and value(s) and leave the encryption key as the default.

Proceed to the next screen and enter a meaningful secret name and optional description. Skip through to the end and copy the generated sample code from the C# tab.

Add a secret using the AWS CLI

If you prefer the simplicity of a command line tool, or intend to create secrets programmatically, the creation process can also be performed using the AWS CLI. The following command, using PowerShell, creates the same secret as above (with a different name):

aws secretsmanager create-secret --name TestApiKey-CLI --secret-string '{\"API_KEY\": \"3456a36b-f462-4380-854f-d23872815e34\"}'

# Output
# {
#     "ARN": "arn:aws:secretsmanager:eu-west-2:xxxxxxxxxxxx:secret:TestApiKey-CLI-Sv5aKJ",
#     "Name": "TestApiKey-CLI",
#     "VersionId": "954fa2ea-d194-445d-950d-287ba41a2098"
# }

You can confirm the creation by calling get-secret-value, or by checking the console.

aws secretsmanager get-secret-value --secret-id TestApiKey-CLI

# Output
# {
#     "ARN": "arn:aws:secretsmanager:eu-west-2:xxxxxxxxxxxx:secret:TestApiKey-CLI-Sv5aKJ",
#     "Name": "TestApiKey-CLI",
#     "VersionId": "954fa2ea-d194-445d-950d-287ba41a2098",
#     "SecretString": "{\"API_KEY\": \"3456a36b-f462-4380-854f-d23872815e34\"}",
#     "VersionStages": [
#         "AWSCURRENT"
#     ],
#     "CreatedDate": "2024-03-17T10:37:53.389000+00:00"
# }

Full CLI documentation for Secrets Manager can be found here.

Use the AWS SDK to retrieve a secret

We will now use the SDK to retrieve our saved secret so that it can be used in our application. In your solution, start by add the AWSSDK.SecretsManager NuGet package to your project.

If you created your secret using the console, simply paste the code you copied earlier into the class which will be making use of the secret. If you used the CLI, the example from my secret is shown below:

using Amazon;
using Amazon.SecretsManager;
using Amazon.SecretsManager.Model;

static async Task GetSecret()
{
    string secretName = "TestApiKey";
    string region = "eu-west-2";

    IAmazonSecretsManager client = new AmazonSecretsManagerClient(RegionEndpoint.GetBySystemName(region));

    GetSecretValueRequest request = new GetSecretValueRequest
    {
        SecretId = secretName,
        VersionStage = "AWSCURRENT", // VersionStage defaults to AWSCURRENT if unspecified.
    };

    GetSecretValueResponse response;

    try
    {
        response = await client.GetSecretValueAsync(request);
    }
    catch (Exception e)
    {
        // For a list of the exceptions thrown, see
        // https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
        throw e;
    }

    string secret = response.SecretString;

    // Your code goes here
}

Ensuring you edit the secretName and region as required, you can then call this method to retrieve your secret and view the response.

Since SecretString returns a string consisting of an array of key/value pairs, you'll need to convert and extract the required pair before using the key. This can be done by parsing the JSON with System.Text.Json.JsonSerializer, and then converting the resulting object into a dictionary of strings, then grabbing the applicable item's value. In my case there's only one pair so I can use First():

string? secret = JsonSerializer
    .Deserialize<Dictionary<string, string>>(response.SecretString)?
    .First()
    .Value;