Environment variables were the standards for passing configurations, keys, and deployment assets in the system for a long time. It's even recommended by the twelve-factor apps guide.
No syntax, No composition
But the problem is once your configuration starts getting too complex, it's very tough to manage environment variables. Currently, in one of my projects, the environment variable count is 53 and it's bound to increase substantially when we deploy our next release. If only there was composition we could identify which are common variables to be used in all stages and which are the unique ones. For this problem, what we're doing currently is namespacing the environment variables via underscore.
For example MY_COMMERCE_DB_HOST_NAME. Now, when we deploy another environment it's very tough to identify which environment variable is used in which aspect of the project and how. The complexity and no data types of environment variables is only one part of the problem, and probably a lesser significant one.
Security Issues
Environment variables increase the attack surface for your application when sensitive values like SSH Keys, Database Passwords, authentication credentials, and API Keys are passed via it. Most of the loggers dump all your environment variables into the log file when your application crashes along with the stack trace which silently passes down your secret variables into a plain text log file. Also, the environment variables when supplied to a parent process are passed down to every child process. Now, if you're spawning a subprocess to some third party process it has access to all of your secrets. In ephemeral deployment solutions like AWS Lambda, Azure Functions it maybe not be much of an issue because you can use your own master key to sign your environment variables, and also the container lifecycle is ephemeral which significantly reduces the risk of leaking down your variables but still, it's doable and is happening.
What is my approach?
Store your configuration file in YAML.
It's easy enough to create language and OS agnostic configuration files (INI, YAML, etc.) and it's usually straightforward to make files easy to change between deployments too - e.g. if a deployment is containerized, by mounting the file.
You can replace it like this:
APP_ID=123
APP_HOST=domain.com
APP_LOGGER=logging-service
DB_HOST=domain.com
DB_PORT=5433
DB_USERNAME=mmm
app:
id: 123
host: domain.com
logger: logging-service
db:
host: domain.com
port: 5433
username: mmm
Now, this is significantly easier to parse and pass down the road to required places which also supports the dependency inversion principle. There are packages that can create schemas from YAML files which will even help you further. Also, you can use YAML validators to add different validations, now your configuration has its own syntax, and data types are composite.
Store your secrets in an ephemeral file system.
Different containerization platforms have their own way of passing down secrets in an ephemeral file system. If you're using Docker in swarm mode, you can utilize the full benefit of docker secrets. Kubernetes has its own configuration mechanisms for passing secrets. One of the methods I personally suggest to pass secrets is using Hashicorp Vault which is very good for implementing in zero-trust environments.
Also, the cloud provider has some sort of secrets management service like AWS Secrets Manager which has a variety of security mechanisms as a service.
