Context and Cloud Infrastructure Choices.
Lenstra was approached by a financial services company seeking to enhance its security measures during their migration to the Cloud. This transition resulted in a hybrid infrastructure where applications were deployed across both on-premise and Cloud-based Kubernetes clusters.To securely manage production secrets in this new environment, the company used HashiCorp Vault. Each application project was assigned its own Vault namespace, ensuring strict isolation and security.
Challenges of Cloud Migration.
Although this approach gave excellent security results, it presented several challenges:
- Manual Integration: Kubernetes deployments had to be integrated with HashiCorp Vault manually by each development team, either through direct application integration or by running a Vault sidecar alongside the application.
- Increased Development Time: The manual process led to redundant efforts, as each team repeated similar tasks.
- Inconsistent Configurations: Different teams approached the integration in slightly varied ways, leading to inconsistencies across the organization.
- Higher Risk of Misconfigurations: These inconsistencies increased the likelihood of misconfigurations, potentially compromising both security and efficiency.
Lenstra’s Objective:
Improve the integration process to maximize productivity while minimizing the risk of misconfigurations.
The Solution
Solutions proposed to ease the transition from the existing environment to the cloud system :
- Agent Injector: Automates the injection of Vault agents into Kubernetes pods, supports templating, dynamic secrets, and most authentication methods. It ensures that no durable secrets are stored outside Vault, with all secrets written to ephemeral in-memory volumes.
- Vault Secrets Operator: Integrates HashiCorp Vault with Kubernetes' native secret management system, providing an excellent user experience for developers. However, it syncs Vault secrets to Kubernetes, which acts as a durable cluster-local cache—potentially compromising security by storing secrets outside of Vault.
- Vault CSI Provider: Manages secrets as volumes within Kubernetes, offering a standardized and efficient method for handling secrets. However, it currently lacks support for caching, secret rotation and secret templating.
Selecting the best tool to facilitate cloud migration without compromising security.
Our goal was not only to improve the developer experience but also to maintain the same security standards by ensuring that secrets were not stored outside HashiCorp Vault. This requirement ruled out the Vault Secrets Operator: Kubernetes does not currently support ephemeral secrets and saves them to the etcd store.After testing, we determined that the Agent Injector was the most suitable solution. It is a mature tool that supports secret templating and provides an excellent user experience, making it the best choice for our needs.
The Bottom Line
By automating the integration between HashiCorp Vault and Kubernetes using the Agent Injector, we expect to achieve faster software delivery, consistent configurations, and reduced risk of misconfigurations while maintaining the current security standards.We are also monitoring for new features in the integration of Vault and Kubernetes, hoping to improve the developer experience further. For instance, the support for secrets rotation in the Secrets Store CSI Driver, currently in alpha, should be included in a future Kubernetes release.
