Considerations for Dynamic Secrets Management
As computing becomes more powerful and Quantum computing looms, the time to break a key or a password is drastically reduced, especially when considering Shor’s algorithm. Factor in that statistically, only 50% of the attempts to break a password are required to actually crack a password, and the need for short lived, randomized credentials becomes blatantly apparent. Dynamic credentials are highly effective at mitigating this risk and shrinking the potential blast radius of a credential breach. By generating credentials on demand for consuming workloads which are limited to a number of uses or a short time frame ensures that even leaked credentials, whether the leak is human, pipeline, or machine based, has effectively mitigated that threat vector.
Dynamic salting of passwords is not a new concept to credential management, particularly when storing passwords at rest. Modern hashing algorithms, such as Argon2, use randomized salt values of a specified length to ensure the proper randomization of hashed passwords at rest. The question remains, if ensuring proper entropy and randomization of hashing salts is considered best practice for password protection and recommended by the OWASP Top 10, why are best practices surrounding credential generation, randomization, and duration of validity left seemingly ad hoc?
What Are Dynamic Secrets?
Dynamic Secrets are the idea of generating credentials as needed, at the moment they are required. The concept is that credentials are issued in a randomized manner, with enough entropy that they cannot be predetermined and with enough frequency to reduce the risk of credentials being leveraged as an initial attack vector if they are leaked due to the fact that they should be rotated before the opportunity to leverage them in an attack arises. This is a common practice among Cloud platforms for Just in Time or Temporary Security credentials which are largely used for Privileged Access Management, as part of Change Management or a Break Glass procedure. Typically, a human user is given temporary local admin access to affect a task directly on an application or platform. Even though dynamic secrets are infrequently implemented on a larger scale, this principle can also be leveraged to ensure an automated secrets lifecycle for regular machine-to-machine authentication to assure rotation and revocation at a rate that is adequately adapted to the speed of the infrastructure and the risk appetite of the company. Dynamic Secrets are not without complexity, so the feasibility of implementation must be regarded with a holistic approach with the assurance that the proper resources are in place to adequately manage such a program.
Considerations
In order to maximize the efficiency and security of dynamic Secrets, automation should be a primary consideration in retrieving, revoking, and generating credentials. While dynamics help mitigate the risk of credential leakage, secrets should still not be overtly exposed. By automating these processes in a secure manner, the risk of incorrectly storing the secret is reduced nor is it visible to human eyes.
Infrastructure Speed
By nature, pipelines and containers are leaky and Secrets are easily exposed with minimal effort. While there is no direct link, there appears to be a correlation between the ephemerality of the underlying infrastructure and the lack of confidentiality regarding sensitive information. For example, when leveraging Kubernetes, the Kubernetes native secret is simply base 64 encoded and placed on an unencrypted file. This leaves the Secret vulnerable, particularly if it is reused for a long duration, which is typically the case of a certificate, for example. This is susceptibility is further compounded by the highly complex, configurable nature of these technologies and a propensity for human error.
When implementing a Secrets Management program, it is also important to consider the speed of the underlying architecture to ensure that the Secret adequately secures the infrastructure that it is supporting. If the duration of the Secret is too long and the infrastructure too fast, the risk of credential leakage is arguably high. If the duration of the Secret is too short for slower infrastructure, there is a possibility of failed authentication or authorization.
Tooling
When implementing dynamic secrets, there are several technical possibilities depending on the underlying needs, infrastructure, and feasible overhead.
Cloud Native
All of the major cloud platforms are equipped with their own flavor of a key vault which can be programmatically accessed via APIs. These key vaults allow for access or restriction based on designated roles and identities, whether they are human or machine. Platform specific identities can then be given authorization for bilateral or unilateral communication, depending on the requirement and the permissions scoped based on the need of the workload. The advantage to this is that the identities are issued and managed by the platform, making it convenient and relatively straightforward to establish a Zero Trust architecture scoped to Role Based Access Control.
In order to make Cloud Native solutions dynamic, an additional Serverless function is required. The code is generally provided as Open Source by the cloud providers and listens either for an event or can be triggered via a pipeline. Secrets may be expired at a particular interval to ensure revocation after programmatic rotation. Unfortunately, the minimum duration of a Secret is not always clear within the documentation and rotating secrets at high frequency is not always possible due to the underlying infrastructure, and in some cases, rate limiting.
Cloud Agnostic
While there are limited possibilities on the market currently that cater to dynamic secrets, both Hashicorp Vault and Akeyless offer products that manage a wide breadth of use cases for multi and hybrid cloud. The benefit of independent secrets management tools, in addition to avoiding vendor lock-in, is the assurance that dynamic secrets are properly implemented and easily configurable. By leveraging a cloud agnostic solution, the frequency of rotation can be increased, which is a significant advantage when matching the speed of a Secrets Management program to an ephemeral ecosystem, such as containerized workloads, serverless, or CI/CD pipelines.
As with any solution, there are clearly some costs and potential overhead when using an external provider. Akeyless provides a SaaS solution with a 99.99% uptime, so the infrastructure overhead is minimal, but the criticality of the workload and the particular compliance regulations may dictate whether it is a feasible solution. Hashicorp Vault may be run entirely on a company’s own infrastructure, which can have associated costs with infrastructure, implementation, and upkeep, depending on the architecture and the resources, but which permits mitigation of uptime, single point of failure, and compliance concerns as the company is fully in control of the deployment, disaster recovery, and backup solutions. Additionally the self managed Vault product has the greatest number of dynamic Secrets Engines available, with the possibility of customizing plugins if the desired Secrets Engine is not available. If Vault is still the desired solution, but the overhead of running the infrastructure is too great, it is also possible to leverage the Hashicorp Cloud Platform which offers many of the Vault Secrets Engines and associated functionalities with a graduated SLA of 99.9%. So again, the criticality of the workload and the particular compliance regulations become considerations.
Risk Considerations
By incorporating security into the development process, the Protect cycle is shortened and remediation can be expedited. But, as with all Risk profiles, there is a balance between what is reasonable and what may be ideal. In order to fully assess the optimal frequency of Secret generation, there are multiple factors which must be taken into account.
The overall ability of teams to automate and maintain pipelines is a critical consideration. Regardless of whether a Cloud native solution or a Cloud agnostic solution is preferable, the ability to automate and integrate the chosen solution and guarantee that Secrets are not revealed to humans is fundamental. If Secrets are revealed to humans or can be easily retrieved, while the attack window may be reduced, it is still exploitable. Additionally, if the acquisition of a Secret is misconfigured, the Availability of a workload may be jeopardized.
Additionally, any required compliance frameworks must be taken into consideration. Out of an abundance of caution for Schrems II, sensitive data and keys may require storage outside of a Cloud native vault or for PCI-DSS or SOX, there may be a retention policy for Secret data. If there are too many iterations of a Secret, retrieving Secret data from a particular point in time may be increasingly complex and could incidentally be purged as a result.
Enterprise Maturity
As with any DevOps oriented process which embraces the integration of security, technology and tooling are not the only considerations. For successful adoption, people and the overall maturity must be addressed. While Dynamic Secrets are gaining traction, the complexity and lack of best practice guidance may hinder adoption. Companies should be reasonably certain that standardized processes, such as deployments, requiring Secrets are automated and can sustain the required I/O with a minimal reduction in service speed. It is also imperative that the Secrets integration process itself has been automated to ensure that rotated credentials can be acquired and integrated without jeopardizing availability.
Technical maturity aside, the responsiveness of teams to understand the benefits of dynamic secrets, despite the added overhead and complexity, is critical to a successful implementation of Dynamic Secrets, and a Secrets Management program more generally. Without an understanding of the benefits surrounding the adoption of new tooling and the possible consequences of not adopting new practices that considerably improve the security posture.
What is the magic solution
As always, it depends. With all new adoptions and program changes, success is determined by multiple factors. In the case of building a Secrets Management program which leverages dynamic credentials, while tooling is critical, automation and mindset are equally important in the success of the program. The first step in moving to dynamics is an objective assessment of capabilities, requirements, and underlying infrastructure. Once these requirements have been assessed and an ideal future state identified, it may be beneficial to instantiate a gradual migration towards dynamics by first ensuring proper secrets hygiene and the implementation of a Secrets Management program.
