Helm chart setup workflow
- Initially get things working with a default helm install into dev.
- Does it have ldap or oidc integration or some other reason to need to verify onprem CA chain? Yes, figure out how to get CA chain installed
- Setup oidc if possible, setup ldap if needed and oidc is not available.
- Is it possible to set values in helm chart to get CA chain installed as part of the helm install? Yes, modify yaml, no fork helm install and add steps, see if project owner is OK with pull request or not. (better in Integrate with helm chart than manage a fork).
- Configure something on server you want to persist.
- Helm uninstall and reinstall, did you lose the setting? … figure out steps to get data to persist.
- Try to increase the storage size of PVCs which might need to be increased in production? Is this possible without taking down the application? Figure out what is required for this use case which will inevitably come up in the future. Document and be ready. It may be wise to implement a pipeline for this purpose.
- Cordon involved node(s) and drain, uncordon node(s), was data lost when things came back up? Figure out how to ensure data persists.
- Does server have a method to export configuration or otherwise backup the server. Configure automated backups.
- Is it possible to Configure server as HA? Can it be configured for HA later or must it be configured from initial setup? Can a single instance be migrated to HA. Decide if HA needs to be setup or is a single instance good enough. If HA is desired then figure out how to set that up and go through this list again.
- Are there options to configure metrics for the application? Often these exist in helm installs. (lower priority when initially working to get something up)
- If there is an option to use a log aggregator set that up or possibly setup a sidecar with logging. (lower priority when initially working to get something up)
- Server is now ready to release into test.
- Configure permissions of those with accounts accessed via oidc / ldap. Note a program which supports ldap but not oidc is not as evolved. Check for a plugin/extension if oidc does not appear to be available. Oidc enables almost any identity provider & SSO, and is always preferred.
- Does the minimum requested CPU and Memory match what’s actually needed?
- Someone needs to perform some manual testing or work on automated testing.
- If no one ever tries restoring from a backup, there is a good chance the process might not work, might want to try that out before there is a fire.
- No system may be released into production without an automated method of registering its ip in dns (e.g. external-dns) and also an automated method of updating its ssl certificates (e.g. cert-man), verify these work.
- Be sure to test rolling up to the next release of a helm chart as well as rolling back (and all tests still pass).
- If all testing passes then ready for production.
- An update strategy needs to be established and followed just prior to release into production. Schedule: Monthly, quarterly, every 6 months, or upon release of a new version. Version: always run the latest, or version just prior to latest major release (and with all the updates). Some programs such as WordPress can/will update plugins automatically… is this ok?
- Generally automation is desired to roll something out into production. When an update is ready automation should be used to update first in dev, perform automated testing, then roll out into dev with the ok of someone (or automatically rolled out into prod if all tests passed and its decided that is good enough).
- Also, a pipeline for rolling back to a previous version is a good idea, in case a deployment to production fails.