@ -163,9 +163,9 @@ TIP: The https://github.com/pivotal-cf/java-cfenv/[Java CFEnv] project is a bett
[[cloud-deployment-kubernetes]]
[[cloud-deployment-kubernetes]]
=== Kubernetes
=== Kubernetes
Spring Boot auto-detects Kubernetes deployment environments by checking the environment for `"*_SERVICE_HOST"` and `"*_SERVICE_PORT"` variables.
Spring Boot auto-detects Kubernetes deployment environments by checking the environment for `"*_SERVICE_HOST"` and `"*_SERVICE_PORT"` variables.
You can override this detection with the configprop:management.health.probes.enabled[] configuration property.
You can override this detection with the configprop:spring.main.cloud-platform[] configuration property.
Spring Boot helps you to <<spring-boot-features.adoc#boot-features-application-availability-state,manage the state of your application>> and export it with <<production-ready-features.adoc#production-ready-kubernetes-probes, HTTP Kubernetes Probes using Actuator>>.
Spring Boot helps you to <<spring-boot-features.adoc#boot-features-application-availability,manage the state of your application>> and export it with <<production-ready-features.adoc#production-ready-kubernetes-probes, HTTP Kubernetes Probes using Actuator>>.
@ -876,8 +876,8 @@ TIP: You can use `@Qualifier("groupname")` if you need to register custom `Statu
Applications deployed on Kubernetes can provide information about their internal state with https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes[Container Probes].
Applications deployed on Kubernetes can provide information about their internal state with https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes[Container Probes].
Depending on https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/[your Kubernetes configuration], the kubelet will call those probes and react to the result.
Depending on https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/[your Kubernetes configuration], the kubelet will call those probes and react to the result.
Spring Boot manages your <<spring-boot-features.adoc#boot-features-application-availability-state,Application Availability State>> out-of-the-box.
Spring Boot manages your <<spring-boot-features.adoc#boot-features-application-availability,Application Availability State>> out-of-the-box.
If deployed in a Kubernetes environment, Actuator will gather the "Liveness" and "Readiness" information from the `ApplicationAvailabilityProvider` and use that information in dedicated <<production-ready-health-indicators,Health Indicators>>: `LivenessProbeHealthIndicator` and `ReadinessProbeHealthIndicator`.
If deployed in a Kubernetes environment, actuator will gather the "Liveness" and "Readiness" information from the `ApplicationAvailability` interface and use that information in dedicated <<production-ready-health-indicators,Health Indicators>>: `LivenessStateHealthIndicator` and `ReadinessStateHealthIndicator`.
These indicators will be shown on the global health endpoint (`"/actuator/health"`).
These indicators will be shown on the global health endpoint (`"/actuator/health"`).
They will also be exposed as separate HTTP Probes using <<production-ready-health-groups, Health Groups>>: `"/actuator/health/liveness"` and `"/actuator/health/readiness"`.
They will also be exposed as separate HTTP Probes using <<production-ready-health-groups, Health Groups>>: `"/actuator/health/liveness"` and `"/actuator/health/readiness"`.
@ -904,33 +904,37 @@ NOTE: If an application takes longer to start than the configured liveness perio
The `"startupProbe"` is not necessarily needed here as the `"readinessProbe"` fails until all startup tasks are done, see <<production-ready-features.adoc#production-ready-kubernetes-probes-lifecycle,how Probes behave during the application lifecycle>>.
The `"startupProbe"` is not necessarily needed here as the `"readinessProbe"` fails until all startup tasks are done, see <<production-ready-features.adoc#production-ready-kubernetes-probes-lifecycle,how Probes behave during the application lifecycle>>.
WARNING: If your Actuator endpoints are deployed on a separate management context, be aware that endpoints are then not using the same web infrastructure (port, connection pools, framework components) as the main application.
WARNING: If your Actuator endpoints are deployed on a separate management context, be aware that endpoints are then not using the same web infrastructure (port, connection pools, framework components) as the main application.
In this case, a Probe check could be successful even if the main application does not work properly (for example, it cannot accept new connections).
In this case, a probe check could be successful even if the main application does not work properly (for example, it cannot accept new connections).
==== Checking external state with Kubernetes Probes
==== Checking external state with Kubernetes Probes
Actuator configures the "liveness" and "readiness" Probes as Health Groups; this means that all the <<production-ready-health-groups, Health Groups features>> are available for them.
Actuator configures the "liveness" and "readiness" probes as Health Groups; this means that all the <<production-ready-health-groups, Health Groups features>> are available for them.
You can, for example, configure additional Health Indicators:
You can, for example, configure additional Health Indicators:
By default, Spring Boot does not add other Health Indicators to these groups.
By default, Spring Boot does not add other Health Indicators to these groups.
The "liveness" Probe should not depend on health checks for external systems.
The "`liveness`" Probe should not depend on health checks for external systems.
If the <<spring-boot-features.adoc#boot-features-application-availability-liveness,Liveness State of an application>> is broken, Kubernetes will try to solve that problem by restarting the application instance.
If the <<spring-boot-features.adoc#boot-features-application-availability-liveness-state,Liveness State of an application>> is broken, Kubernetes will try to solve that problem by restarting the application instance.
This means that if an external system fails (e.g. a database, a Web API, an external cache), Kubernetes might restart all application instances and create cascading failures.
This means that if an external system fails (e.g. a database, a Web API, an external cache), Kubernetes might restart all application instances and create cascading failures.
As for the "readiness" Probe, the choice of checking external systems must be made carefully by the application developers, i.e. Spring Boot does not include any additional health checks in the readiness probe.
As for the "`readiness`" Probe, the choice of checking external systems must be made carefully by the application developers, i.e. Spring Boot does not include any additional health checks in the readiness probe.
If the <<spring-boot-features.adoc#boot-features-application-availability-readiness,Readiness State of an application instance>> is unready, Kubernetes will not route traffic to that instance.
If the <<spring-boot-features.adoc#boot-features-application-availability-readiness-state,Readiness State of an application instance>> is unready, Kubernetes will not route traffic to that instance.
Some external systems might not be shared by application instances, in which case they could quite naturally be included in a readiness probe.
Some external systems might not be shared by application instances, in which case they could quite naturally be included in a readiness probe.
Other external systems might not be essential to the application (the application could have circuit breakers and fallbacks), in which case they definitely should not be included.
Other external systems might not be essential to the application (the application could have circuit breakers and fallbacks), in which case they definitely should not be included.
Unfortunately, an external system that is shared by all application instances is common, and you have to make a judgement call: include it in the readiness probe and expect that the application is taken out of service when the external service is down, or leave it out and deal with failures higher up the stack, e.g. using a circuit breaker in the caller.
Unfortunately, an external system that is shared by all application instances is common, and you have to make a judgement call: include it in the readiness probe and expect that the application is taken out of service when the external service is down, or leave it out and deal with failures higher up the stack, e.g. using a circuit breaker in the caller.
NOTE: If all instances of an application are unready, a Kubernetes Service with `type=ClusterIP` or `NodePort` will not accept any incoming connections. There is no HTTP error response (503 etc.) since there is no connection. A Service with `type=LoadBalancer` might or might not accept connections, depending on the provider. A Service that has an explicit https://kubernetes.io/docs/concepts/services-networking/ingress/[Ingress] will also respond in a way that depends on the implementation - the ingress service itself will have to decide how to handle the "connection refused" from downstream. HTTP 503 is quite likely in the case of both load balancer and ingress.
NOTE: If all instances of an application are unready, a Kubernetes Service with `type=ClusterIP` or `NodePort` will not accept any incoming connections.
There is no HTTP error response (503 etc.) since there is no connection.
A Service with `type=LoadBalancer` might or might not accept connections, depending on the provider.
A Service that has an explicit https://kubernetes.io/docs/concepts/services-networking/ingress/[Ingress] will also respond in a way that depends on the implementation - the ingress service itself will have to decide how to handle the "connection refused" from downstream.
HTTP 503 is quite likely in the case of both load balancer and ingress.
Also, if an application is using Kubernetes https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/[autoscaling] it may react differently to applications being taken out of the load-balancer, depending on its autoscaler configuration.
Also, if an application is using Kubernetes https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/[autoscaling] it may react differently to applications being taken out of the load-balancer, depending on its autoscaler configuration.
@ -948,18 +952,18 @@ When a Spring Boot application starts:
|Application startup phase |Liveness State |Readiness State |Notes
|Application startup phase |Liveness State |Readiness State |Notes
|Starting
|Starting
|broken
|`BROKEN`
|unready
|`REFUSING_TRAFFIC`
|Kubernetes checks the "liveness" Probe and restarts the application if it takes too long.
|Kubernetes checks the "liveness" Probe and restarts the application if it takes too long.
|Started
|Started
|live
|`CORRECT`
|unready
|`REFUSING_TRAFFIC`
|The application context is refreshed. The application performs startup tasks and does not receive traffic yet.
|The application context is refreshed. The application performs startup tasks and does not receive traffic yet.
|Ready
|Ready
|live
|`CORRECT`
|ready
|`ACCEPTING_TRAFFIC`
|Startup tasks are finished. The application is receiving traffic.
|Startup tasks are finished. The application is receiving traffic.
@ -193,31 +193,34 @@ See the {spring-boot-module-api}/builder/SpringApplicationBuilder.html[`SpringAp
[[boot-features-application-availability-state]]
[[boot-features-application-availability]]
=== Application Availability State
=== Application Availability
When deployed on plaftorms, applications can provide information about their availability to the platform using infrastructure like https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/[Kubernetes Probes].
When deployed on platforms, applications can provide information about their availability to the platform using infrastructure such as https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/[Kubernetes Probes].
Spring Boot manages this application state with the `ApplicationAvailabilityProvider` and makes it available to application components and the platform itself.
Spring Boot includes out-of-the box support for the commonly used "`liveness`" and "`readiness`" availability states.
If you are using Spring Boot's "`actuator`" support then these states are exposed as health endpoint groups.
In addition, you can also obtain availability states by injecting the `ApplicationAvailability` interface into your own beans.
The "Liveness" state of an application tells whether its internal state allows it to work correctly, or recover by itself if it's currently failing.
The "`Liveness`" state of an application tells whether its internal state allows it to work correctly, or recover by itself if it's currently failing.
An invalid "Liveness" state means that the application is in a broken state and cannot recover from it; the infrastructure should then restart the application to mitigate that problem.
A broken "`Liveness`" state means that the application is in a state that it cannot recover from, and the infrastructure should restart the application.
NOTE: In general, the "Liveness" state should not be based on external checks, such as <<production-ready-features.adoc#production-ready-health, Health checks>>.
NOTE: In general, the "Liveness" state should not be based on external checks, such as <<production-ready-features.adoc#production-ready-health, Health checks>>.
If it did, a failing external system (a database, a Web API, an external cache) would trigger massive restarts and cascading failures across the platform.
If it did, a failing external system (a database, a Web API, an external cache) would trigger massive restarts and cascading failures across the platform.
The internal state of Spring Boot applications is mostly represented by the Spring application context.
The internal state of Spring Boot applications is mostly represented by the Spring `ApplicationContext`.
If the application context has started successfully, Spring Boot assumes that the application is in a valid state.
If the application context has started successfully, Spring Boot assumes that the application is in a valid state.
An application is considered live as soon as the context has been refreshed, see <<boot-features-application-events-and-listeners, Spring Boot application lifecycle and related Application Events>>.
An application is considered live as soon as the context has been refreshed, see <<boot-features-application-events-and-listeners, Spring Boot application lifecycle and related Application Events>>.
The "Readiness" state of an application tells whether the application is ready to handle traffic.
The "`Readiness`" state of an application tells whether the application is ready to handle traffic.
A failing "Readiness" state tells the platform that it should not route traffic to the application for now.
A failing "`Readiness`" state tells the platform that it should not route traffic to the application for now.
This typically happens during startup, while `CommandLineRunner` and `ApplicationRunner` components are being processed, or at any time if the application decides that it's too busy for additional traffic.
This typically happens during startup, while `CommandLineRunner` and `ApplicationRunner` components are being processed, or at any time if the application decides that it's too busy for additional traffic.
An application is considered ready as soon as application and command-line runners have been called, see <<boot-features-application-events-and-listeners, Spring Boot application lifecycle and related Application Events>>.
An application is considered ready as soon as application and command-line runners have been called, see <<boot-features-application-events-and-listeners, Spring Boot application lifecycle and related Application Events>>.
@ -226,9 +229,9 @@ TIP: Tasks expected to run during startup should be executed by `CommandLineRunn
Application components can retrieve the current availability state at any time, by injecting the `ApplicationAvailabilityProvider` and calling methods on it.
Application components can retrieve the current availability state at any time, by injecting the `ApplicationAvailability` interface and calling methods on it.
More often, applications will want to listen to state updates or update the state of the application.
More often, applications will want to listen to state updates or update the state of the application.
For example, we can export the "Readiness" state of the application to a file so that a Kubernetes "exec Probe" can look at this file:
For example, we can export the "Readiness" state of the application to a file so that a Kubernetes "exec Probe" can look at this file:
@ -236,15 +239,15 @@ For example, we can export the "Readiness" state of the application to a file so
[source,java,indent=0]
[source,java,indent=0]
----
----
@Component
@Component
public class ReadinessStateExporter implements ApplicationListener<ReadinessStateChangedEvent> {
public class ReadinessStateExporter {
@Override
@EventListener
public void onApplicationEvent(ReadinessStateChangedEvent event) {
public void onStateChange(AvailabilityChangeEvent<ReadinessState> event) {
switch (event.getReadinessState().getStatus()) {
switch (event.getState()) {
case READY:
case ACCEPTING_TRAFFIC:
// create file /tmp/healthy
// create file /tmp/healthy
break;
break;
case UNREADY:
case REFUSING_TRAFFIC:
// remove file /tmp/healthy
// remove file /tmp/healthy
break;
break;
}
}
@ -271,7 +274,7 @@ We can also update the state of the application, when the application breaks and
@ -282,6 +285,7 @@ Spring Boot provides <<production-ready-features.adoc#production-ready-kubernete
You can get more guidance about <<deployment.adoc#cloud-deployment-kubernetes,deploying Spring Boot applications on Kubernetes in the dedicated section>>.
You can get more guidance about <<deployment.adoc#cloud-deployment-kubernetes,deploying Spring Boot applications on Kubernetes in the dedicated section>>.
In addition to the usual Spring Framework events, such as {spring-framework-api}/context/event/ContextRefreshedEvent.html[`ContextRefreshedEvent`], a `SpringApplication` sends some additional application events.
In addition to the usual Spring Framework events, such as {spring-framework-api}/context/event/ContextRefreshedEvent.html[`ContextRefreshedEvent`], a `SpringApplication` sends some additional application events.
@ -307,9 +311,9 @@ Application events are sent in the following order, as your application runs:
. An `ApplicationContextInitializedEvent` is sent when the `ApplicationContext` is prepared and ApplicationContextInitializers have been called but before any bean definitions are loaded.
. An `ApplicationContextInitializedEvent` is sent when the `ApplicationContext` is prepared and ApplicationContextInitializers have been called but before any bean definitions are loaded.
. An `ApplicationPreparedEvent` is sent just before the refresh is started but after bean definitions have been loaded.
. An `ApplicationPreparedEvent` is sent just before the refresh is started but after bean definitions have been loaded.
. An `ApplicationStartedEvent` is sent after the context has been refreshed but before any application and command-line runners have been called.
. An `ApplicationStartedEvent` is sent after the context has been refreshed but before any application and command-line runners have been called.
. An `LivenessStateChangedEvent` is sent right after to indicate that the application is considered as live.
. An `AvailabilityChangeEvent` is sent right after with `LivenessState.CORRECT` to indicate that the application is considered as live.
. An `ApplicationReadyEvent` is sent after any application and command-line runners have been called.
. An `ApplicationReadyEvent` is sent after any application and command-line runners have been called.
. An `ReadinessStateChangedEvent` is sent right after to indicate that the application is ready to service requests.
. An `LivenessState` is sent right after with `ReadinessState.ACCEPTING_TRAFFIC` to indicate that the application is ready to service requests.
. An `ApplicationFailedEvent` is sent if there is an exception on startup.
. An `ApplicationFailedEvent` is sent if there is an exception on startup.
The above list only includes ``SpringApplicationEvent``s that are tied to a `SpringApplication`.
The above list only includes ``SpringApplicationEvent``s that are tied to a `SpringApplication`.