How Zerops scales Java
Zerops performs an automated scaling of hardware resources required to run your runtime application based on its usage. If the current use of your application does not require as much performance or disk space the auto scaling reduces the resources and thus reduces the costs. If your application is under heavy load or needs to store more data, then auto scaling increases the resources to make sure it runs smoothly.
Vertical and horizontal auto scaling
Each application you deploy starts with the minimum hardware resources: CPU cores, RAM and Disk. Zerops monitors the usage of these 3 resources and if the usage exceeds a set threshold, more CPU cores, RAM or Disk is allocated to the service. This is called vertical scaling.
Horizontal scaling adds or removes whole containers.
Zerops has a preference for vertical scaling because it's faster and more precise. If the vertical auto scaling hits the defined maximum a new container is started automatically. When your application doesn't need so much power and all containers are vertically scaled down, Zerops will gradually remove whole containers.
Configure auto scaling
To change the auto scaling settings go to the Java service detail and choose Automatic scaling configuration in the left menu.
CPU mode
Shared
Your application gets a full physical CPU core, but it is shared with up to 10 other applications. In this mode the power your application gets is depended on other applications running on the same CPU core. At best case scenario your application gets 10/10 of CPU core power and 1/10 at worst case scenario.
Dedicated
The CPU core is dedicated to your application.
See the pricing for the difference between CPU modes.
The CPU mode doesn't change automatically.
Vertical auto scaling
Vertical auto scaling has following default configuration:
| Minimum resource | Maximum resource | |
|---|---|---|
| CPU cores | 1 | 5 |
| RAM | 0.25 GB | 32 GB |
| Disk | 1 GB | 100 GB |
Java service always starts with the minimal resources.
For most cases, the default parameters will work without issues. If you need to limit the cost of the Java service, lower the maximal resources. Zerops will never scale above the selected maximums.
When you are experiencing problems with insufficient Java performance or capacity, increase the minimal resources. Zerops will never scale below the selected minimums.
Learn more about how to fine tune the auto scaling to fit your application needs.
Horizontal auto scaling
When a container needs more CPU or RAM but already consumes maximal resources defined for the vertical auto scaling, Zerops will add a new container to your Java service. When your Java service doesn't need so much power and all containers are vertically scaled down in such a way their CPU allocation is near the minimal resources, Zerops will gradually remove whole containers.
Horizontal auto scaling has following default configuration:
| minimum containers | 1 |
| maximum containers | 6 |
Java service always starts with the minimum number of containers.
You can increase the minimum or decrease the maximum number of containers. The horizontal scaling parameters can be changed later.
Single container vs. High Availability
When creating a new runtime service, you can choose the minimum and maximum number of containers. If you set the maximum limit to one, the service will be run in a single container and no horizontal scaling will occur.
If the single container fails, Zerops will start a new container and deploy your application automatically. The application won't be available for a short period. This mode is recommended for non-critical applications or dev environments.
By increasing the maximum number of containers for your service, you enable horizontal auto scaling. Zerops will then add containers depending on your application’s load. Application running on two or more containers is in High Availability mode, which we highly recommend for production. When the load drops, containers will be gradually removed to the defined minimum.
Each container of the same service is strictly installed on a different server. This prevents the temporary outage in case any of Zerops servers fail. If the connection to a container is broken, Zerops immediately redirects incoming traffic to other containers. A new container will be started automatically and the broken container will be deleted.
Check if your application is ready to be run in multiple containers.
Fine-tune the auto scaling
Advanced CPU settings
If you've experienced problems with not enough power when your application starts, increase the default Start CPU core count. Alternatively switch the CPU mode to dedicated to allocate the stable CPU power to your application.
If your application doesn't need so much power after it is started, Zerops will scale down the allocated CPU cores to the defined minimum.
You can disable the CPU vertical auto scaling by setting the minimum and maximum to the same value. This setting doesn't affect the RAM and Disk scaling. Zerops scales each hardware resource independently.
Advanced RAM settings
By default Zerops keeps a minimum free RAM in each container. This setting will ensure that most applications will run smoothly. Zerops monitors the minimum free RAM every 10 seconds.
But if your application need a more memory faster or if you have experienced problems with insufficient memory or even restarts due to Out Of Memory (OOM) errors, we recommend
- Increasing the minimum RAM for the auto scaling
- or increasing the minimum free RAM in GB
- or setting the minimum free RAM in % of the RAM assigned to the container
You can set the minimum free RAM both in GB and in percent, Zerops will apply the larger value based on the current RAM assigned to the container.
You can disable the RAM vertical auto scaling by setting the minimum and maximum to the same value. This setting doesn't affect the CPU core and Disk scaling. Zerops scales each hardware resource independently.
Technical details
Automatic scale up
Zerops monitors CPU, RAM and Disk usage in all running containers each 10 seconds. The scale up threshold is derived from following minimum free resources:
- 0.1 CPU core
- 0.125 GB RAM (You can fine tune this setting)
- 0.5 GB disk
If the minimum free CPU, RAM or disk usage of a container is lower than the defined scale up threshold, Zerops scales the container up.
The scale up of RAM or disk is immediate. The scale up of CPU is configured to be a little less aggressive. Two consecutive measurements of free CPU with values under the scale up threshold are required to trigger the scale up. This rule prevents excessive fluctuations of scaling up and down due to sudden changes in CPU usage.
The minimum step for the vertical scaling is
- 1 CPU core
- 0.25 GB RAM
- 0.5 GB disk
When the application is under a heavy load and needs to scale up faster, the scaling step will increase automatically.
Maximal resources are defined for each Java service. Zerops will never scale above the entered values. If your application is in [highly available mode], maximal resources are identical for all containers of the Java service.
Not enough resources to scale up
If one of the Java containers needs more resources but there are not enough of them on the underlying machine, a new container with the required hardware resources will be started on another machine. When the new container is ready, it will be added to the service balancer. The old container will be removed from the balancer and deleted.
Automatic scale down
When the application no longer needs as much power or disk space, each container is gradually scaled down to the defined minimum. The automatic scale down is configured to be more cautious and defensive to prevent the application from scaling up and down rapidly.
Consecutive measurements during:
- 1 minute for CPU
- 2 minutes for RAM
- 5 minutes for disk
with free resources safely above the minimum threshold are required to scale down the appropriate resource.
The minimum step for the scale down is identical to the minimum step for scale up. When several scale down events are triggered in a short period of time, the scaling step increases automatically.
Horizontal autoscaling
Zerops prefers vertical scaling over horizontal scaling because vertical scaling is faster and allows finer adjustment to the required performance. Horizontal scaling can be disabled by setting the same number for the minimum and maximum container count. Zerops will then scale the Java service only vertically.
Your application is created with the defined minimum number of containers. Zerops will add a new container when any of the service's containers reaches the maximum limit for vertical scaling for CPU cores or RAM. Zerops doesn't start a new container when the maximum disk space is reached. No more containers are added when the defined maximum container limit is reached.
The new container is started with a minimum disk size and with an average CPU cores and RAM of the existing containers.
By customising the vertical auto scaling limits, you can cause the horizontal scaling to start earlier. For example if you lower the vertical auto scaling maximum to 1 CPU core, Zerops will start a new container if some of the running containers are using the whole CPU core for more than 20 seconds.
If the application no longer needs as much power, Zerops will gradually remove containers to the defined minimum count. The container is removed after its CPU cores are scaled down to the defined minimum and the free CPU is safely above the minimum threshold for vertical scaling. Zerops only removes containers with a minimum 15 minute lifetime.
Monitor Java resources
Zerops provides information about how much hardware resources the Java service is currently using. Go to the service detail in Zerops GUI and select Service dashboard & runtime containers in the left menu. Zerops also provides the history of resource usage.