mex-docs

Autoscale Testing for TEF

This document provides the steps necessary to perform autoscale testing on the MobiledgeX platform.

Requirements

1. You have a copy of the mexload scripts; this is in the form of a tarball.
2. You have the ability to deploy to the cloudlets you wish to test.

General Setup

1. Download the mexload tarball to your workstation. This can be downloaded from This Link
2. Extract the files.

Deploy the API Server

1. Change to the apiserver/k8 subdirectory.
2. Adjust the jaypiserver.yaml file as desired for your testing.
3. Using either mcctl or the web gui, create an autoscale policy. A sample policy is shown below.
   • The sample policy calls for a minimum of 1 node and a maximum of 4 nodes.
   • The cluster will be scaled when the cpu usage exceeds 15% for 30 seconds or more.
   • The cluster will be scaled down when the cpu usage is below 5% for 30 seconds or more.
4. Using either mcctl or the web gui, create a cluster to be used by the application instances. A sample cluster configuration is shown below.
   • We are using the m4.small flavor in order to ensure that we reach a load that causes an autoscale trigger.
   • Both types of IP access can be used; this sample uses Shared.
   • Be sure you attach your policy to the cluster.
5. Using either mcctl or the web gui, create your application definition. A sample application configuration is shown below.
   • Ensure that you set scalewithcluster to true.
   • TLS should be set to true
6. Using either mcctl or the web gui, create your application instance.
   • Use the application definition defined above.
   • Use the cluster you created above.
7. Validate that you are able to reach the application instance.
   • Check the uri and the public port allocated to the application.
   • You should be able to curl https://uri:publicport/posts

Sample Policy

- key:
    organization: <yourorg>
    name: <yourpolicyname>
  minnodes: 1
  maxnodes: 4
  scaleupcputhresh: 15
  scaledowncputhresh: 5
  triggertimesec: 30

Sample Cluster

- key:
    clusterkey:
      name: <yourcluster>
    cloudletkey:
      organization: <youroperator>
      name: <yourcloudlet>
    organization: <yourorg>
  flavor:
    name: m4.small
  ipaccess: IpAccessShared
  nodeflavor: m4.small
  deployment: kubernetes
  nummasters: 1
  numnodes: 1
  autoscalepolicy: <yourpolicyname>

Sample Application

- key:
    organization: <yourorg>
    name: <yourappname>
    version: "<yourappversion>"
  imagetype: ImageTypeDocker
  accessports: tcp:3000:tls
  defaultflavor:
    name: m4.small
  deployment: kubernetes
  deploymentmanifest: "apiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: jaypiserver\n
    \ labels:\n    app: jaypiserver\nspec:\n  selector:\n    matchLabels:\n      app:
    jaypiserver\n  replicas: 1 \n  template:\n    metadata:\n      labels:\n        app:
    jaypiserver\n    spec:\n      containers:\n      - name: jaypiserver\n        image:
    qdzlug/jaypiserver\n        ports:\n        - containerPort: 3000\n---\napiVersion:
    v1\nkind: Service\nmetadata:\n  name: jaypiserver\nspec:\n  selector:\n    app:
    jaypiserver\n  ports:\n    - protocol: TCP\n      port: 3000\n      targetPort:
    3000\n  type: LoadBalancer"
  scalewithcluster: true
  accesstype: AccessTypeLoadBalancer

Deploy the Load Generator

The load generator is intended to be run as a docker image; once built it can be used on multiple machines to generate load from different clients. There is a detailed README.md file in the loadgen directory that explains the steps required in addition to the information here.

1. Change to the loadgen directory.
2. Download the vegeta load testing utility (linux/amd64) from the vegeta releases page.
3. Build the docker image and tag it as loadgen
   • docker build -t loadtest .
4. Push the image to the test workstations/servers.

Generate Load

In testing with the m4.small based clusters, using 2-4 load generators managed to create sufficient load to observe scaling. The container takes three variables to help you tune and direct the load generations. These are:

• THERATE which is how many transactions to push against the API every second.
• THEDURATION which is how long to run the test; this paramter requires that you provide the unit of time (so, 60s for a 60 second test).
• THEURL which is the full URL of the test, including the port and the path.

Most of the testing that I’ve performed has used the following command line.

$ docker run --env THERATE=600 --env THEDURATION=240s --env THEURL=https://URI:PUBLICPORT/posts  loadgen

Validation

1. You can watch the load against the backend either via the monitoring functionality in the web gui, or by using the metrics subcommands to mcctl.
2. You can see the current number of worker nodes for the cluster via either the web gui, or by using the region ShowClusterInst subcommand to mcctl.
3. Cluster scale events can be seen by using the events subcommand to mcctl. Note: As of this writing there is a change required for these events to show correctly that is currently in qa.