Introduction#

This note shows how to getting started with aws distro for opentelemetry (ADOT)

  • run an simple flask app
  • setup adot on ec2
  • send logs and metrics to standard output
  • send logs and metrics to cloudwatch

Application#

Create a simple flask app. Here is app.py content

from random import randint
from flask import Flask, request
import logging


app = Flask(__name__)
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)




@app.route("/rolldice")
def roll_dice():
    player = request.args.get('player', default=None, type=str)
    result = str(roll())
    if player:
        logger.warning("%s is rolling the dice: %s", player, result)
    else:
        logger.warning("Anonymous player is rolling the dice: %s", result)
    return result




def roll():
    return randint(1, 6)

ADOT Collector#

Download the app

wget https://aws-otel-collector.s3.amazonaws.com/amazon_linux/amd64/latest/aws-otel-collector.rpm

Install the app

sudo rpm -Uvh  ./aws-otel-collector.rpm

Start the ADOT

sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -c </path/config.yaml> -a start

the config.yaml is optional

sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -a start

Stop

sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl  -a stop

Start with config.yaml

sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl -c config.yaml -a start

Check status

sudo /opt/aws/aws-otel-collector/bin/aws-otel-collector-ctl  -a status

Monitor Application#

First, active the python environment.

source .env/bin/activate

Second, run the application and export logs, metrics to standard output.

OTEL_PYTHON_DISTRO="aws_distro"\
OTEL_PYTHON_CONFIGURATOR="aws_configurator"\
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true\
opentelemetry-instrument\
    --traces_exporter console\
    --metrics_exporter console\
    --logs_exporter console,otlp\
    --service_name dice-server\
    flask run -p 8080

Third, send some request to the falsk app. Observe logs and metrics in standard output and cloudwatch logs.

curl http://127.0.0.1:8080/rolldice

Finally, run the application and export logs, metrics to standard cloudwatch.

OTEL_PYTHON_DISTRO="aws_distro"\
OTEL_PYTHON_CONFIGURATOR="aws_configurator"\
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true\
opentelemetry-instrument\
    --logs_exporter console,otlp\
    --service_name dice-server\
    flask run -p 8080

In the aws cloudwatch logs search for dice-server and testing-logs-11 log group names. Here is a sample dice-server metric.

{
  "OTelLib": "opentelemetry.instrumentation.system_metrics",
  "Version": "1",
  "_aws": {
    "CloudWatchMetrics": [
      {
        "Namespace": "dice-server",
        "Dimensions": [
          ["state", "cpu", "OTelLib"],
          ["OTelLib"],
          ["OTelLib", "state"],
          ["OTelLib", "cpu"]
        ],
        "Metrics": [
          {
            "Name": "system.cpu.utilization",
            "Unit": "1"
          }
        ]
      }
    ],
    "Timestamp": 1715742885287
  },
  "cpu": "3",
  "state": "system",
  "system.cpu.utilization": 0
}

Here is a sample of log testing-logs-11

{
  "body": "Configuration of aws_configurator not loaded, configurator already loaded",
  "severity_number": 13,
  "severity_text": "WARNING",
  "scope": {
    "name": "opentelemetry.sdk._logs._internal"
  },
  "resource": {
    "service.name": "dice-server",
    "telemetry.auto.version": "0.43b0",
    "telemetry.sdk.language": "python",
    "telemetry.sdk.name": "opentelemetry",
    "telemetry.sdk.version": "1.22.0"
  }
}

ADOT Collector Config#

The default config.yaml located at

extensions:
  health_check:


receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  awsxray:
    endpoint: 0.0.0.0:2000
    transport: udp


processors:
  batch/traces:
    timeout: 1s
    send_batch_size: 50
  batch/metrics:
    timeout: 60s
  batch/logs:
    timeout: 60s


exporters:
  awsxray:
  awsemf:
  awscloudwatchlogs:
    log_group_name: 'testing-logs-11'
    log_stream_name: 'testing-integrations-stream-11'


service:
  pipelines:
    traces:
      receivers: [otlp, awsxray]
      processors: [batch/traces]
      exporters: [awsxray]
    metrics:
      receivers: [otlp]
      processors: [batch/metrics]
      exporters: [awsemf]
    logs:
      receivers: [otlp]
      processors: [batch/logs]
      exporters: [awscloudwatchlogs]


  extensions: [health_check]

Troubleshooting#

Run application sample

docker run --rm -it -e "OTEL_OTLP_ENDPOINT=172.17.0.1:4317" -e "otlp_instance_id=test_insance_rpm" -e "OTEL_RESOURCE_ATTRIBUTES=service.namespace=ADOTCollectorRPMDemo,service.name=ADOTCollectorRPMDemoService" -e S3_REGION=us-west-2 aottestbed/aws-otel-collector-sample-app:java-0.1.0

Just test

OTEL_PYTHON_DISTRO="aws_distro" \
OTEL_PYTHON_CONFIGURATOR="aws_configurator" \
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \
opentelemetry-instrument \
    --service_name dice-server \
    flask run -p 8080

Reference#