Skip to content

opentelemetry-exporter-prometheus - - Invalid Metric Labels #4418

Open
@jseiser

Description

Describe your environment

[packages]
opentelemetry-distro = "==0.49b2"
opentelemetry-exporter-otlp = "==1.28.2"
opentelemetry-exporter-prometheus = "==0.49b2"
opentelemetry-instrumentation-asgi = "==0.49b2"
opentelemetry-instrumentation-fastapi = "==0.49b2"
opentelemetry-instrumentation-httpx = "==0.49b2"
opentelemetry-instrumentation-jinja2 = "==0.49b2"
opentelemetry-instrumentation-logging = "==0.49b2"
opentelemetry-instrumentation-urllib = "==0.49b2"
opentelemetry-instrumentation-urllib3 = "==0.49b2"
opentelemetry-instrumentation-aiohttp-client = "==0.49b2"
fastapi = "*"
uvicorn = {extras = ["standard"]}

What happened?

The labels being assigned to the generated metrics, are wrong/out of order.

Steps to Reproduce

MVC: https://github.com/jseiser/otlp_fastapi_repl/tree/main

Expected Result

Correct metric labels

Actual Result

  1. Start the app
pipenv run opentelemetry-instrument uvicorn ui.main:app --proxy-headers --host=0.0.0.0 --port=5000 --log-level=error
Loading .env environment variables..
  1. Check Metrics - makes sense, no traffic has happened
 curl http://localhost:9464/metrics 
# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 2258.0
python_gc_objects_collected_total{generation="1"} 2915.0
python_gc_objects_collected_total{generation="2"} 60.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 170.0
python_gc_collections_total{generation="1"} 15.0
python_gc_collections_total{generation="2"} 1.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="11",patchlevel="10",version="3.11.10"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 4.3268096e+08
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 7.2364032e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.73869183682e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.66
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 18.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024.0
  1. Hit Site once
curl http://localhost:5000         
{"message":"Hello World"}
  1. Check Metrics - Looks Good - note http_target, and not_host_port.
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:5000",http_method="GET",http_scheme="http",http_server_name="localhost:5000",http_status_code="200",http_target="/",le="0.0",net_host_port="5000"} 0.0

http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:5000",http_method="GET",http_scheme="http",http_server_name="localhost:5000",http_status_code="200",http_target="/",le="0.0",net_host_port="5000"} 0.0
  1. Hit site a few more times
curl http://localhost:5000         
{"message":"Hello World"}% 
curl http://localhost:5000         
{"message":"Hello World"}% 
  1. Check Metrics - still good
  2. Hit something else
curl http://localhost:5000/fakepath
{"detail":"Not Found"}
  1. Check Metrics - Broken
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:5000",http_method="GET",http_scheme="http",http_server_name="localhost:5000",http_status_code="200",le="0.0",net_host_port="/"} 5.0                           

http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:5000",http_method="GET",http_scheme="http",http_server_name="localhost:5000",http_status_code="404",le="0.0",net_host_port="5000"} 1.0

http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:5000",http_method="GET",http_scheme="http",http_server_name="localhost:5000",http_status_code="404",le="0.0",net_host_port="5000"} 0.0

Additional context

No response

Would you like to implement a fix?

None

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions