Skip to content

RichConsoleSpanExporter causes app to freeze #3254

Open
@randy-coburn-zeam

Description

Describe your environment

OS: (MacOS)
Python version: (3.12.4)
Package version: 0.51b0

What happened?

When using the BatchSpanProcessor with RichConsoleSpanExporter, my application stops at the end of the code path and hangs.
I have left the code to run for several minutes and it just never prints out the spans to the console.

This does not happen when using the far simpler ConsoleSpanExporter.

I originally wrote this to learn a bit of tracing via python but got stuck pretty quickly when the application would not exit.

I have included some code that I have put together to reproduce the issue pretty reliably for me.

By switching the below lines you can see the error:

trace_provider.add_span_processor(rich_processor)
# trace_provider.add_span_processor(console_processor)

rich_processor runs forever, console_process exits as expected although gives sad tracing.

This is the stack trace I see when I ctl+c the app.

Exception ignored in atexit callback: <bound method TracerProvider.shutdown of <opentelemetry.sdk.trace.TracerProvider object at 0x105b2fcb0>>
Traceback (most recent call last):
  File "/Volumes/vera_work/paytec/repos/stellar-swap-quotes/venv/lib/python3.12/site-packages/opentelemetry/sdk/trace/__init__.py", line 1282, in shutdown
    self._active_span_processor.shutdown()
  File "/Volumes/vera_work/paytec/repos/stellar-swap-quotes/venv/lib/python3.12/site-packages/opentelemetry/sdk/trace/__init__.py", line 180, in shutdown
    sp.shutdown()
  File "/Volumes/vera_work/paytec/repos/stellar-swap-quotes/venv/lib/python3.12/site-packages/opentelemetry/sdk/trace/export/__init__.py", line 403, in shutdown
    self.worker_thread.join()
  File "/Users/randyc/.pyenv/versions/3.12.4/lib/python3.12/threading.py", line 1147, in join
    self._wait_for_tstate_lock()
  File "/Users/randyc/.pyenv/versions/3.12.4/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):

Steps to Reproduce

Run this code and change the trace_provider

import os
import logging
import time

# Traces
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.exporter.richconsole import RichConsoleSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

# Logging
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import SimpleLogRecordProcessor, ConsoleLogExporter
from opentelemetry._logs import set_logger_provider

# Tracing
# Create Tracer and set it up
APP_SERVICE_NAME = "otel-test"
trace_provider = TracerProvider(
    resource=Resource.create({SERVICE_NAME: APP_SERVICE_NAME})
)
# Get a span_processor if you want to use the spans locally
# in our case its console logging.
rich_processor = BatchSpanProcessor(
    RichConsoleSpanExporter(
        service_name=APP_SERVICE_NAME,
    )
)

console_processor = BatchSpanProcessor(
    ConsoleSpanExporter(service_name=APP_SERVICE_NAME)
)

# Add the span_processor to the tracer
############################################
# Add the rich_processor to the trace_provider to make the app hang.
# Add the console_processor to the trace_provider to make the app work.
############################################
trace_provider.add_span_processor(rich_processor)
# trace_provider.add_span_processor(console_processor)

trace.set_tracer_provider(trace_provider)

tracer = trace.get_tracer(__name__)

logger_provider = LoggerProvider(
    resource=Resource.create(
        {
            SERVICE_NAME: APP_SERVICE_NAME,
            "service.instance.id": os.uname().nodename,
        }
    ),
)
set_logger_provider(logger_provider)
log_exporter = ConsoleLogExporter()
log_record_processor = SimpleLogRecordProcessor(log_exporter)
logger_provider.add_log_record_processor(log_record_processor)
otel_log_handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)


def main():
    logger = logging.getLogger()
    logger.addHandler(otel_log_handler)
    logger.setLevel(logging.INFO)

    with tracer.start_as_current_span("testies"):
        logging.info("Hello, World!")
        for i in range(10):
            with tracer.start_as_current_span(f"testies-{i}"):
                logging.info(f"Hello, World! {i}")
        logging.info("Sleeping!")
        time.sleep(10)
        logging.info("Goodbye, World!")

    ##################
    # App hangs here #
    ##################

    # Sometimes if you force flush it works although very unreliable.
    # rich_processor.force_flush()
    # Calling shutdown worked at some point but is also unreliable.
    # rich_processor.shutdown()
    logging.info("Done!")


if __name__ == "__main__":
    main()

Expected Result

The application should exit without input required from the user and display the spans collected.

Actual Result

Application freeze indefinitely

Additional context

Same result in python 3.13 and 3.12.

Lib versions

pip freeze | grep -E "opentelemetry|rich"
opentelemetry-api==1.30.0
opentelemetry-exporter-otlp==1.30.0
opentelemetry-exporter-otlp-proto-common==1.30.0
opentelemetry-exporter-otlp-proto-grpc==1.30.0
opentelemetry-exporter-otlp-proto-http==1.30.0
opentelemetry-exporter-richconsole==0.51b0
opentelemetry-instrumentation==0.51b0
opentelemetry-instrumentation-logging==0.51b0
opentelemetry-proto==1.30.0
opentelemetry-sdk==1.30.0
opentelemetry-semantic-conventions==0.51b0
rich==13.9.4

Would you like to implement a fix?

None

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions