⚠ This page is served via a proxy. Original site: https://github.com
This service does not collect credentials or authentication data.
Skip to content

Conversation

@ditadi
Copy link
Contributor

@ditadi ditadi commented Jan 12, 2026

Summary

  • Add comprehensive logging package with structured logging, standardized errors, and request-scoped wide-event tracking
  • Implement WideEvent class for collecting rich request metadata (method, path, user, component, execution, stream, error context)
  • Add configurable sampling with intelligent defaults (always sample errors, slow requests, and cache info)
  • Use AsyncLocalStorage for automatic context propagation across interceptors and nested calls

Details

Standardized Errors

New error hierarchy with AppKitError base class providing:

  • Error codes for programmatic handling (code, statusCode, isRetryable)
  • Specialized error types: ValidationError, AuthenticationError, ConfigurationError, ConnectionError, ExecutionError, InitializationError, ServerError, TunnelError
  • Automatic sensitive field redaction in toJSON() serialization
  • Factory methods for common patterns (e.g., ValidationError.missingField("warehouseId"))

Structured Logging

createLogger(scope) returns a logger with:

  • debug/info/warn/error levels with printf-style formatting
  • Request-aware overloads: logger.info(req, "Processing %s", id) tracks logs in WideEvent
  • logger.event(req?) returns the current WideEvent (from AsyncLocalStorage or explicit req)

Wide-Event Tracking (Honeycomb-style)

Request-scoped WideEvent collects:

  • Request metadata (method, path, status_code, duration_ms, trace_id)
  • Component info (plugin/connector name + operation)
  • Execution context (cache_hit, retry_attempts, timeout)
  • Stream metadata (events_sent, reconnections)
  • Error context (type, code, message, retriable)
  • Scoped context for plugin-specific data

Sampling

Configurable via APPKIT_SAMPLE_RATE env var with smart defaults:

  • Always sample: errors, status >= 400, duration >= 5s, cache info present
  • Deterministic sampling based on request ID hash
  • Path exclusions for health checks, static assets, etc.

@ditadi ditadi changed the title feat(observability): add standard errors on appkit feat(observability): add structured logging Jan 13, 2026
@ditadi ditadi marked this pull request as ready for review January 13, 2026 18:25
abstract readonly statusCode: number;

/** Whether this error type is generally safe to retry */
abstract readonly isRetryable: boolean;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see here we are using isRetryable, but then in the wide-event file we have this

  setError(error: Error): this {
    const isAppKitError = "code" in error && "statusCode" in error;
    const errorCause = (error as any).cause;

    this.data.error = {
      type: error.name,
      code: isAppKitError ? (error as any).code : "UNKNOWN_ERROR",
      message: error.message,
      retriable: isAppKitError ? (error as any).retriable : false,
      cause: errorCause ? String(errorCause) : undefined,
    };

    return this;
  }

shouldn't it be

      retriable: isAppKitError ? (error as any).isRetryable : false,


// Check path prefixes
for (const prefix of EXCLUDED_PATH_PREFIXES) {
if (cleanPath.startsWith(prefix) || cleanPath.includes(prefix)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could just be

Suggested change
if (cleanPath.startsWith(prefix) || cleanPath.includes(prefix)) {
if (cleanPath.includes(prefix)) {

no?

*
* @example
* ```typescript
* import { ValidationError, AuthenticationError } from "@databricks/appkit";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is what we want to do, we are missing all the error exports in the root index file 😄

}

// set in AsyncLocalStorage so downstream code can access via logger.event()
eventStorage.enterWith(wideEvent);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as talked in the call, let's make sure that we need to use enterWith instead of run 😄

type: error.name,
code: isAppKitError ? (error as any).code : "UNKNOWN_ERROR",
message: error.message,
retriable: isAppKitError ? (error as any).retriable : false,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
retriable: isAppKitError ? (error as any).retriable : false,
retriable: isAppKitError ? (error as any).isRetryable : false,

Copy link
Member

@pkosiec pkosiec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job overall! A few comments from my side

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General error comment: I think I mentioned that before, but I'm not a fan of centralizing e.g. errors which are used just once in a given package or scoped directory.
If they are used in multiple places (e.g. across different plugins), then sure, having them in a single central place makes sense. But for the package-specific ones, I'd rather keep them close to the usage (e.g. server plugin errors within the server dir) and just use the AppKitError abstract class.

It's not a merge blocker, just an opinion, perhaps it's worth to discuss our approach within the team.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question - have you researched existing logging solutions? There may be an already existing option that meets our requirements and would eliminate the need for separate maintenance 🤔

resource: this.createResource(config),
autoDetectResources: false,
sampler: new AlwaysOnSampler(),
sampler: new AppKitSampler(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please test a deployed app with OpenTelemetry enabled in Apps runtime? I think it would be great to double-check everything works flawlessly there 👍

From my observations sampling result might differ between local dev and runtime, that's why the AlwaysOnSampler was set before (same as previous default OpenTelemetry config).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants